Multimodal Representation Learning for Healthcare Data Analytics

From ISLAB/CAISR
Revision as of 23:16, 16 November 2026 by Cclab (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search
Title Multimodal Representation Learning for Healthcare Data Analytics
Summary Develop and evaluate multimodal representation learning methods to integrate heterogeneous healthcare data for improved clinical risk prediction.
Keywords Multimodal Learning, Healthcare Analytics, Representation Learning, Data Fusion, Deep LearningProperty "Keywords" has a restricted application area and cannot be used as annotation property by a user.
TimeFrame Spring 2027 / Autumn 2027
References [[References::[1] Xu, Z., So, D. R., & Dai, A. M. (2021, May). Mufasa: Multimodal fusion architecture search for electronic health records. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 12, pp. 10532-10540).

[2] Li, B., Du, B., & Ye, J. (2025). Prime: Pretraining for patient condition representation with irregular multimodal electronic health records. ACM Transactions on Knowledge Discovery from Data, 19(7), 1-39.]]

Prerequisites Basic knowledge of machine learning and deep learning, proficiency in Python, and experience with PyTorch or a similar framework. Familiarity with time-series analysis, representation learning, or healthcare data is an advantage.
Author
Supervisor Guojun Liang, Alexandra Wernersson
Level Master
Status Open


      1. Multimodal Representation Learning for Healthcare Data Analytics
    • Background and Research Question**

Healthcare data are inherently multimodal, comprising structured clinical records, longitudinal measurements, medical images, and clinical notes. These data sources provide complementary information about patient health, yet many existing predictive models rely on only a single modality.

This project aims to investigate multimodal representation learning methods for integrating heterogeneous healthcare data to improve clinical risk prediction. Depending on the available datasets and student interests, the thesis may focus on a specific challenge, such as cross-modal alignment, temporal representation learning, or multimodal information fusion.

The central research question is:

  • How can multimodal representation learning effectively integrate heterogeneous healthcare data to improve clinical prediction?*
    • Technical Requirements**

This is a software-oriented research project involving machine learning and deep learning. The implementation will primarily use Python and PyTorch. No specialized hardware development is required.

    • Work Packages**

1. **Literature Review and Problem Definition:** Review relevant approaches to multimodal representation learning and identify a focused research question based on the available healthcare datasets.

2. **Data Preparation and Baseline Implementation:** Explore and preprocess the selected multimodal healthcare dataset and implement suitable baseline methods.

3. **Model Development:** Investigate a selected representation learning approach, potentially involving cross-modal alignment, temporal modelling, attention-based fusion, or self-supervised learning.

4. **Experimental Evaluation:** Evaluate the proposed approach against baseline methods using relevant clinical prediction metrics and analyse its strengths and limitations.

    • Expected Outcomes**

The project is expected to deliver a literature review, a reproducible implementation, experimental results, and a Master's thesis. Depending on the research findings, the results may also contribute to future scientific publications.

    • Research Environment**

The project builds on ongoing CAISR Health research activities at Halmstad University and existing collaborations with Region Halland. Potential collaboration with industrial partners, including Novartis, may be explored depending on project scope and partner availability.