Yu Han

Modern healthcare generates vast streams of heterogeneous data, including physiological time series, biomedical images, and clinical narratives. Unlocking the full prognostic and therapeutic value of this multimodal information requires foundation models that can generalize across tasks, modalities, and patient populations. However, current multimodal representation learning faces persistent bottlenecks: uncertainty in token embedding, fragility of augmentation strategies, inefficiency of Transformer-based architectures for long sequences, overemphasis on global classification tasks, and fragmented, task-specific pipelines. This project proposes a systematic exploration of these challenges through three interlinked sub-projects. First, I will design a modality-adaptive patch embedding strategy that dynamically adjusts to local signal complexity, improving token efficiency without sacrificing clinical fidelity. Second, I will develop DiffAug4MH, a diffusion-based augmentation framework coupled with a Mamba-based architecture to generate semantically faithful perturbations and achieve scalable, memory-efficient modeling of long clinical sequences. Third, I will integrate these components into a unified, multi-task multimodal foundation model capable of supporting classification, forecasting, imputation, anomaly detection, synthesis, and clinical report generation within a single architecture. Using large-scale datasets such as PhysioNet and PTB-XL, I will benchmark and open-source the tokenizer, augmentation library, pre-trained weights, and curated datasets. Expected outcomes include new standards for multimodal healthcare tokenization and augmentation, a robust open-source library for clinical AI, and a general-purpose multimodal foundation model deployable in real-world hospital settings. Beyond advancing the scientific state of the art, this work aims to deliver tangible societal benefits, including personalized preventive care, improved clinical decision support, and reduced barriers for hospitals and start-ups to innovate in digital health.