Yinghong Yu

Vision foundation models have matured on 2D natural images, whereas clinical imaging such as CT is volumetric and paired 3D image-report data remain far scarcer than 2D X-ray pairs. This project leverages 2D resources, namely pretrained foundation models and large-scale X-ray-report pairs, to build a clinically accurate and self-improving vision-language foundation model for 3D medical imaging, with CT report generation as the core task. The research proceeds in four stages.

Stage 1: 2D-to-3D lifting (completed). PlaneCycle (MICCAI 2026) lifts pretrained 2D foundation models to 3D tasks without training or adapters.

Stage 2: X-ray-to-CT transfer (submission in preparation). The lifted model serves as a shared backbone for X-ray and CT, so that the vision-language alignment learned from abundant X-ray-report pairs transfers to the data-scarce CT-report task.

Stage 3: Clinically accurate report generation. Larger and more diverse 3D image-report corpora and stronger visual grounding improve clinical accuracy, evaluated with clinical metrics and radiologist review.

Stage 4: Self-improving loop. The system automatically ingests new 3D image-report data and improves itself through cycles of generation, verification and self-training.

Overall, the project will deliver an open-source 3D medical vision-language foundation model that sets a new state of the art in clinically accurate CT report generation, serving as a ready-to-use and extensible baseline for the community.