Mateusz Pach

Multimodal Foundation Models (MFMs) combine data from sources such as text, images, and audio to perform a wide range of tasks, including those beyond their original training. We have yet to fully explore their capabilities, particularly their strong generalization and zero-shot abilities. As MFMs are increasingly used in high-stakes scenarios, ensuring their interpretability becomes crucial and is often mandated by regulations. This project has two primary goals: 1) to push the boundaries of what MFMs can achieve, and 2) to leverage them to create interpretable systems suited for critical applications. To achieve these goals, the research will investigate innovative techniques to enhance MFM performance, discover new applications, and develop robust frameworks for interpretability. Ultimately, the project aims to bridge the gap between advanced MFM functionality and human-centric understanding, promoting the responsible and effective use of these models in essential fields.