Multimodal Fact Checking

Jonathan Tonglet (Ph.D. Student)

Recent crises, such as the COVID-19 pandemic, have shown that social media are a major conduit for the dissemination of misinformation. Misinformation proliferates even more rapidly when presented in a multimodal format, such as memes, images with captions, or audio and video files. This multimodal way of conveying content is becoming more and more central on the main social media platforms. In this project, our primary objective is to develop advanced systems capable of automatically fact-checking claims that encompass two or more modalities, including text, image, audio, and video. We envision creating new datasets and methodologies specifically designed to address the unique challenges of multimodal fact-checking. We hope this project can help prevent the spread of non-factual and potentially harmful content on social media, with a particular emphasis on mitigating the impact during crisis contexts.

Primary Host:	Iryna Gurevych (Technical University of Darmstadt)
Exchange Host:	Marie-Francine Moens (KU Leuven)
PhD Duration:	01 September 2023 - 31 August 2027
Exchange Duration:	01 September 2024 - 31 May 2025 - Ongoing