Recent crises, such as the COVID-19 pandemic, have shown that social media are a major conduit for the dissemination of misinformation. Misinformation proliferates even more rapidly when presented in a multimodal format, such as memes, images with captions, or audio and video files. This multimodal way of conveying content is becoming more and more central on the main social media platforms. In this project, our primary objective is to develop advanced systems capable of automatically fact-checking claims that encompass two or more modalities, including text, image, audio, and video. We envision creating new datasets and methodologies specifically designed to address the unique challenges of multimodal fact-checking. We hope this project can help prevent the spread of non-factual and potentially harmful content on social media, with a particular emphasis on mitigating the impact during crisis contexts.