Multi-modal Continual Learning

Vishaal Udandarao (Ph.D. Student)

The ability to learn continually across a diverse set of tasks and domains is an important fundamental problem in machine learning. Continual learning has emerged to be a pivotal research area in which systems are trained on diverse downstream tasks sequentially, while performing well on all the tasks equally. Most current continual learning systems operate in the uni-modal regime. Multi-modal systems have however been the recent rage in the machine learning ecosystem with the training of several large-scale multi-modal foundation models. This has been fostered by the ease of access to compute, the optimism proffered by several scaling laws, and curation of web-scale datasets. Yet, only a handful of works study the ability of these large-scale multi-modal models to continually adapt to tasks. Therefore, during the course of the PhD, our goal is to build efficient tools for enabling effective coalitions of continual learning and multi-modal learning systems. This is essential for building intelligent, generalist systems.

Primary Host:	Matthias Bethge (University of Tübingen)
Exchange Host:	Samuel Albanie (University of Cambridge)
PhD Duration:	01 October 2022 - 30 September 2025
Exchange Duration:	01 January 2024 - 01 September 2024 - Ongoing