Multi-modal Continual Learning
Vishaal Udandarao (Ph.D. Student)
The ability to learn continually across a diverse set of tasks and domains is an important fundamental problem in machine learning. Continual learning has emerged to be a pivotal research area in which systems are trained on diverse downstream tasks sequentially, while performing well on all the tasks equally. Most current continual learning systems operate in the uni-modal regime. Multi-modal systems have however been the recent rage in the machine learning ecosystem with the training of several large-scale multi-modal foundation models. This has been fostered by the ease of access to compute, the optimism proffered by several scaling laws, and curation of web-scale datasets. Yet, only a handful of works study the ability of these large-scale multi-modal models to continually adapt to tasks. Therefore, during the course of the PhD, our goal is to build efficient tools for enabling effective coalitions of continual learning and multi-modal learning systems. This is essential for building intelligent, generalist systems.
|Primary Host:||Matthias Bethge (University of Tübingen)|
|Exchange Host:||Samuel Albanie (University of Cambridge)|
|PhD Duration:||01 October 2022 - 30 September 2025|
|Exchange Duration:||01 January 2024 - 01 September 2024 - Ongoing|