Understanding the Mechanisms of Representation Learning in Foundation Models

Rajat Sahay (Ph.D. Student)

Despite the success of deep learning models in general and foundation models in particular, there is little understanding of the exact properties of the trained models and even less so on how these properties get established during the training process. This makes it difficult to ensure a desired behavior of the models for a wide variety of inputs and tasks, and limits the use of the models for high-risk applications, where argument about the reliability of the system are required. This project aims to analyze the most successful foundation models and their future successors empirically to find likely properties and behaviors of the models. This includes the development of appropriate methods to probe networks and of a taxonomy to define properties that matter and are measurable. On top of these properties, the goal is to also explain by which mechanisms of the network and/or the training process they get established. The latter will allow us to make predictions under which circumstances a model will show a certain behavior.

Primary Advisor:	Thomas Brox (University of Freiburg)
Industry Advisor:	Volker Fischer (Bosch Center for AI)
PhD Duration:	01 September 2024 - 31 August 2028