Gustav Schmidt

Humans can judge with little effort how similar two objects are, and recent work has shown that these judgments can improve Al models by bringing their representations closer to those of humans. At present, however, this alignment remains relatively superficial. We do not yet know human representations in enough detail to align models with the rich conceptual structure that supports human knowledge. This project therefore addresses both problems of (1) identifying human representations at much finer resolution and (2) aligning deep learning models with these representations. However, since the number of possible stimuli and comparisons is far too large to sample exhaustively, we will use iterative adaptive sampling, in which each new stimulus or comparison is selected based on the representational structure recovered so far. This closed loop will allow us to identify detailed human representations efficiently and use the resulting structure as a target for training and evaluating Al models and for identifying new candidate stimuli. The initial work will focus on visual representations. We will then test whether the same principles extend to language and vision-language models. The project will thereby develop a common framework in which adaptive sampling progressively improves both our understanding of human knowledge and the alignment of Al systems with it.