ELLIS Summer School in Tübingen Marks 50th Machine Learning Summer School with Over 250 Students

This year marks a major milestone: the 50th edition of the Machine Learning Summer School. Since its founding, MLSS has brought together students, researchers, and leading experts from around the world to share knowledge and shape the future of machine learning.

Reaching the 50th edition of the Machine Learning Summer School is a milestone I am very proud of. MLSS has always been a place where new ideas take shape, and this year was no exception: the discussions on causality, AI safety, and the future of foundation models will stay with this community and help drive the next advances in the field. It was wonderful to see a new generation of researchers engage with these questions here in Tübingen. 

- Bernhard Schölkopf (ELLIS Fellow, ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems) co-founder of the Machine Learning Summer Schools and co-organiser of this year’s school

Speaker Highlights

Among the highlights was Yoshua Bengio's lecture on AI safety. Bengio (Université de Montréal; LawZero; Mila) presented his "Scientist AI" architecture, a new type of model that he argues should be safe by design and an alternative to LLMs. Alex Smola's (Boson AI) lectures on using hardware efficiently to run LLMs were another favourite.

Arthur Mensch, CEO of Mistral AI, made a surprise appearance. He spoke about his own earlier attendance at a previous edition of the Tübingen MLSS, his path from researcher to entrepreneur and the state of AI in Europe. 

Topics That Sparked Discussion

Causality was a recurring theme, as it could be an important ingredient of future AI systems. The speakers disagreed on how it will get there. Some expect LLMs to pick it up from more data and more scaling, while others build causality into the architecture itself. In the words of the organisers, the question comes down to genotype versus phenotype. Does a model carry causality in its "genes", or does it simply learn to behave causally from data? There is no consensus yet.

The first talk by Giambattista Parascandolo (former ELLIS PhD student, now at OpenAI), who pioneered reasoning models, set students thinking with a provocative question: what if your PhD could be just a prompt in a frontier model in a few years? The question is about the role a PhD student, and in general a scientist, has in the era of LLMs. 

This was at the start of the summer school, sparking thoughts between the students and the speakers throughout the two weeks. Speakers such as Arthur Mensch and Olivier Bousquet (Mistral AI) brought new arguments in this topic by stating that a PhD offers more skills than just solving scientific problems, useful in one's career. They also pointed out the role of understanding in science and formulating the questions and problems that a scientific community needs to pose.

A panel on foundation models brought together Alex Smola, Frank Hutter (Prior Labs; University of Freiburg) and Carl-Johann Simon-Gabriel (Mirelo AI). They shared their experiences of founding startups and offered advice to students considering the same path.

Overview of Speakers and Topics

Lectures

• Alex Smola (Boson AI) on "Inference in ML"

• Arthur Gretton (University College London; Google DeepMind) on "Causal Effect Estimation With Context and Confounders"

• Bernhard Schölkopf (ELLIS Institute Tübingen and MPI-IS) on “From language models to causal world models: A tale of spurious correlations”

• Christoph Lampert (Institute of Science and Technology Austria) on "Private Machine Learning"

• Francesco Locatello (Institute of Science and Technology Austria)

• Frank Hutter (Prior Labs; University of Freiburg) on "Tabular Foundation Models"

• Isabel Valera (Saarland University) on "Causal Generative Models: From Theory to Practice"

• Jan Peters (TU Darmstadt; DFKI) on "Inductive Biases for Robot Learning"

• Jonas Geiping (ELLIS Institute Tübingen; Max Planck Institute for Intelligent Systems) on "Safety and Security of LLM Chain-of-Thought"

• Jonas Peters (ETH Zurich) on "Causality"

• Kun Zhang (Carnegie Mellon University; MBZUAI) on "Causal Representation Learning, GenAI, and Beyond"

• Maksym Andriushchenko (ELLIS Institute Tübingen; Max Planck Institute for Intelligent Systems) on "Measuring Capabilities and Risks of Recursive Self-Improvement"

• Mario Krenn (University of Tübingen) on "Towards an Artificial Muse for New Ideas in Physics"

• Matthew Blaschko (KU Leuven) on "From Images to Measurement: Vision-Based Metrology via Calibrated Uncertainty"

• Olivier Bousquet (Mistral AI) on "ML Research in the Age of LLMs"

• Philipp Hennig (University of Tübingen; Tübingen AI Center) on "Probabilistic ML"

• Roger Grosse (University of Toronto; Anthropic) on "Efficient Retrieval of Influential LLM Training Examples"

• Suvrit Sra (Technical University of Munich; Octet) on "Selected Topics in Optimization for Deep Learning"

• Ulrike von Luxburg (University of Tübingen) on "Learning Theory"

• Yarin Gal (University of Oxford) on "Uncertainty in Deep Learning"

• Yoshua Bengio (Université de Montréal; LawZero; Mila)

• Zeynep Akata (Technical University of Munich; Helmholtz Munich) on "Explainability and Adaptability of Multimodal LLMs"

Tutorials and hands-on sessions

• Antonio Orvieto (ELLIS Institute Tübingen) on "Optimization"

• Julius von Kügelgen (ETH Zurich) on "Hands-On Causal Inference"

• Mateo Rojas-Carulla (Lakera; Check Point Software), Julia Bazinska (Lakera) and Niklas Pfister (Lakera) on "How to Break a Coding Agent: On Harnesses, Context, and Where to Hide Things"

• Maximilian Dax (ELLIS Institute Tübingen; Max Planck Institute for Intelligent Systems) on "Simulation-Based Inference"

• Nicolas Zucchet (Stanford University) on "Scaling Laws on LLMs: Practice and Theory"

• Patrik Reizinger (Max Planck Institute for Intelligent Systems) on "Principles for (Agentic) Research"

• Ricardo Dominguez-Olmedo (Max Planck Institute for Intelligent Systems) on "Benchmarks and How They Drive Progress in AI"

• Shiwei Liu (ELLIS Institute Tübingen) on "From Depth to Latency: Architecture, Compression, and Efficient Inference"

• Tim Weiland and Marvin Pförtner (University of Tübingen) on "Probabilistic Numerics"

• Zhijing Jin (University of Toronto; Max Planck Institute for Intelligent Systems) on "Causal Reasoning for LLMs"

• Ziheng Chen (Max Planck Institute for Intelligent Systems) on "Riemannian Deep Learning: Foundations, Architectures, and Practice"

Panels

• "Deploying Foundation Models": Carl-Johann Simon-Gabriel (Mirelo AI), Frank Hutter (Prior Labs) and Alex Smola (Boson AI), hosted by Bernhard Schölkopf

• "Industrial Research in ML": Olivier Bousquet (Mistral AI), Denny Zhou (Google DeepMind) and Gökhan Bakır (Enzian Labs), hosted by Bernhard Schölkopf

• "Reasoning and Safety in LLMs": Mateo Rojas-Carulla (Lakera; Check Point Software) and Giambattista Parascandolo (OpenAI), hosted by Bernhard Schölkopf

• "Societal Impacts of Artificial Intelligence": Sahar Abdelnabi (ELLIS Institute Tübingen) and Maksym Andriushchenko (ELLIS Institute Tübingen; Max Planck Institute for Intelligent Systems), hosted by Ulrich Hemel (Institute of Social Strategy)

Cyber Valley Day

In the second week, participants joined Cyber Valley Day, which celebrated the tenth anniversary of Cyber Valley at its building in Tübingen. The day featured speeches by Cem Özdemir, Baden-Württemberg’s Minister President, Boris Palmer, Tübingen's mayor, Yoshua Bengio, and Bernhard Schölkopf.

The programme also included talks by leading researchers and innovators, an exhibition of cutting-edge research projects, a start-up basecamp, and an investor speed dating session. In the evening, participants could network over refreshments from food trucks, with plenty of opportunities to connect across disciplines and organisations.

Social Programme and Life in Tübingen

The school was designed to be as social as it was scientific, with one social event every day. On the first Friday, the organisers surprised participants with a jazz band led by Asmar Nadzhafova and pizza in the Max Planck garden, and turned the Max Planck lecture hall into a nightclub. Other highlights included punting on the Neckar and a barbecue dinner in the second week.

The programme also changed how some participants see the city. One student said that before MLSS, Tübingen was not really on his map, and that afterwards he saw it as an amazing place to work and do research.

For many early-career researchers, the school was also a chance to build connections that could last well beyond the two weeks. Two attendees reflected on what that meant for them:

I truly believe that when I look back in 20 years, this edition of MLSS will be an essential part of my career. As I'm just starting my PhD, it was inspiring to see that many of the lecturers had once been MLSS students themselves, and that friendships formed there had lasted for years. A PhD is not only about your own work, but also about having great people around you to exchange ideas with, challenge you, and help you grow. I feel I found many of those important people at MLSS.

- Alejandro Hernandez Artiles (ELLIS PhD Student, MPI for Human Development and MPI for Intelligent Systems)

This MLSS is an amazing experience where we come from everywhere around the world to learn knowledge, to grow, to share ideas and build friendships that last across time and space. I truly appreciate everyone who makes it possible: all the organizers, speakers, and all the participants. It will always be a remarkable memory and I look forward to seeing many of these wonderful people again.

- Evie Qiu (ELLIS PhD Student, ISTA and MPI for Intelligent Systems)

Main Organisers

• Lancelot Da Costa

• Simon Buchholz

• Cedric Ewen

• Hsiao-Ru Pan

• Nikos Papanikolaou

• Bernhard Schölkopf

Contributors to this article: Lancelot Da Costa & Nikos Papanikolaou