Beyond the Thesis with Steffen Schneider

Steffen Schneider is a Tenure Track Group Leader at Helmholtz Munich and the Munich Center for Machine Learning (MCML), where he develops AI methods for dynamical systems, time series, and systems neuroscience. His work focuses on developing novel machine learning methods for interpreting complex scientific data and advancing life sciences through AI-driven discovery. During his ELLIS PhD with Matthias Bethge (University of Tübingen) and Mackenzie Mathis (EPFL), he focused on model robustness and self-supervised learning, culminating in CEBRA, a contrastive learning algorithm for scientific data analysis.

Name: Steffen Schneider

Affiliation: Helmholtz Munich & Munich Center for Machine Learning (MCML)

Role: Tenure Track Group Leader

Areas of research: Dynamical systems, self-supervised learning, time series, explainable AI, systems neuroscience

Further info: https://dynamical-inference.ai / https://stes.io

PhD Advisors:  Mackenzie Mathis (ELLIS Scholar) and Matthias Bethge (ELLIS Fellow) 

ELLIS Experience

Can you briefly describe your PhD research and how the ELLIS network or its resources supported it?

I did my ELLIS PhD with Matthias Bethge in Tübingen, and with Mackenzie Mathis at EPFL. This PhD setup allowed me to work within the AI community in Tübingen on a range of topics around adaptation, self-supervised learning and object centric learning, before embarking on building CEBRA, contrastive learning algorithm for scientific data analysis and a major part of my PhD thesis. I split my time roughly 50:50 between Tübingen and EPFL, and also spent some time at the Rowland Institute at Harvard before Mackenzie’s lab moved to Switzerland. Besides these frequent exchanges, I attended the ELLIS PhD symposia, the ELLIS summer school on probabilistic machine learning in Cambridge, as well as an ELLIS meeting in Crete.

What was the most valuable aspect of being part of the ELLIS community during your PhD?

Engaging in very interdisciplinary research, and being part of a community with a lot of talented and inspiring individuals. To me, ELLIS opened the opportunity to find a perfect match to my interest by splitting my time between two labs. At every point along the way, I got to know new research environments, made new friends and learned new ideas, new methods, new perspectives, and new ways of working that shaped my path forward.

Since I was in the first batch of students, it was also great to see how ELLIS grew and evolved over the years into the leading network for AI research in Europe. 

Current Role & Career Path

Can you briefly describe your career trajectory since graduating from the ELLIS PhD program?

I explored different options before and during my PhD. I spent a great year at Meta before starting my PhD, working on self-supervised contrastive learning for speech recognition. During my PhD, I did a (remote) internship at Amazon, and another research intern at Meta in New York. At some point I decided to center my career around using machine learning to create an impact in neuroscience and life science more broadly. When many companies got focused on scaling larger scale text/vision models, I finally settled on a career in academia.

I went on my job search in winter 2022/23 already. After graduating from ELLIS in January 2024, I started a tenure track position at Helmholtz Munich in March 2024. Since then, I’ve been building up my group in Munich. Part-time besides research, I continue to build my non-profit startup “KI macht Schule” (AI@Schools) which I started with fellow doctoral students before my PhD. KI macht Schule is our effort of responsibly reshaping education in the age of AI. We build and design teaching materials on AI, and provide infrastructure for using AI responsibly in school. Our approach is really to create links between science and education, and bridge barriers between state of the art AI research and what is taught in school.

How did your experience in the ELLIS PhD program prepare you for your current role?

Research communities in ML and neuroscience are very different, from the style of paper writing to how a project is conceived and conducted. It was very beneficial to dive into both worlds during a single PhD project, and ELLIS was an enabler for that.

What are some of your most significant research contributions since completing your PhD?

An important milestone for me was to get the first papers of my group published and presented at conferences, that was a great experience. In one, we showed that self-supervised learning is able to perform identification of non-linear systems from observational data, and built an algorithm, “dynamics contrastive learning”, which we now apply in numerous collaborations in life science contexts at Helmholtz and beyond.

We also presented new work on sparse autoencoders, which is an exciting new way analysing foundation models. This also sparked a lot of interesting collaborations, and we are presenting the first application to medical data later this year at MICCAI.

Impact of Your Work in AI

In what ways does your work help drive progress in AI and its impact on society?

We do both fundamental AI research and the transfer to actual experimental setups. Our fundamental work sparks new methods for interpretability and explainability especially in the context of time-series analysis and system identification. But we then take these new methods, team up with experimental labs, and co-design analysis approaches to advance life scientific discovery. At Helmholtz I am working in a department for computational health, and I believe that AI will make a big difference in both fundamental and clinical contexts.

We have an increasing pile of data in the life sciences generated by very sophisticated experimental setups, but very few algorithms to actually make sense of the data and draw scientific conclusions from them. I am really eager to contribute to closing this methodological gap.

What are the most exciting or promising areas of AI research that you see on the horizon?

I see big opportunities for using AI/ML as tools for scientific discovery, especially within “lab in the loop” setups where an ML model is trained on data collected in a scientific experiment, and makes inferences about the most worthwhile next step in the experiment. This is possible because our theoretical understanding of ML algorithms improved substantially in the last decade, and it is now possible to leverage these tools for statistical modeling of very complex systems.

Watch the interview highlights in the video below: