The Helmholtz-ELLIS Workshop Explored the Role of Foundation Models Across Diverse Research Fields
The Helmholtz-ELLIS Workshop on Foundation Models in Science brought top researchers together to explore AI’s transformative role across disciplines like materials science, genomics, and astronomy. Unlike larger conferences, it fostered deep, cross-disciplinary discussions in an intimate setting, enabling collaboration between academia and industry leaders. Key highlights included AI-driven material generation, metagenomic modeling for pandemic tracking, and new approaches to foundation models in tabular data. The event underscored both technical advancements and critical challenges, such as benchmarking, evaluation, and ethical deployment, shaping the future of AI in scientific discovery.
The Helmholtz-ELLIS Workshop on Foundation Models in Science took place on March 18-19, 2025, in Berlin. Jointly organized by the Helmholtz Foundation Model Initiative and the ELLIS Units in Heidelberg, Munich and Potsdam, the event highlighted how large language and multimodal models are reshaping scientific research across disciplines - including materials science, astronomy, and physics. Leading researchers from institutions such as Microsoft Research, Meta, OpenAI, and top universities shared their insights, demonstrating the growing impact of foundation models in accelerating scientific discovery.
The workshop aimed not only to share current research progress but also to spark cross-disciplinary collaboration. A highlight were the close interactions in a stimulating environment, offering attendees a rare chance to engage directly with leaders in the field. As co-organizer and ELLIS Scholar Stefan Bauer (Helmholtz Center Munich) noted:
Most ML conferences are still in the US and Canada and by now often too large to offer meaningful interactions. From coffee breaks to an evening apero beneath towering dinosaur skeletons in the natural history museum, it was an opportunity to interact and have intense scientific discussions with leading researchers in person.
ELLIS Related Contributions
Among the contributions from the ELLIS network was a presentation by ELLIS Member Gaël Varoquaux (Inria), who introduced CARTE, a model that could potentially lay the groundwork for foundation models in tabular data - an area that remains underdeveloped compared to vision and language domains. If successful, CARTE could open new possibilities for applying AI to structured scientific datasets, from clinical records to experimental data tables.
(Gaël Varoquaux, Inria)
ELLIS Fellow Lena Maier-Hein (DKFZ & University of Heidelberg) delivered a thought-provoking critique of current evaluation methodologies used in assessing foundation model performance. She emphasized the need for more rigorous and context-sensitive strategies, particularly as these models are applied in high-stakes fields like medicine, where robustness and reliability are paramount.
(Lena Maier-Hein giving her talk on “Rethinking Foundation Model Evaluation”.)
Matthias Bethge (University of Tübingen, ELLIS Fellow) gave a talk on Cosmopolitan Foundation Models emphasizing that such models must be designed to serve people effectively, highlighting the need for human-centered approaches in their development and deployment.
(Matthias Bethge presenting on the topic of "Cosmopolitan Foundation Models".)
Günther Klambauer (Johannes Kepler University, ELLIS Scholar) introduced xLSTM-based foundation models tailored for sequence data in DNA and proteins, demonstrating their enhanced efficiency for applications like drug discovery.
Scientific Highlights and Advancements
Beyond ELLIS-affiliated talks, the workshop featured a diverse array of interdisciplinary contributions by A-class speakers in the field. Kangwook Lee(University of nWisconsin-Madison) proposed a novel method of “sentencifying” non-text data to fine-tune large language models (LLMs), effectively bridging the gap between traditional machine learning and language-based AI.
A standout moment for many was Shirley Ho’s (Simons Foundation & NYU) talk on Polymathic AI, a vision for cross-disciplinary foundation models capable of transferring insights across physics, chemistry, and biology - a notion that resonated with many in attendance, highlighting the blurring boundaries between scientific domains.
Material science also featured prominently, with Tian Xie (Microsoft Research AI for Science) showcasing MatterGen, a generative model capable of designing entirely new materials with desired properties - marking a shift from simulation to generation in material discovery.
In the life sciences, Tim Hempel and Michael Gastegger (Microsoft Research AI for Science) presented BioEmu, a model that, unlike AlphaFold, focuses on predicting protein function by modeling the distribution of possible structures - a significant step toward solving one of biology’s most complex challenges. Willie Neiswanger’s (USC) talk introduced METAGENE-1, a 7-billion-parameter metagenomic foundation model trained on human wastewater DNA and RNA sequences, designed for pandemic monitoring and showcasing state-of-the-art performance in pathogen detection and genomic sequence embedding.
We wanted to bring the global community working on foundation models for science together to strengthen it, to grow it, and to brainstorm the next steps, and also perhaps come up with some more outlandish future directions. (Dagmar Kainmüller, head of the Integrative Imaging Data Sciences lab at the Max Delbrück Center, workshop organizer)
Sarath Chandar (MILA & Polytechnique Montreal) emphasized that bigger models don’t always equate to better outcomes. Chandar’s team’s Amplify 350M model counters biases found in large models like ESMB2, while BindGPT innovatively uses 3D molecular generation within protein pockets, outperforming existing GNN models through reinforcement learning.
(Panel discussion with l/r Shirley Ho, Michal Valko, Johannes von Oswald, Kangwook Lee and moderator Stefan Kesselheim)
The workshop also featured keynotes from Piotr Bojanowski (Meta FAIR), who showcased thow self-supervised models like DINOv2 perform strongly in domains such as satellite and medical imaging, matching CLIP-like models in classification and setting new standards in tasks like segmentation and depth estimation. Anna Scaife (University of Manchester) addressed the need for AI in handling exascale data from the upcoming Square Kilometre Array, showcasing self-supervised models that can identify rare galaxy types without labeled examples. Michal Valko (CMO Stealth Startup, Inria & ENS MVA) explored new approaches for aligning language models with human preferences, proposing alternatives to existing RLHF methods, offering stronger theoretical foundations and practical improvements. Johannes von Oswald (ETH & Google Zurich) examined transformer alternatives, such as Mamba and RWKV, and discussed their potential for more efficient, scalable language modeling. These models offer promising directions beyond current architectures by optimizing for memory and computational efficiency.
Lucas Beyer (OpenAI), discussed the pre-training and post-training aspects of both image and vision-language foundation models, focusing on the transferable underlying principles. His presentation also highlighted a broader challenge in the field - the need for meaningful benchmarking across modalities and domains, an issue that remains unresolved even for major technology companies.
The Value of Interdisciplinary Workshops
Beyond technical insights, the workshop highlighted the value of community. Such interactions enable more effective feedback, richer idea exchange, and the formation of future research collaborations. For junior researchers, it provided a digestible overview of recent advancements. Additionally, practitioners applying AI in various domains benefited from first hand exposure to cutting-edge developments, broadening their perspectives and sparking new ideas.
I was incredibly impressed with the quality of speakers and diversity of topics. Although I was already familiar with most of the big breakthroughs presented, there were important take-aways in every single talk from the power of leveraging large language models for tabular foundation models presented by Gaël Varoquaux to the impact of foundation models in astronomy presented by Shirley Ho. (Lena Maier-Hein, DKFZ & University of Heidelberg, ELLIS Fellow)
From an ELLIS perspective, the workshop exemplified the value of interdisciplinary dialogue and the open exchange of ideas. As foundation models continue to transform the landscape of science, fostering these connections will be essential to guiding their development in ways that are robust, ethical, and impactful across disciplines.
For further insights on the idea behind the workshop and current challenges in the field, we recommend reading the interview with event organizer Dagmar Kainmüller, head of the Integrative Imaging Data Sciences lab at the Max Delbrück Center.
The event was sponsored by ELLIS, the Helmholtz Foundation Model Initiative, Helmholtz AI, Helmholtz Imaging, HEIBRIDS, and ZUSE School ELIZA.
Pictures credit to Svea Pietschmann for Max Delbrück Center.
