Cross-border collaboration in Machine Learning and Computer Vision: ELLIS program directors share insights

Computer vision is a transformative technology with large implications for our everyday lives. It empowers computers to analyze and interpret visual data such as images and videos, and forms the backbone of numerous applications. From medical imaging and improved diagnosis in healthcare to self-driving cars, photo filters on social media and QR code scanners: Computer vision has become indispensable across industries.

The ELLIS Program ‘Machine Learning and Computer Vision’ fosters collaboration among researchers in this dynamic field in Europe and Israel, and aims to bridge the gap between classical vision algorithms and modern machine learning to allow for greater insights and breakthroughs.

Three renowned researchers lead the program: Bernt Schiele, Director at the Max Planck Institute for Informatics (Germany), Cordelia Schmid, Research Director at INRIA (France), and Yair Weiss, Professor at the School of Computer Science and Engineering at the Hebrew University of Jerusalem (Israel).

The three directors jointly prepared the following answers to give us an insight into how the members of their program collaborate across borders and to explain emerging trends in the area of machine learning and computer vision.
 

Computer vision technologies are developing rapidly and the field has seen great advances in recent years. What are typical scenarios where people might engage with these technologies in their daily routines?

This question is actually harder to answer today than it was 10 years ago, but for a good reason. In the past, there were a handful of successful deployments of computer vision and so we could immediately point to one of these applications when asked. Today, there are so many success stories that it is difficult to choose which ones to highlight so we will give a few examples from our personal routine. In the neighborhood supermarket, we can take fruit and vegetables to the self-checkout line and a computer vision algorithm correctly identifies them (often better than we can). When we join a Zoom meeting from our home, a computer vision algorithm segments us from the messy background and replaces it with a cleaner background. When we travel to a conference and rent a car, a computer vision algorithm warns us if we forget to signal before changing lanes.

 

More generally speaking: What are some of the emerging trends in computer vision that will shape the future of the field?

Probably the strongest sentiment that is shared by most computer vision experts is the sense of wonder at how well things work today compared to where we were 10-15 years ago. Invariably, this progress was made possible by the widespread use of Deep Learning. At the same time, this reliance on Deep Learning has made modern methods reliant on obtaining high quality training data that can faithfully capture the statistics of the world in which the systems will be deployed. Whenever there is a mismatch between the training statistics and the real world statistics, the performance of current methods can be surprisingly poor. Perhaps this is the reason why self-driving cars have been described as ‘a year away’ for more than 10 years.

 

Could you provide an overview of your research program? What issues do you focus on, and what are ongoing projects?

Our Fellows work on the full range of Computer Vision problems: from relatively 'low-level' problems such as estimating the 3D motion of a camera to much more high 'high-level' problems such as learning how to change a flat tire from an instructional video and transferring that knowledge to a robot. What is common to all the research done in the program, however, is the strong reliance on Machine Learning and this serves as a common language between the different projects.

 

Your program has been fostering connections among researchers across Europe and Israel since 2019. Reflecting on these years, could you share some of the highlights from this international collaboration? Has the cross-national collaboration paid off?

When we were putting together the list of people to invite as Fellows of the program, we did not really take into account their nationality. Many of us have served together on program committees for the major vision conferences and have known each other for many years by meeting regularly at these conferences. However, meeting regularly at our workshop does deepen our mutual understanding, and has led to cross-Europe publications and collaborations. For example, a recent publication at NeurIPS23 featured three of our Fellows with co-authors from Prague and Paris.

 

What are your plans and wishes for the future of the program?

A major disadvantage of the success of computer vision is that our main conferences have become gigantic with thousands of attendees and at the same time the talks have gotten shorter and shorter. This makes it impossible for us to interact at conferences in a way that is scientifically meaningful. Our wish is for the ELLIS workshops to provide this meaningful scientific interaction on a smaller scale.

 

More information

View all members of the ELLIS Program ‘Machine Learning and Computer Vision’ here.  

ELLIS research programs focus on areas with the potential to move the needle in modern AI - from basic research in theory and algorithms to applications in health and climate sciences. They include outstanding researchers as ELLIS Fellows and Scholars. Learn more about all 16 ELLIS Programs here. ELLIS program workshops are often supported by the EU-funded ELISE project which is building on ELLIS.

Find ELLIS contact details here.

 

Social media

Follow ELLIS on Twitter, LinkedIn, Mastodon and Facebook, and subscribe to the monthly ELLIS newsletter.