ELLIS Unit Trento: Bridging Disciplines for Foundational and Applied AI

Mission & Vision

The mission of ELLIS Unit Trento is to conduct cutting-edge, multidisciplinary research in artificial intelligence, addressing both foundational and application-oriented challenges. Our work spans several core domains of AI, including machine learning, computer vision, remote sensing, and natural language processing. This broad scope enables us to contribute meaningfully to both theoretical advancements and real-world impact.

We are deeply committed to training the next generation of AI talent through dedicated academic programs such as the Master in Artificial Intelligence Systems (taught in English) and the EIT Digital Master School program. At the doctoral level, our IECS Doctoral School supports over 250 active PhD students, including several participants in the ELLIS PhD program.

In addition to academic excellence, Unit Trento places strong emphasis on transferring research outcomes to society and the market. We maintain close collaborations with industry partners and public institutions, actively contributing to innovation and public service. Our research has directly supported the creation of high-impact companies and spin-offs such as SpazioDati, PerVoice, U-Hopper, Eoptis, Eye Pro System, and Matecat - demonstrating our commitment to fostering socio-economic development and delivering real-world value through AI.

Key Numbers

The ELLIS Unit Trento officially joined the ELLIS network in 2023. The motivation to become part of this prestigious European initiative stemmed from a shared commitment to excellence in AI research and innovation. Trento offers a unique ecosystem, thanks to the close collaboration between DISI (University of Trento), FBK (Fondazione Bruno Kessler), and CIMeC (Center for Mind/Brain Sciences). This colocation enables high-impact, interdisciplinary research at the intersection of computer science, cognitive sciences, and applied AI.

Joining the ELLIS network was a strategic step to strengthen international collaborations, attract top talent, and contribute to advancing human-centered, cutting-edge AI research. It also aligns with their broader mission to ensure that AI technologies benefit both the economy and society, building on successful initiatives in Industry 4.0, healthcare, and smart cities.

The Unit involves 21 Members out of which there are 6 Fellows:

Bruno Lepri (FBK, Unit Director)
Nicu Sebe (UNITN, Unit Director)
Andrea Passerini (UNITN)
Elisa Ricci (UNITN/FBK)
Lorenzo Bruzzone (UNITN)
Raffaella Bernardi (UNITN/University of Bolzano)

And 1 Scholar:

Jacopo Staiano (UNITN)

Achievements & Milestones

Pioneering Research in Human-Centric AI: ELLIS Unit Trento has significantly contributed to areas such as human action recognition, emotion analysis, generative AI, and multimodal learning, emphasizing the development of AI systems that understand and interpret human behavior.

Innovative Applications of Deep Learning: The research group has explored the use of deep learning techniques in various domains, including healthcare, surveillance, creative technologies, and human-computer interaction, pushing the boundaries of how machines perceive and interact with human-centric data.

Collaborative Projects and Industry Partnerships: The unit has engaged in numerous collaborative projects, both within academia and with industry partners, facilitating the translation of research findings into practical applications and fostering innovation in multimedia understanding.

Recognition in Academic Conferences: The unit member's work has been recognized in top-tier conferences such as CVPR, ICCV, ECCV, ICLR, NeurIPS, ICML, ACL, EMNLP, AAAI, etc. highlighting their contributions to the research advancements of computer vision, human-centric machine learning, natural language processing, multimedia, geometric deep learning, etc.

Collaborating across ELLIS

There are several collaborations with other ELLIS Units. Most notable are the ones with the ELLIS Units Modena, Alicante and Tübingen.

UNITN (part of ELLIS Unit Trento) and UNIMORE (ELLIS Unit Modena) have cultivated a productive collaboration in the field of computer vision, focusing on enhancing the efficiency and reliability of deep learning models. Their partnership exemplifies a commitment to advancing the capabilities of artificial intelligence through innovative approaches to model optimization. Their joint research includes the paper "Optimizing Resource Consumption in Diffusion Models through Hallucination Early Detection," presented at the European Conference on Computer Vision Workshops in 2024. This study addresses the challenge of computational resource management in diffusion models by introducing a method for early detection of hallucinations, thereby improving model performance and efficiency.

Another joint work published in ACM Multimedia 2023 presents ViCE, an automated framework that evaluates how well generated images match textual prompts by mimicking human cognitive processes. Combining LLMs and VQA, ViCE identifies key concepts, generates questions, and scores images accordingly. It outperforms existing metrics like CLIPScore and LLMScore, offering a scalable, human-aligned evaluation method.

FBK, UNITN (ELLIS Unit Trento) and ELLIS Unit Alicante have forged a robust and impactful collaboration at the intersection of human-centric machine learning and computational social science. In particular, the two units are collaborating on studying the effects of human and AI biases in decision-making tasks. A first collaborative work, titled “What is beautiful is still good: the attractiveness halo effect in the era of beauty filters”, has been published in the Royal Society Open Science journal. Other works are currently under submission.

UNITN and MPI (part of ELLIS Unit Tübingen) have established a dynamic research collaboration at the forefront of computer vision and quantum computing. Their joint efforts have led to significant advancements in motion segmentation and model fitting. Notably, they co-authored the paper "Quantum Motion Segmentation", introducing the first algorithm for motion segmentation that leverages adiabatic quantum optimization, achieving performance on par with state-of-the-art methods. Additionally, their collaboration on “Quantum Multi-Model Fitting” presented at CVPR 2023 explored quantum-inspired techniques for fitting multiple models to complex visual data.

These interdisciplinary projects underscore their commitment to integrating quantum computing principles into computer vision, pushing the boundaries of what's possible in the field.

Research Areas

The Unit's activities are centered on conducting cutting-edge research across several key areas in artificial intelligence, ranging from foundational studies in machine learning, computer vision, remote sensing, and natural language processing to application-driven initiatives. Our research efforts are primarily supported through national and international projects acquired via competitive funding schemes. The outcomes of these activities are consistently targeted toward publication in leading scientific venues.

Members of Unit Trento have established a strong track record of collaboration with numerous Fellows and Members of the ELLIS network. They intend to further expand our network of collaborations, laying the groundwork for new large-scale initiatives at the European level. In support of this goal, DISI has dedicated funding to host international researchers, and these resources will be prioritized for welcoming members of the ELLIS community.

The research activities within the ELLIS Unit Trento are highly interdisciplinary, encompassing both theoretical and applied dimensions of AI:

Learning from Visual Data: Our work focuses on developing innovative frameworks for learning from large-scale visual datasets, with particular emphasis on weakly supervised, unsupervised, and continual learning settings. This includes domain adaptation and self-supervised approaches. We also design advanced algorithms for the automatic classification and synthesis of visual data, drawing on recent advances in vision transformers, deep generative models, diffusion models, and neural radiance fields (NeRFs).
Explainable, Trustworthy, and Cooperative AI: We aim to transition explainability from a predominantly post-hoc analysis to an integral part of interactive learning strategies, thereby enhancing model reliability and interpretability. Our research includes neuro-symbolic integration—combining deep learning with automated reasoning techniques—to develop inherently interpretable and trustworthy AI systems. In parallel, we explore socially inspired cooperative AI by integrating machine learning with complex systems methodologies. These efforts align closely with the objectives of the ELLIS programs on Semantic, Symbolic and Interpretable Machine Learning and Human-centric Machine Learning.
Bringing Human Diversity into AI: We are bridging two previously distinct research directions—bias and disagreement in NLP datasets, and multimodal conversational AI—to more accurately reflect human diversity in AI systems. Our work addresses how varying perspectives can be incorporated into both data creation and classification processes. We are also developing adaptive multimodal conversational systems tailored to diverse user profiles, thereby reinforcing ELLIS goals in the field of natural language processing.
AI for Remote Sensing and Data Fusion: We design AI-driven methods for the analysis and integration of remote sensing data, employing supervised, unsupervised, and self/semi-supervised learning techniques. Our research addresses classification, change detection in imagery and time series, and domain adaptation, with applications in climate change, disaster response, environmental monitoring, and agriculture. Additionally, we investigate AI-based approaches for the simulation, processing, and interpretation of remote sensing data in planetary exploration, with a focus on radar sounder technologies.

Ongoing Research Projects

ELLIS Unit Trento is leading ELIAS - European Lighthouse of AI for Sustainability, a European Project focused on fostering cooperation among more than 30 partner organisations. ELIAS stands out as a pioneering endeavor due to both its scope and innovative approach. With a mission to galvanize the field of machine learning, ELIAS aims to establish itself as a central reference point by fostering collaboration among 34 active affiliates in this sector, along with a cohort of associated partners hailing from 17 different countries.

Researchers are developing new computational systems and mathematical models to provide original and achievable solutions to urgent contemporary issues. ELIAS' challenge is to bridge the academic and business worlds at the European level. The tools to achieve this include research, the mobility of doctoral students, and support for young talents who, upon completing their studies, can establish innovative startups.

The research areas are diverse but interconnected. For instance, artificial intelligence can be used to identify the best materials for constructing green buildings or environmentally friendly industrial components. It can also be employed to monitor the migration of people arriving in Europe and investigate the motivations behind their journeys. Additionally, AI can be applied to develop cybersecurity prevention systems and models for analysing climate change.

ELIAS is embarking on collaborative initiatives in tandem with other European Network of Excellence (NoE) projects, such as dAIEdge, Enfield, ELSA, as well as ELLIS Units. This collaborative effort establishes a comprehensive ecosystem for the advancement of artificial intelligence.

The University of Trento is coordinating the collaborative efforts among different partners, encompassing universities, research institutions, and industrial entities, all united in their commitment to addressing the pressing challenges of our time: environmental sustainability, equitable economic development, and societal well-being.

Unit Director Niculae Sebe, a professor at the Department of Information Engineering and Computer Science (DISI) and the project’s coordinator, highlighted the project’s mission:

Our objective is to ensure that technological breakthroughs in the realm of machine learning not only elevate the quality of life for individuals, but also benefit society as a whole and contribute to the well-being of our entire planet. Our endeavors will revolve around the development of algorithms designed to uncover solutions that promote and attain this sustainability.

The project has received funding from the European Union's Horizon Europe research and innovation program under grant agreement No 101120237.

Support for Young Talents

Several educational initiatives have been implemented with the purpose of training the next generation of AI talents. UNITN has a Master degree in Artificial Intelligence Systems (all in English) and since many years has participated in the EIT Digital Master School program, a two-year master program fostering student mobility in Europe comprising several technical courses covering key areas of AI (e.g. Autonomous Systems).

At doctoral level, the IECS Doctoral School has 200+ active PhD students, some of them already enrolled in the ELLIS PhD program.

Moreover, the unit participates in many summer schools and training initiatives. For example, ELLIS VISMAC Winter School in Brunico 2025, which focused on large-scale foundation models, self-supervised and multi-modal representation learning, multi-modal learning for robotics, and machine learning for sustainable solutions.