Weakly-Supervised Multi-Modal Learning for Scene Understanding

Antonín Vobecký (Ph.D. Student)

The objective of the research is to improve the accuracy of machine learning models under the conditions of low amounts of annotated training/testing data as well as the lack of correctly defined data distributions in the context of autonomous driving applications. The key open problem in these low-data regimes is low model accuracy in situations that are rare or missing in the training data or overfitting to the training data distribution that inherently differs from testing and validation. On the other hand, large amounts of un-annotated data with multiple sensor modalities (multiple sensory captures for the same scene and time) as well as virtual data are usually available in the automotive set-ups. Among the methods that we will investigate for addressing these problems are distribution aware generative models, domain adaptation, learning with privileged information (when multimodal data are available at training time), weakly supervised and semi-supervised learning, adversarial learning techniques to identify the rare cases, and help to identify the right training data distribution.

Primary Advisor:	Josef Sivic (Czech Technical University, École Normale Supérieure & INRIA)
Industry Advisor:	Patrick Pérez (Valeo.ai)
PhD Duration:	01 October 2019 - 01 September 2023