Comprehensive scene understanding and state estimation are pivotal for achieving reliable robot
autonomy in human-centred environments. Yet, the availability of reliable training data undermines
the robustness - and thus safety - of such systems, especially in edge cases not or barely covered.
This project focuses on learning methods without extensive manual human supervision to facilitate
efficient learning and transferability of learned models in panoptic segmentation and tracking tasks to
enable superior spatio-temporal reasoning. Further, the research is directed toward the fusion of
multiple modalities to increase the robustness of predictions in challenging environmental conditions
and allow online learning by enforcing consistency among them. Cross-modality of this kind will also
be directly focused on introspection, where explainability of failures (a crucial capability for trustworthy
autonomous systems) in one sensor stream will be boosted by other sensor streams.