Tony Danjun Wang
PhD
Technical University of Munich (TUM)

This PhD project aims to develop machine-learning methods for modeling embodied agents and physical processes in the dynamic environment of the surgical operating room. The research will investigate multimodal world models that integrate visual, spatial, kinematic, and contextual information into structured representations of agents, patients, instruments, and their interactions. These models will learn the dynamics of surgical scenes and predict how they may evolve under partial observability. Rather than predicting a single most likely outcome, the proposed methods will represent multimodal distributions over plausible futures, capturing uncertainty in agent behavior, procedural progression, and physical consequences. These predictive representations will enable counterfactual simulation, calibrated forecasting, and risk-aware planning. A central challenge will be to jointly model multi-agent coordination and physical dynamics while incorporating procedural and safety constraints. The resulting methods will be evaluated through surgical workflow anticipation, interaction prediction, uncertainty estimation, and downstream planning or decision-support tasks. Ultimately, the project seeks to establish embodied world models as a foundation for safe, adaptive, and context-aware surgical intelligence.

Academic Track
March 1st, 2026 - February 1st, 2030
ELLIS Edge Newsletter
Join the 6,000+ people who get the monthly newsletter filled with the latest news, jobs, events and insights from the ELLIS Network.