Dario Cioni

PhD
Queen Mary University of London (QMUL)

Embodied conversational agents must generate a coherent human figure while responding to an interlocutor, whose speech and visible behaviour continue to unfold in real time. This requires causal perception, controllable behavioural generation and continuous rendering at interactive latency without losing audiovisual quality, identity or intelligence. Existing systems can generate compelling audiovisual output, but achieving reliable contextual and affective appropriateness remains difficult when intermediate representations provide limited temporal precision, semantic control or access to the unfolding user interaction. This proposal studies these requirements through three questions. The first asks how causal generative video models can be optimised during or after training to generate realistic human behaviour at interactive latency while preserving identity, visual quality and temporal consistency over conversation-length rollouts. The second asks how such a generator can be conditioned and post-trained to produce affective, contextually appropriate behaviour for both conversational roles, and what control a behavioural interface to a full-duplex dialogue model allows. The third asks which multimodal representation of human behaviour, spanning affect, expression, vocal tone, pose and gesture across both participants, supports perception and cross-person prediction and can condition generation. The work combines renderer studies, controlled conditioning and post-training experiments, and representation learning on dyadic data. The expected outcome is a set of methods and measurements for responsive, controllable and computationally practical embodied conversational agents.

Academic Track
May 5th, 2026 - May 5th, 2030
ELLIS Edge Newsletter
Join the 6,000+ people who get the monthly newsletter filled with the latest news, jobs, events and insights from the ELLIS Network.