Generative Models for Virtual Humans

Anton Zubekhin (Ph.D. Student)

With the recent advancements in generative modelling, it is now possible to create high-quality images and videos from scratch with greater control. Simultaneously, the field of 3D reconstruction and novel view synthesis of human bodies has been rapidly developing. Despite these advancements, many challenges remain open. The generation of diverse virtual human models (shape, appearance, moton) in a controllable way and remains a large open challenge. This PhD project aims to address this issue by focusing on developing new generative models that facilitate the creation of highly realistic full-body human models. The project offers various research routes, including enhancing the photorealism of the generated avatars, improving geometry reconstruction, and introducing versatile controllability into the generation process. Further, the project will explore how state-of-the-art generative model architectures, such as diffusion models, can be further advanced for improved controllability and training on unlabeled or weakly labeled in-the-wild-data in a self-supervised way.

Primary Advisor:	Christian Theobalt (Max Planck Institute for Informatics)
Industry Advisor:	Thabo Beeler (Google)
PhD Duration:	11 September 2023 - Ongoing