Yuxuan Xue

Hi There! I am a Ph.D. student in the Real Virtual Human group at University of Tuebingen , supervised by Prof. Dr. Gerard Pons-Moll. I affiliate with International Max-Planck Research School for Intelligent Systems (IMPRS-IS). My research interests include 3D/4D photorealistic reconstruction, generative models, and understanding of human appearance and behavior.

Prior to that, I spent wonderful time in Max-Planck-Institute for Intelligent Systems. I graduated with the double Master degree in Mechanical Engineering as well as Robotics both with distinction from the Technical University of Munich (TUM) in 2022. I received the Bachelor degree in Mechanical Engineering from the same University in 2020.

Email  /  Google Scholar  /  LinkedIn  /  CV  /  Twitter  /  Github

We propose a new generative model which generates photorealistic 3D avatars from text description with precise control in body shape and wearing clothing. Our key idea is that the foundation models already understand human appearance and we distill 3D information from them, resulting a datasets with 111K diverse subjects.

We propose a general reconstruction framework for recovering holistic human and environment from monocular images. Leveraging to foundational depth and pose estimation models, we optimize towards physical constraints and result in physically plausible 3D human-scene interaction and contact.

We extend the idea of Human-3Diffusion to general objects. Our Gen-3Diffusion reconstructs high-fidelity 3D representation from single RGB Image within 22 seconds and 11 GB GPU memory, which allows an efficient large-scale 3D generation.

We propose a new approach to reconstruct high-fidelity avatar in 3D Gaussian Splats from single RGB Image. Our approach improves 2D multi-view diffusion process by using reconstructed 3D representation to guarantee 3D consistency at reverse sampling steps.

We propose E-LnR, an event-based approach which reconstruct non-rigid deformation in the low rank parametrized space. This is a journal extension of our BMVC 2022 paper.

We propose BOFT (Orthogonal Butterfly), a general orthogonal finetuning technique with butterfly factorization that effectively adapts foundation models to different tasks such as Vision, NLP, Math QA, and Controllable Generation.

We propose a new approach to define a neural field on the surface for reconstructing animatable clothed human from monocular depth observation. Our approach directly outputs coherent meshes across different poses at arbitrary resolution.

We propose Orthogonal Finetuning (OFT), a fine-tuning approach for adapting text-to-image diffusion models to downstream tasks. OFT can preserve hyperspherical energy to maintain the semantic generation ability of the foundation models.

We propose a new approach for reconstructing fast non-rigid object deformations using measurements from event-based cameras. Our approach estimates object deformation from events at the object contour within a probabilistic optimization (EM) framework.

We present a new method for detecting event actions from skeletal information in RGBD videos. The proposed method uses a Gradual Online Dictionary Learning algorithm to cluster and filter skeleton frames. Additionally, the method includes a latent unit temporal structure to better distinguish event actions from similar actions.

Master Thesis in Robotics, Cognition, Intelligence (RCI) at TUM. Deformable objects tracking using monocular event stream.

Bachelor thesis in Mechanical Engineering at TUM. High precision Lane Following control algorithm developed for autonomous driving car.