Carsten Stoll

I am a Research Scientist at Meta, specializing in the development next generation digital humans.

prof_pic.jpg

I earned my PhD in Computer Graphics and Computer Vision from the Max-Planck Institute for Informatics with Prof. Christian Theobalt. I also lead a group on Optical Performance Capture at the Max-Planck Center for Visual Computing and Communication and spent a year as visiting researacher at Weta Digital. I cofounded The Captury, a company focused on real-time markerless motion capture, and then joined Facebook Reality Labs to work on full body virtual humans. In 2020 I joined Epic Games to work on next generation Metahumans and machine learning methods for real time worlds. In 2025 I rejoined Meta to help push Virtual Humans to the next level.

My research interests span a wide range of topics from computer graphics, geometric modeling, animation, and computer vision. My primary goal is to advance the technology and tools to create and control fully photorealistic and believable digital human avatars.

Projects

Metahuman Bodies
At Epic Games I lead the resarch and technical development of a parametric body model for MetaHumans, released with Unreal Engine 5.6. While prior versions of MetaHumans were based on discrete body types, the parametric models allow fine grained direct and measurement based editing of body shapes.
ML Deformer
I also developed the ML model driving Unreal Engine's ML Deformer plugin that can be used to create a high-fidelity next-generation character with deformations driven by full muscle, flesh, and cloth simulation running in real-time. ML Deformer is now widely used in game development such as for example the Witcher 4 tech demo shown at Unreal Fest 2025.
Momentum
At Facebook/Oculus Research I developed the Momentum library for human kinematics and numerical optimization that was used to prototype markerless motion capture algorithms and was used for early versions of the Oculus VR headset upper body tracking.

Publications

(in reverse chronological order)

2024

  1. 2024_eccv_humos.jpg
    HUMOS: Human Motion Model Conditioned on Body Shape
    Shashank Tripathi, Omid Taheri, Christoph Lassner, Michael J. Black, Daniel Holden, and Carsten Stoll
    In ECCV, 2024
    arXiv PDF
  2. 2024_eccv_epoch.jpg
    EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans
    Nicola Garau, Giulia Martinelli, Niccolo Bisagno, Denis Tome, and Carsten Stoll
    In ECCV T-CAP Workshop, 2024
    arXiv PDF
  3. 2024_arxiv_fnerf.jpg
    fNeRF: High Quality Radiance Fields from Practical Cameras
    Yi Hua, Christoph Lassner, Carsten Stoll, and Iain Matthews
    In arxiv, 2024
    arXiv PDF
  4. 2024-icassp_phisanet.jpg
    PhisaNet: Phonetically Informed Speech Animation Network
    Salvador Medina, Sarah L. Taylor, Carsten Stoll, Gareth Edwards, Alex Hauptmann, Shinji Watanabe, and Iain Matthews
    In ICASSP, 2024
    PDF

2023

  1. 2023-iccvw-personalized_pose_and_shape.jpg
    Personalized 3D Human Pose and Shape Refinement
    Tom Wehrbein, Bodo Rosenhahn, Iain Matthews, and Carsten Stoll
    In IEEE/CVF ICCVW, 2023
    arXiv PDF Supp

2022

  1. 2022-cvpr-speech_driven_tongue_animation.jpg
    Speech Driven Tongue Animation
    Salvador Medina, Denis Tomè, Carsten Stoll, Mark Tiede, Kevin Munhall, Alex Hauptmann, and Iain Matthews
    In IEEE/CVF CVPR, 2022
    PDF Code

2021

  1. 2021_cvpr_anr.jpg
    ANR: Articulated Neural Rendering for Virtual Avatars
    Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, and Christoph Lassner
    In IEEE CVPR, 2021
    arXiv PDF

2020

  1. 2020-eccv-texmesh.jpg
    TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video
    Tiancheng Zhi, Christoph Lassner, Tony Tung, Carsten Stoll, Srinivasa G. Narasimhan, and Minh Vo
    ECCV, 2020
    arXiv PDF
  2. 2020-eccv-patch_nets.jpg
    PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations
    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, and Christian Theobalt
    In ECCV, 2020
    arXiv PDF

2015

  1. 2015_cgf_outdoor_capture.jpg
    Outdoor Human Motion Capture by Simultaneous Optimization of Pose and Camera Parameters
    Ahmed Elhayek, Carsten Stoll, Kwang In Kim, and Christian Theobalt
    Eurographics, 2015
    PDF Video

2013

  1. 2013_cgf_relightable_performance.jpg
    Capturing Relightable Human Performances under General Uncontrolled Illumination
    Guannan Li, Chenglei Wu, Carsten Stoll, Yebin Liu, Kiran Varanasi, Qionghai Dai, and Christian Theobalt
    Eurographics, 2013
    PDF Video
  2. 2013_pami_multi_character.jpg
    Markerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
    Yebin Liu, Juergen Gall, Carsten Stoll, Qionghai Dai, Hans-Peter Seidel, and Christian Theobalt
    IEEE PAMI, 2013
    PDF Video
  3. 2013_sig_onset_perfcap.jpg
    On-set performance capture of multiple actors with a stereo camera
    Chenglei Wu, Carsten Stoll, Levi Valgaerts, and Christian Theobalt
    SIGGRPAH Asia, 2013
    PDF Video

2012

  1. 2012_cgf_spatiotemporal_filtering.jpg
    Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ Videos
    Christian Richardt, Carsten Stoll, Neil A. Dodgson, Hans-Peter Seidel, and Christian Theobalt
    Eurographics, 2012
    PDF Video
  2. 2012_cvpr_spactime_tracking.jpg
    Spatio-temporal motion tracking with unsynchronized cameras
    Ahmed Elhayek, Carsten Stoll, Nils Hasler, Kwang In Kim, Hans-Peter Seidel, and Christian Theobalt
    In IEEE CVPR, 2012
    PDF Video
  3. 2012_dagm_video_synchronization.jpg
    Feature-Based Multi-video Synchronization with Subframe Accuracy
    Ahmed Elhayek, Carsten Stoll, Kwang In Kim, Hans-Peter Seidel, and Christian Theobalt
    In DAGM, 2012
    PDF

2011

  1. 2011_sig_video_based_characters.jpg
    Video-based characters: creating new human performances from a multi-view video database
    Feng Xu, Yebin Liu, Carsten Stoll, James Tompkin, Gaurav Bharaj, Qionghai Dai, Hans-Peter Seidel, Jan Kautz, and Christian Theobalt
    SIGGRAPH Asia, 2011
    PDF Video
  2. 2011_cvpr_interacting_characters.jpg
    Markerless motion capture of interacting characters using multi-view image segmentation
    Yebin Liu, Carsten Stoll, Juergen Gall, Hans-Peter Seidel, and Christian Theobalt
    In , 2011
    PDF Video
  3. 2011_iccv_sum_of_gaussians.jpg
    Fast articulated motion tracking using a sums of Gaussians body model
    Carsten Stoll, Nils Hasler, Juergen Gall, Hans-Peter Seidel, and Christian Theobalt
    In ICCV, 2011
    PDF Video

2010

  1. 2010_sig_video_reconstruction.jpg
    Video-based reconstruction of animatable human characters
    Carsten Stoll, Juergen Gall, Edilson Aguiar, Sebastian Thrun, and Christian Theobalt
    SIGGRAPH Asia, 2010
    PDF
  2. 2010_eccv_motion_structure.jpg
    Joint Estimation of Motion, Structure and Geometry from Stereo Sequences
    Levi Valgaerts, Andrés Bruhn, Henning Zimmer, Joachim Weickert, Carsten Stoll, and Christian Theobalt
    In ECCV, 2010
    PDF

2009

  1. 2009_template_processing.jpg
    Template based shape processing
    Carsten Stoll
    Saarland University, 2009
    PDF
  2. 2009_cag_estimating_body.jpg
    Estimating body shape of dressed humans
    Nils Hasler, Carsten Stoll, Bodo Rosenhahn, Thorsten Thormählen, and Hans-Peter Seidel
    Shape Modeling International, 2009
    PDF
  3. 2009_cgf_statistical_model.jpg
    A Statistical Model of Human Pose and Body Shape
    Nils Hasler, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and Hans-Peter Seidel
    Eurographics, 2009
    PDF
  4. 2009_cvpr_mocap_skeleton.jpg
    Motion capture using joint skeleton tracking and surface estimation
    Juergen Gall, Carsten Stoll, Edilson Aguiar, Christian Theobalt, Bodo Rosenhahn, and Hans-Peter Seidel
    In IEEE (CVPR), 2009
    PDF Video

2008

  1. 2008_sig_performance_capture.jpg
    Performance capture from sparse multi-view video
    Edilson Aguiar, Carsten Stoll, Christian Theobalt, Naveed Ahmed, Hans-Peter Seidel, and Sebastian Thrun
    ACM SIGGRAPH, 2008
    PDF Video

2007

  1. 2007_cvpr_markerless_deformable.jpg
    Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture
    Edilson Aguiar, Christian Theobalt, Carsten Stoll, and Hans-Peter Seidel
    In IEEE (CVPR), 2007
    PDF
  2. 2007_tr_volumetric_editing.jpg
    A Volumetric Approach to Interactive Shape Editing
    Carsten Stoll, Edilson Aguiar, Christian Theobalt, and Hans-Peter Seidel
    In Technical Report, 2007
    PDF
  3. 2007_vr_rapid_animation.jpg
    Rapid Animation of Laser-scanned Humans
    Edilson Aguiar, Christian Theobalt, Carsten Stoll, and Hans-Peter Seidel
    In IEEE VR, 2007
    PDF
  4. 2007_pg_geodesics.jpg
    Geodesics guided constrained texture deformation
    Carsten Stoll, Zachi Karni, and Hans-Peter Seidel
    In Pacific Graphics, 2007
    PDF

2006

  1. 2006_sma_bsp_shapes.jpg
    BSP Shapes
    Carsten Stoll, Hans-Peter Seidel, and Marc Alexa
    In Shape Modeling and Applications, 2006
    PDF
  2. 2006_rt_raycasting_quadratic.jpg
    Incremental Raycasting of Piecewise Quadratic Surfaces on the GPU
    Carsten Stoll, Stefan Gumhold, and Hans-Peter Seidel
    In Symposium on Interactive Raytracing, 2006
    PDF Video
  3. 2006_pbg_template_deformation.jpg
    Template Deformation for Point Cloud Fitting
    Carsten Stoll, Zachi Karni, Christian Rössl, Hitoshi Yamauchi, and Hans-Peter Seidel
    In Point Based Graphics@SIGGRAPH, 2006
    PDF

2005

  1. 2005_vis_stylized_lines.jpg
    Visualization with stylized line primitives
    Carsten Stoll, Stefan Gumhold, and Hans-Peter Seidel
    In IEEE Vis, 2005
    PDF

2003

  1. 2003_wscg_direction_fields.jpg
    Direction Fields over Point-Sampled Geometry
    Marc Alexa, Tobias Klug, and Carsten Stoll
    In WSCG, 2003
    PDF