Carsten Stoll

Research Scientist at Epic Games.

prof_pic.jpg

I am a Research Scientist at Epic Games, specializing in the development next generation digital humans.

I earned my PhD in Computer Graphics and Computer Vision from the Max-Planck Institute for Informatics with Prof. Christian Theobalt. I also lead a group on Optical Performance Capture at the Max-Planck Center for Visual Computing and Communication and spent a year as visiting researacher at Weta Digital. I cofounded The Captury, a company focused on real-time markerless motion capture, and then joined Facebook Reality Labs to work on full body virtual humans, before joining Epic in 2020.

My research interests span a wide range of topics from computer graphics, geometric modeling, animation, and computer vision. My primary goal is to advance the technology and tools to create and control fully photorealistic and believable digital human avatars.

Projects

ML Deformer
At Epic Games I developed the ML model driving Unreal Engine's ML Deformer plugin that can be used to create a high-fidelity next-generation character with deformations driven by full muscle, flesh, and cloth simulation running in real-time.
Momentum
At Facebook/Oculus Research I developed the Momentum library for human kinematics and numerical optimization that was used to prototype markerless motion capture algorithms and was used for early versions of the Oculus VR headset upper body tracking.

Publications

(in reverse chronological order)

2024

  1. 2024_eccv_humos.jpg
    HUMOS: Human Motion Model Conditioned on Body Shape
    Shashank Tripathi, Omid Taheri, Christoph Lassner, Michael J. Black, Daniel Holden, and Carsten Stoll
    In ECCV, 2024
  2. 2024_eccv_epoch.jpg
    EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans
    Nicola Garau, Giulia Martinelli, Niccolo Bisagno, Denis Tome, and Carsten Stoll
    In ECCV T-CAP Workshop, 2024
  3. 2024_arxiv_fnerf.jpg
    fNeRF: High Quality Radiance Fields from Practical Cameras
    Yi Hua, Christoph Lassner, Carsten Stoll, and Iain Matthews
    In arxiv, 2024
  4. 2024-icassp_phisanet.jpg
    PhisaNet: Phonetically Informed Speech Animation Network
    Salvador Medina, Sarah L. Taylor, Carsten Stoll, Gareth Edwards, Alex Hauptmann, Shinji Watanabe, and Iain Matthews
    In ICASSP, 2024

2023

  1. 2023-iccvw-personalized_pose_and_shape.jpg
    Personalized 3D Human Pose and Shape Refinement
    Tom Wehrbein, Bodo Rosenhahn, Iain Matthews, and Carsten Stoll
    In IEEE/CVF ICCVW, 2023

2022

  1. 2022-cvpr-speech_driven_tongue_animation.jpg
    Speech Driven Tongue Animation
    Salvador Medina, Denis Tomè, Carsten Stoll, Mark Tiede, Kevin Munhall, Alex Hauptmann, and Iain Matthews
    In IEEE/CVF CVPR, 2022

2021

  1. 2021_cvpr_anr.jpg
    ANR: Articulated Neural Rendering for Virtual Avatars
    Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, and Christoph Lassner
    In IEEE CVPR, 2021

2020

  1. 2020-eccv-texmesh.jpg
    TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video
    Tiancheng Zhi, Christoph Lassner, Tony Tung, Carsten Stoll, Srinivasa G. Narasimhan, and Minh Vo
    ECCV, 2020
  2. 2020-eccv-patch_nets.jpg
    PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations
    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, and Christian Theobalt
    In ECCV, 2020

2015

  1. 2015_cgf_outdoor_capture.jpg
    Outdoor Human Motion Capture by Simultaneous Optimization of Pose and Camera Parameters
    Ahmed Elhayek, Carsten Stoll, Kwang In Kim, and Christian Theobalt
    Eurographics, 2015

2013

  1. 2013_cgf_relightable_performance.jpg
    Capturing Relightable Human Performances under General Uncontrolled Illumination
    Guannan Li, Chenglei Wu, Carsten Stoll, Yebin Liu, Kiran Varanasi, Qionghai Dai, and Christian Theobalt
    Eurographics, 2013
  2. 2013_pami_multi_character.jpg
    Markerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
    Yebin Liu, Juergen Gall, Carsten Stoll, Qionghai Dai, Hans-Peter Seidel, and Christian Theobalt
    IEEE PAMI, 2013
  3. 2013_sig_onset_perfcap.jpg
    On-set performance capture of multiple actors with a stereo camera
    Chenglei Wu, Carsten Stoll, Levi Valgaerts, and Christian Theobalt
    SIGGRPAH Asia, 2013

2012

  1. 2012_cgf_spatiotemporal_filtering.jpg
    Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ Videos
    Christian Richardt, Carsten Stoll, Neil A. Dodgson, Hans-Peter Seidel, and Christian Theobalt
    Eurographics, 2012
  2. 2012_cvpr_spactime_tracking.jpg
    Spatio-temporal motion tracking with unsynchronized cameras
    Ahmed Elhayek, Carsten Stoll, Nils Hasler, Kwang In Kim, Hans-Peter Seidel, and Christian Theobalt
    In IEEE CVPR, 2012
  3. 2012_dagm_video_synchronization.jpg
    Feature-Based Multi-video Synchronization with Subframe Accuracy
    Ahmed Elhayek, Carsten Stoll, Kwang In Kim, Hans-Peter Seidel, and Christian Theobalt
    In DAGM, 2012

2011

  1. 2011_sig_video_based_characters.jpg
    Video-based characters: creating new human performances from a multi-view video database
    Feng Xu, Yebin Liu, Carsten Stoll, James Tompkin, Gaurav Bharaj, Qionghai Dai, Hans-Peter Seidel, Jan Kautz, and Christian Theobalt
    SIGGRAPH Asia, 2011
  2. 2011_cvpr_interacting_characters.jpg
    Markerless motion capture of interacting characters using multi-view image segmentation
    Yebin Liu, Carsten Stoll, Juergen Gall, Hans-Peter Seidel, and Christian Theobalt
    In , 2011
  3. 2011_iccv_sum_of_gaussians.jpg
    Fast articulated motion tracking using a sums of Gaussians body model
    Carsten Stoll, Nils Hasler, Juergen Gall, Hans-Peter Seidel, and Christian Theobalt
    In ICCV, 2011

2010

  1. 2010_sig_video_reconstruction.jpg
    Video-based reconstruction of animatable human characters
    Carsten Stoll, Juergen Gall, Edilson Aguiar, Sebastian Thrun, and Christian Theobalt
    SIGGRAPH Asia, 2010
  2. 2010_eccv_motion_structure.jpg
    Joint Estimation of Motion, Structure and Geometry from Stereo Sequences
    Levi Valgaerts, Andrés Bruhn, Henning Zimmer, Joachim Weickert, Carsten Stoll, and Christian Theobalt
    In ECCV, 2010

2009

  1. 2009_template_processing.jpg
    Template based shape processing
    Carsten Stoll
    Saarland University, 2009
  2. 2009_cag_estimating_body.jpg
    Estimating body shape of dressed humans
    Nils Hasler, Carsten Stoll, Bodo Rosenhahn, Thorsten Thormählen, and Hans-Peter Seidel
    Shape Modeling International, 2009
  3. 2009_cgf_statistical_model.jpg
    A Statistical Model of Human Pose and Body Shape
    Nils Hasler, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and Hans-Peter Seidel
    Eurographics, 2009
  4. 2009_cvpr_mocap_skeleton.jpg
    Motion capture using joint skeleton tracking and surface estimation
    Juergen Gall, Carsten Stoll, Edilson Aguiar, Christian Theobalt, Bodo Rosenhahn, and Hans-Peter Seidel
    In IEEE (CVPR), 2009

2008

  1. 2008_sig_performance_capture.jpg
    Performance capture from sparse multi-view video
    Edilson Aguiar, Carsten StollChristian Theobalt, Naveed Ahmed, Hans-Peter Seidel, and Sebastian Thrun
    ACM SIGGRAPH, 2008

2007

  1. 2007_cvpr_markerless_deformable.jpg
    Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture
    Edilson Aguiar, Christian TheobaltCarsten Stoll, and Hans-Peter Seidel
    In IEEE (CVPR), 2007
  2. 2007_tr_volumetric_editing.jpg
    A Volumetric Approach to Interactive Shape Editing
    Carsten Stoll, Edilson Aguiar, Christian Theobalt, and Hans-Peter Seidel
    In Technical Report, 2007
  3. 2007_vr_rapid_animation.jpg
    Rapid Animation of Laser-scanned Humans
    Edilson Aguiar, Christian TheobaltCarsten Stoll, and Hans-Peter Seidel
    In IEEE VR, 2007
  4. 2007_pg_geodesics.jpg
    Geodesics guided constrained texture deformation
    Carsten Stoll, Zachi Karni, and Hans-Peter Seidel
    In Pacific Graphics, 2007

2006

  1. 2006_sma_bsp_shapes.jpg
    BSP Shapes
    Carsten StollHans-Peter Seidel, and Marc Alexa
    In Shape Modeling and Applications, 2006
  2. 2006_rt_raycasting_quadratic.jpg
    Incremental Raycasting of Piecewise Quadratic Surfaces on the GPU
    Carsten Stoll, Stefan Gumhold, and Hans-Peter Seidel
    In Symposium on Interactive Raytracing, 2006
  3. 2006_pbg_template_deformation.jpg
    Template Deformation for Point Cloud Fitting
    Carsten Stoll, Zachi Karni, Christian Rössl, Hitoshi Yamauchi, and Hans-Peter Seidel
    In Point Based Graphics@SIGGRAPH, 2006

2005

  1. 2005_vis_stylized_lines.jpg
    Visualization with stylized line primitives
    Carsten Stoll, Stefan Gumhold, and Hans-Peter Seidel
    In IEEE Vis, 2005

2003

  1. 2003_wscg_direction_fields.jpg
    Direction Fields over Point-Sampled Geometry
    Marc Alexa, Tobias Klug, and Carsten Stoll
    In WSCG, 2003