Jose Serra
Doug Roble
ABSTRACT
We present a unified pipeline for high-resolution facial capture that replaces the initial traditional seated capture with a head-mounted camera setup. At its core, our approach relies on improving roughly personalized blendshapes by fitting handle vertices, in a Laplacian framework, to depth and image data. Thus, refining the geometry. This pipeline has been used in production to generate high quality animation to train our proprietary marker-based solution, leading to large time and cost savings.
Supplemental Material
Available for Download
- T. Beeler, F. Hahn, D. Bradley, B. Bickel, P. Beardsley, C. Gotsman, R. Sumner, and M. Gross. 2011. High-Quality Passive Facial Performance Capture Using Anchor Frames. ACM Trans. Graph. 30, 4, Article 75 (July 2011), 10 pages.Google Scholar
Digital Library
- DI4D. 2021. DI4D Pro. Retrieved Feb 10, 2021 from https://www.di4d.com/di4d-pro/Google Scholar
- G. Fyffe, K. Nagano, L. Huynh, S. Saito, J. Busch, A. Jones, H. Li, and P. Debevec. 2017. Multi-View Stereo on Consistent Face Topology. CGF 36(2017), 295–309.Google ScholarDigital Library
- D. Hendler, L. Moser, R. Battulwar, D. Corral, P. Cramer, R. Miller, R. Cloudsdale, and D. Roble. 2018. Avengers: Capturing Thanos’s Complex Face. In ACM SIGGRAPH 2018 Talks. Article 58, 2 pages.Google Scholar
- ICT. 2021. ICT - Light Stages. Retrieved Feb 10, 2021 from https://vgl.ict.usc.edu/LightStages/Google Scholar
- L. Moser, D. Hendler, and D. Roble. 2017. Masquerade: Fine-Scale Details for Head-Mounted Camera Motion Capture Data. In ACM SIGGRAPH 2017 Talks. New York, NY, USA, Article 18, 2 pages.Google Scholar
- S. Park, X. Zhang, A. Bulling, and O. Hilliges. 2018. Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings. In ACM Symposium on Eye Tracking Research and Applications (Warsaw, Poland). ACM.Google Scholar
- O. Sorkine. 2005. Laplacian Mesh Processing. In Eurographics 2005 - State of the Art Reports. The Eurographics Association.Google Scholar
Index Terms
Simplified facial capture with head mounted cameras
Recommendations
Accelerating facial motion capture with video-driven animation transfer
SIGGRAPH '22: ACM SIGGRAPH 2022 TalksWe describe a hybrid pipeline that leverages: 1) video-driven animation transfer [Moser et al. 2021] for regressing high-quality animation under partially-controlled conditions from a single input image, and 2) a marker-based tracking approach [Moser ...
Parallax error in the monocular head-mounted eye trackers
UbiComp '12: Proceedings of the 2012 ACM Conference on Ubiquitous ComputingThis paper investigates the parallax error, which is a common problem of many video-based monocular mobile gaze trackers. The parallax error is defined and described using the epipolar geometry in a stereo camera setup. The main parameters that change ...
Motion capture from body-mounted cameras
SIGGRAPH '11: ACM SIGGRAPH 2011 papersMotion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large ...
Comments