Abstract
The human face is an anatomical system exhibiting heterogenous and anisotropic mechanical behavior. This leads to complex deformations even in a neutral facial expression due to external forces such as gravity. We start by building a volumetric model from magnetic resonance images of a neutral facial expression. To obtain data on facial deformations we capture and register 3D scans of the face with different gravity directions and with various facial expressions. Our main contribution consists in solving an inverse physics problem where we learn mechanical properties of the face from our training data (3D scans). Specifically, we learn heterogenous stiffness and prestrain (which introduces anisotropy). The generalization capability of our resulting physics-based model is tested on 3D scans. We demonstrate that our model generates predictions of facial deformations more accurately than recent related physics-based techniques.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Building Accurate Physics-based Face Models from Data
- Jascha Achenbach, Robert Brylka, Thomas Gietzen, Katja zum Hebel, Elmar Schömer, Ralf Schulze, Mario Botsch, and Ulrich Schwanecke. 2018. A multilinear model for bidirectional craniofacial reconstruction. In Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine. Eurographics Association, 67--76. Google Scholar
Digital Library
- Oleg Alexander, Mike Rogers, William Lambeth, Jen-Yuan Chiang, Wan-Chun Ma, Chuan-Chang Wang, and Paul Debevec. 2010. The digital Emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, IEEE (2010). Google Scholar
Digital Library
- Michael Bao, Matthew Cong, Stéphane Grabli, and Ronald Fedkiw. 2018. High-Quality Face Capture Using Anatomical Muscles. CoRR abs/1812.02836 (2018). arXiv:1812.02836 http://arxiv.org/abs/1812.02836Google Scholar
- GG Barbarino, M Jabareen, J Trzewik, A Nkengne, G Stamatas, and E Mazza. 2009. Development and validation of a three-dimensional finite element model of the face. Journal of biomechanical engineering 131, 4 (2009), 041006.Google Scholar
Cross Ref
- Vincent Barrielle, Nicolas Stoiber, and Cedric Cagniart. 2016. Blendforces, a Dynamic Framework for Facial Animation. Comp. Graph. Forum (2016). Google Scholar
Digital Library
- Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, and Markus Gross. 2010. High-quality single-shot capture of facial geometry. In ACM Transactions on Graphics (ToG), Vol. 29. ACM, 40. Google Scholar
Digital Library
- Thabo Beeler and Derek Bradley. 2014. Rigid stabilization of facial expressions. ACM Trans. Graph. 33, 4 (2014), 44. Google Scholar
Digital Library
- Thabo Beeler, Fabian Hahn, Derek Bradley, Bernd Bickel, Paul Beardsley, Craig Gotsman, Robert W Sumner, and Markus Gross. 2011. High-quality passive facial performance capture using anchor frames. In ACM Trans. on Graphics (TOG). Google Scholar
Digital Library
- Bernd Bickel, Moritz Bächer, Miguel A Otaduy, Wojciech Matusik, Hanspeter Pfister, and Markus Gross. 2009. Capture and modeling of nonlinear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (2009). Google Scholar
Digital Library
- Bernd Bickel, Peter Kaufmann, Mélina Skouras, Bernhard Thomaszewski, Derek Bradley, Thabo the, Phil Jackson, Steve Marschner, Wojciech Matusik, and Markus Gross. 2012. Physical face cloning. ACM Trans. Graph. 31, 4 (2012), 118. Google Scholar
Digital Library
- Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. of the 26th annual conf. on Comp. graph. and interactive techniques. 187--194. Google Scholar
Digital Library
- Silvia S Blemker, Peter M Pinsky, and Scott L Delp. 2005. A 3D model of muscle reveals the causes of nonuniform strains in the biceps brachii. Journal of biomechanics 38, 4 (2005), 657--665.Google Scholar
Cross Ref
- Xiang Chen, Changxi Zheng, Weiwei Xu, and Kun Zhou. 2014. An asymptotic numerical method for inverse elastic shape design. ACM Trans. Graph. 33, 4 (2014), 95. Google Scholar
Digital Library
- Matthew Cong, Michael Bao, Kiran S Bhat, Ronald Fedkiw, et al. 2015. Fully automatic generation of anatomical face simulation models. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. ACM, 175--183. Google Scholar
Digital Library
- Matthew Cong, Kiran S Bhat, and Ronald Fedkiw. 2016. Art-directed muscle simulation for high-end facial animation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. 119--127. Google Scholar
Digital Library
- Ali-Hamadi Dicko, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure, Olivier Palombi, and Marie-Paule Cani. 2013. Anatomy transfer. ACM Trans. Graph. 32, 6 (2013), 188. Google Scholar
Digital Library
- Paul Ekman and Wallace V Friesen. 1977. Facial action coding system. (1977).Google Scholar
- Benjamin J Ellis, Trevor J Lujan, Michelle S Dalton, and Jeffrey A Weiss. 2006. Medial collateral ligament insertion site and contact forces in the ACL-deficient knee. Journal of Orthopaedic Research 24, 4 (2006), 800--810.Google Scholar
Cross Ref
- Ye Fan, Joshua Litven, David IW Levin, and Dinesh K Pai. 2013. Eulerian-on-lagrangian simulation. ACM Transactions on Graphics (TOG) 32, 3 (2013), 22. Google Scholar
Digital Library
- Ye Fan, Joshua Litven, and Dinesh K Pai. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. (2014). Google Scholar
Digital Library
- François Faure, Christian Duriez, Hervé Delingette, Jérémie Allard, Benjamin Gilles, Stéphanie Marchesseau, Hugo Talbot, Hadrien Courtecuisse, Guillaume Bousquet, Igor Peterlik, et al. 2012. Sofa: A multi-model framework for interactive physical simulation. In Soft Tissue Biomechanical Modeling for Computer Assisted Surgery. Springer, 283--321.Google Scholar
- Andriy Fedorov, Reinhard Beichel, Jayashree Kalpathy-Cramer, Julien Finet, Jean-Christophe Fillion-Robin, Sonia Pujol, Christian Bauer, Dominique Jennings, Fiona Fennessy, Milan Sonka, et al. 2012. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic resonance imaging 30, 9 (2012), 1323--1341.Google Scholar
- Babarenda Gamage, P Thiranja, Vijayaraghavan Rajagopal, Matthias Ehrgott, Martyn P Nash, and Poul MF Nielsen. 2011. Identification of mechanical properties of heterogeneous soft bodies using gravity loading. International journal for numerical methods in biomedical engineering 27, 3 (2011), 391--407.Google Scholar
- Michael W Gee, Ch Förster, and WA Wall. 2010. A computational strategy for prestressing patient-specific biomechanical problems under finite deformation. International Journal for Numerical Methods in Biomedical Engineering 26, 1 (2010), 52--72.Google Scholar
Cross Ref
- Lianghao Han, John H Hipwell, Christine Tanner, Zeike Taylor, Thomy Mertzanidou, Jorge Cardoso, Sebastien Ourselin, and David J Hawkes. 2011. Development of patient-specific biomechanical models for predicting large breast deformation. Physics in Medicine and Biology 57, 2 (2011), 455.Google Scholar
Cross Ref
- Hibbitt, Karlsson, and Sorensen. 2001. ABAQUS/standard User's Manual. Vol. 1. Hibbitt, Karlsson & Sorensen.Google Scholar
- Alexandru Ichim, Ladislav Kavan, Merlin Nimier-David, and Mark Pauly. 2016. Building and Animating User-Specific Volumetric Face Rigs. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Google Scholar
Digital Library
- Alexandru Eugen Ichim, the Bouaziz, and Mark Pauly. 2015. Dynamic 3D Avatar Creation from Hand-held Video Input. ACM Trans. Graph. (2015). Google Scholar
Digital Library
- Alexandru-Eugen Ichim, Petr Kadlecek, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based Face Modeling and Animation. ACM Trans. Graph. 36, 4 (2017). Google Scholar
Digital Library
- Petr Kadlecek, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Krivanek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6 (2016). Google Scholar
Digital Library
- Meekyoung Kim, Gerard Pons-Moll, Sergi Pujades, Seungbae Bang, Jinwook Kim, Michael J Black, and Sung-Hee Lee. 2017. Data-driven physics for human soft tissue animation. ACM Transactions on Graphics (TOG) 36, 4 (2017), 54. Google Scholar
Digital Library
- Yeara Kozlov, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross. 2017. Enriching Facial Blendshape Rigs with Physical Simulation. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 75--84. Google Scholar
Digital Library
- Lana Lan, Matthew Cong, and Ronald Fedkiw. 2017. Lessons from the evolution of an anatomical facial muscle model. In Proceedings of the ACM SIGGRAPH Digital Production Symposium. ACM, 11. Google Scholar
Digital Library
- Rudy J Lapeer, Paul D Gasson, and Vasudev Karri. 2011. A hyperelastic finite-element model of human skin for interactive real-time surgical simulation. IEEE Transactions on Biomedical Engineering 58, 4 (2011), 1013--1022.Google Scholar
Cross Ref
- Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (2009), 99. Google Scholar
Digital Library
- David IW Levin, Benjamin Gilles, Burkhard Mädler, and Dinesh K Pai. 2011. Extracting skeletal muscle fiber fields from noisy diffusion tensor data. Medical Image Analysis 15, 3 (2011), 340--353.Google Scholar
Cross Ref
- John P Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Frederic H Pighin, and Zhigang Deng. 2014. Practice and Theory of Blendshape Facial Models.. In Eurographics (State of the Art Reports). 199--218.Google Scholar
- Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. 2017. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics 36, 6 (Nov. 2017), 194:1--194:17. Two first authors contributed equally. Google Scholar
Digital Library
- John E Lloyd, Ian Stavness, and Sidney Fels. 2012. ArtiSynth: A fast interactive biomechanical modeling toolkit combining multi-body and finite element simulation. In Soft tissue biomechanical modeling for computer assisted surgery. Springer.Google Scholar
- Vincent Luboz, Emmanuel Promayon, and Yohan Payan. 2014. Linear elastic properties of the facial soft tissues using an aspiration device: towards patient specific characterization. Annals of biomedical engineering 42, 11 (2014), 2369--2378.Google Scholar
Cross Ref
- Wan-Chun Ma, Yi-Hua Wang, Graham Fyffe, Bing-Yu Chen, and Paul Debevec. 2012. A blendshape model that incorporates physical interaction. Computer Animation and Virtual Worlds 23, 3-4 (2012). Google Scholar
Digital Library
- Steve A Maas, Benjamin J Ellis, Gerard A Ateshian, and Jeffrey A Weiss. 2012. FEBio: finite elements for biomechanics. Journal of biomechanical engineering 134, 1 (2012), 011005.Google Scholar
Cross Ref
- Steve A Maas, Ahmet Erdemir, Jason P Halloran, and Jeffrey A Weiss. 2016. A general framework for application of prestrain to computational models of biological materials. journal of the mechanical behavior of biomedical materials 61 (2016), 499--510.Google Scholar
- Wouter Mollemans, Filip Schutyser, Nasser Nadjmi, Frederik Maes, and Paul Suetens. 2007. Predicting soft tissue deformations for a maxillofacial surgery planning system: from computational strategies to a complete clinical validation. Medical image analysis 11, 3 (2007), 282--301.Google Scholar
- Mohammad Ali Nazari, Pascal Perrier, Matthieu Chabanas, and Yohan Payan. 2010. Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation. Computer Methods in Biomechanics and Biomedical Engineering 13, 4 (2010), 469--482.Google Scholar
Cross Ref
- Yves Nubar and Renato Contini. 1961. A minimal principle in biomechanics. The bulletin of mathematical biophysics 23, 4 (1961), 377--391.Google Scholar
- U Ozsoy, R Sekerci, and E Ogut. 2015. Effect of sitting, standing, and supine body positions on facial soft tissue: Detailed 3D analysis. International journal of oral and maxillofacial surgery 44, 10 (2015), 1309--1316.Google Scholar
- Dinesh K Pai, Kees van den Doel, Doug L James, Jochen Lang, John E Lloyd, Joshua L Richmond, and Som H Yau. 2001. Scanning physical interaction behavior of 3D objects. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 87--96. Google Scholar
Digital Library
- MB Rubin and SR Bodner. 2002. A three-dimensional nonlinear model for dissipative response of soft tissue. International Journal of Solids and Structures 39, 19 (2002), 5081--5099.Google Scholar
Cross Ref
- Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational Bodybuilding: Anatomically-based Modeling of Human Bodies. ACM Trans. Graph. 34, 4 (2015). Google Scholar
Digital Library
- Olaf Schenk and Klaus Gärtner. 2004. Solving unsymmetric sparse systems of linear equations with PARDISO. Future Generation Computer Systems 20, 3 (2004), 475--487. Google Scholar
Digital Library
- Minoru Shinohara, Karim Sabra, Jean-Luc Gennisson, Mathias Fink, and Mickaél Tanter. 2010. Real-time visualization of muscle stiffness distribution with ultrasound shear wave imaging during muscle contraction. Muscle & nerve (2010).Google Scholar
- Eftychios Sifakis and Jernej Barbic. 2012. FEM simulation of 3D deformable solids: a practitioner's guide to theory, discretization and model reduction. In ACM SIGGRAPH 2012 Courses. 20. Google Scholar
Digital Library
- Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM Trans. Graph., Vol. 24. 417--425. Google Scholar
Digital Library
- Alex Smith, Sven Pohle, Wan-Chun Ma, Chongyang Ma, Xian-Chun Wu, Yanbing Chen, Etienne Danvoye, Jorge Jimenez, Sanjit Patel, Mike Sanders, and Cyrus A. Wilson. 2017. Emotion Challenge: Building a New Photoreal Facial Performance Pipeline for Games. In Proceedings of the ACM SIGGRAPH Digital Production Symposium (DigiPro '17). ACM, New York, NY, USA, Article 8, 2 pages. Google Scholar
Digital Library
- Ian Stavness, Mohammad Ali Nazari, Cormac Flynn, Pascal Perrier, Yohan Payan, John E Lloyd, and Sidney Fels. 2014. Coupled biomech. modeling of the face, jaw, skull, tongue, and hyoid bone. In 3D Multiscale Physiological Human. Springer. Google Scholar
Digital Library
- Joseph Teran, Sylvia Blemker, V Hing, and Ronald Fedkiw. 2003. Finite volume methods for the simulation of skeletal muscle. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Eurographics Association, 68--74. Google Scholar
Digital Library
- Joseph Teran, Eftychios Sifakis, Silvia S Blemker, Victor Ng-Thow-Hing, Cynthia Lau, and Ronald Fedkiw. 2005a. Creating and simulating skeletal muscle from the visible human data set. Vis. and Computer Graphics, IEEE Trans. on (2005). Google Scholar
Digital Library
- Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005b. Robust quasistatic finite elements and flesh simulation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Google Scholar
Digital Library
- Demetri Terzopoulos and Keith Waters. 1990. Physically-based facial modelling, analysis, and animation. Computer Animation and Virtual Worlds 1, 2 (1990), 73--80.Google Scholar
- HV Tran, F Charleux, M Rachik, Alain Ehrlacher, and MC Ho Ba Tho. 2007. In vivo characterization of the mech. properties of human skin derived from MRI and indentation techniques. Comp. methods in biomech. and biomed. eng. 10, 6 (2007), 401--407.Google Scholar
- Javier von der Pahlen, Jorge Jimenez, Etienne Danvoye, Paul Debevec, Graham Fyffe, and Oleg Alexander. 2014. Digital Ira and Beyond: Creating Real-time Photoreal Digital Actors. In ACM SIGGRAPH 2014 Courses (SIGGRAPH '14). Google Scholar
Digital Library
- Andreas Wächter and Lorenz T Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006). Google Scholar
Digital Library
- Bin Wang, Longhua Wu, KangKang Yin, Uri Ascher, Libin Liu, and Hui Huang. 2015. Deformation capture and modeling of soft objects. ACM Trans. Graph. 34, 4 (2015), 94. Google Scholar
Digital Library
- Jeffrey A Weiss, Bradley N Maker, and Sanjay Govindjee. 1996. Finite element implementation of incompressible, transversely isotropic hyperelasticity. Computer methods in applied mechanics and engineering 135, 1 (1996), 107--128.Google Scholar
- Tim Wu. 2013. A computational framework for modelling the biomechanics of human facial expressions. Ph.D. Dissertation. ResearchSpace@ Auckland.Google Scholar
- Wenwu Yang, Nathan Marshak, Daniel Sýkora, Srikumar Ramalingam, and Ladislav Kavan. 2018. Building Anatomically Realistic Jaw Kinematics Model from Data. CoRR abs/1805.05903 (2018). arXiv:1805.05903 http://arxiv.org/abs/1805.05903 Google Scholar
Digital Library
- Seung-Hyun Yoon, John Lewis, and Taehyun Rhee. 2017. Blending Face Details: Synthesizing a Face Using Multiscale Face Models. IEEE computer graphics and applications 37, 6 (2017), 65--75.Google Scholar
- Eduard Zell, JP Lewis, Junyong Noh, Mario Botsch, et al. 2017. Facial retargeting with automatic range of motion alignment. ACM Transactions on Graphics (TOG) 36, 4 (2017), 154. Google Scholar
Digital Library
- Gaspard Zoss, Derek Bradley, Pascal Bérard, and Thabo Beeler. 2018. An empirical rig for jaw animation. ACM Transactions on Graphics (TOG) 37, 4 (2018), 59. Google Scholar
Digital Library
Index Terms
Building Accurate Physics-based Face Models from Data
Recommendations
Learning an animatable detailed 3D face model from in-the-wild images
While current monocular 3D face reconstruction methods can recover fine geometric details, they suffer several limitations. Some methods produce faces that cannot be realistically animated because they do not model how wrinkles vary with expression. ...
Physical face cloning
We propose a complete process for designing, simulating, and fabricating synthetic skin for an animatronics character that mimics the face of a given subject and its expressions. The process starts with measuring the elastic properties of a material ...
Face transfer with multilinear models
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersFace Transfer is a method for mapping videorecorded performances of one individual to facial animations of another. It extracts visemes (speech-related mouth articulations), expressions, and three-dimensional (3D) pose from monocular video or film ...






Comments