Mariana Dias
ABSTRACT
We propose a fully automated method for realistic 3D face reconstruction from a single frontal photo that produces a high-resolution head mesh and a diffuse map. The photo is input to a convolutional neural network that estimates the weights of a morphable model to produce an initial head shape that is further adjusted through landmark-guided deformation. Two key features of the method are: 1) the network is exclusively trained on synthetic photos that are photo-realistic enough to learn real shape predictive features, making it unnecessary to train with real facial photos and corresponding 3D scans; 2) the landmarking statistical errors are incorporated in the reconstruction for optimal accuracy. While the method is based on a limited amount of real data, we show that it robustly and quickly performs plausible face reconstructions from real photos.
Supplemental Material
- V. Blanz and T. Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH’99. ACM Press/Addison-Wesley Publishing Co., USA, 187–194.Google Scholar
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proc. CVPR’09. 248–255.Google Scholar
- G. Huang, Z. Liu, L. Van Der Maaten, and K.Q. Weinberger. 2017. Densely connected convolutional networks. In Proc. CVPR’20. 4700–4708.Google Scholar
- V. Kazemi and J. Sullivan. 2014. One millisecond face alignment with an ensemble of regression trees. In Proc. CVPR’14. 1867–1874.Google Scholar
- R. Li, K. Bladin, Y. Zhao, C. Chinara, O. Ingraham, P. Xiang, X. Ren, P. Prasad, B. Kishore, J. Xing, and H. Li. 2020. Learning formation of physically-based face attributes. In Proc. CVPR’20. 3410–3419.Google Scholar
- R. Wang, C.-F. Chen, H. Peng, X. Liu, O. Liu, and X. Li. 2019. Digital Twin: Acquiring High-Fidelity 3D Avatar from a Single Image. Technical Report. arxiv:1912.03455Google Scholar
- E. Wood, T. Baltrušaitis, C. Hewitt, S. Dziadzio, T.J. Cashman, and J. Shotton. 2021. Fake It Till You Make It: Face analysis in the wild using synthetic data alone. In Proc. IEEE International Conference on Computer Vision. 3681–3691.Google Scholar
Recommendations
High-quality hair modeling from a single portrait photo
We propose a novel system to reconstruct a high-quality hair depth map from a single portrait photo with minimal user input. We achieve this by combining depth cues such as occlusions, silhouettes, and shading, with a novel 3D helical structural prior ...
Photo-realistic facial expression synthesis
This paper details a procedure for generating a function which maps an image of a neutral face to one depicting a desired expression independent of age, sex, or skin colour. Facial expression synthesis is a growing and relatively new domain within ...
Comments