Jonathan Swartz
ABSTRACT
In visual effects for film, replacement of stunt performers’ facial likeness for their doubled actor counterparts using traditional computer graphics methods is a multi-stage, labor intensive task. Recently, deep learning techniques have made a compelling argument to train neural networks to learn how to take an image of a person’s face and convincingly infer a rendered image of a second person’s face with a previously unseen perspective, pose and lighting environment.
A novel method is discussed for bringing deep neural network face swapping to feature film production which utilizes facial recognition for the discovery of training data. Our method further innovates in the area of utilizing traditional CG assets for informing some of the shortcomings of ML techniques. Connected with a technique for feature engineering during training dataset assembly, our Face Fabrication System enables Wētā FX to deliver final picture quality for use in production.
- Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Z. Li, and Xudong Zou. 2018. Selective Refinement Network for High Performance Face Detection. arXiv:1809.02693 [cs] (Sept. 2018). http://arxiv.org/abs/1809.02693 arXiv:1809.02693.Google Scholar
- Github Deepfakes. 2021. FaceSwap. https://github.com/deepfakes/faceswap Accessed 2021-02-21.Google Scholar
- I. Korshunova, W. Shi, J. Dambre, and L. Theis. 2017. Fast Face-Swap Using Convolutional Neural Networks. In 2017 IEEE International Conference on Computer Vision (ICCV). 3697–3705. https://doi.org/10.1109/ICCV.2017.397 ISSN: 2380-7504.Google Scholar
- Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, 2020. DeepFaceLab: A simple, flexible and extensible face swapping framework. arXiv:2005.05535 [cs, eess] (May 2020). http://arxiv.org/abs/2005.05535Google Scholar
- Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition, 815–823. https://doi.org/10.1109/CVPR.2015.7298682Google Scholar
- Jing Yang, Adrian Bulat, and Georgios Tzimiropoulos. 2020. FAN-Face: a Simple Orthogonal Improvement to Deep Face Recognition. Proceedings of the AAAI Conference on Artificial Intelligence 34, 07, 12621–12628. https://doi.org/10.1609/aaai.v34i07.6953Google ScholarCross Ref
Recommendations
Improving Deep Learning Feature with Facial Texture Feature for Face Recognition
Face recognition in the reality, is a challenging problem, due to varieties in illumination, background, pose etc. Recently, the deep learning based face recognition algorithm is able to learn effective face features to obtain a very impressive ...
Automatic Face Recognition for Film Character Retrieval in Feature-Length Films
CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01The objective of this work is to recognize all the frontal faces of a character in the closed world of a movie or situation comedy, given a small number of query faces. This is challenging because faces in a feature-length film are relatively ...
Illumination-robust face recognition based on Gabor feature face intrinsic identity PCA model
CIMMACS'08: Proceedings of the 7th WSEAS international conference on Computational intelligence, man-machine systems and cyberneticsRobust face recognition under various illumination environments is essential for successful commercialization. Feature-based face recognition relies on a good choice of feature vectors. However, there is no feature vector invariant under illumination ...
Comments