Abstract
Beauty e-Experts, a fully automatic system for makeover recommendation and synthesis, is developed in this work. The makeover recommendation and synthesis system simultaneously considers many kinds of makeover items on hairstyle and makeup. Given a user-provided frontal face image with short/bound hair and no/light makeup, the Beauty e-Experts system not only recommends the most suitable hairdo and makeup, but also synthesizes the virtual hairdo and makeup effects. To acquire enough knowledge for beauty modeling, we built the Beauty e-Experts Database, which contains 1,505 female photos with a variety of attributes annotated with different discrete values. We organize these attributes into two different categories, beauty attributes and beauty-related attributes. Beauty attributes refer to those values that are changeable during the makeover process and thus need to be recommended by the system. Beauty-related attributes are those values that cannot be changed during the makeup process but can help the system to perform recommendation. Based on this Beauty e-Experts Dataset, two problems are addressed for the Beauty e-Experts system: what to recommend and how to wear it, which describes a similar process of selecting hairstyle and cosmetics in daily life. For the what-to-recommend problem, we propose a multiple tree-structured supergraph model to explore the complex relationships among high-level beauty attributes, mid-level beauty-related attributes, and low-level image features. Based on this model, the most compatible beauty attributes for a given facial image can be efficiently inferred. For the how-to-wear-it problem, an effective and efficient facial image synthesis module is designed to seamlessly synthesize the recommended makeovers into the user facial image. We have conducted extensive experiments on testing images of various conditions to evaluate and analyze the proposed system. The experimental results well demonstrate the effectiveness and efficiency of the proposed system.
- Timo Ahonen, Abdenour Hadid, and Matti. Pietikainen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- K. Aucoin. 2000. Face Forward. Little, Brown Company.Google Scholar
- Serge Belongie, Jitendra Malik, and Jan Puzicha. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- Mark Berg, Otfried Cheong, Marc van Kreveld, and Mark Overmars. 2008. Computational Geometry: Algorithms and Applications (3rd ed.). Springer-Verlag. Google Scholar
Cross Ref
- Fred Bookstein. 1989. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- Yuri Boykov and Olga Veksler. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. Google Scholar
Digital Library
- Fangmei Chen and David Zhang. 2010. A benchmark for geometric facial beauty study. In Proceedings of the International Conference on Medical Biometrics. Google Scholar
Digital Library
- C. Chow and C. Liu. 1968. Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory. Google Scholar
Digital Library
- Timothy Cootes, Christopher Taylor, David Cooper, and Jim Graham. 1995. Active shape models-their training and application. Comput. Vision Image Understand. Google Scholar
Digital Library
- Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. Google Scholar
Digital Library
- Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google Scholar
Digital Library
- P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- Ardeshir Goshtasby. 1986. Piecewise linear mapping functions for image registration. Pattern Recogn. Google Scholar
Digital Library
- Dong Guo and Terence Sim. 2009. Digital face makeup by example. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Simon Haykin. 1999. Neural Networks. Prentice Hall.Google Scholar
- Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Guided image filtering. In Proceedings of the European Conference on Computer Vision. Google Scholar
Digital Library
- Gary Huang, Manu Ramesh, Tamara Berg, and Erik Learned. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report. University of Massachusetts.Google Scholar
- Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Google Scholar
Digital Library
- Ian Jolliffe. 2002. Principal Component Analysis. In Encyclopedia of Statistics in Behavioral Science, Springer.Google Scholar
- Michael J. Jones and James M. Rehg. 1999. Statistical color models with application to skin detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Daphne Koller and Nir Friedman. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press. Google Scholar
Digital Library
- Anat Levin, Alex Rav-Acha, and Dani Lischinski. 2007. Spectral matting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- Luoqi Liu, Hui Xu, Junliang Xing, Si Liu, Xi Zhou, and Shuicheng. Yan. 2013. “Wow! You are so beautiful today!” In Proceedings of the 21st ACM International Conference on ACM Multimedia. Google Scholar
Digital Library
- Si Liu, Jiashi Feng, Zheng Song, Tianzhu Zhang, Hanqing Lu, Changsheng Xu, and Shuicheng Yan. 2012. “Hi, magic closet, tell me what to wear”. In Proceedings of the 20th ACM International Conference on ACM Multimedia. Google Scholar
Digital Library
- T. Mensink, J. Verbeek, and G. Csurka. 2013. Tree-structured CRF Models for interactive image labeling. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- Yuki Nagai, Kuniko Ushiro, Yoshiro Matsunami, Tsuyoshi Hashimoto, and Yuusuke Kojima. 2005. Hairstyle suggesting system, hairstyle suggesting method, and computer program product. U.S. Patent US20050251463 A1. Filed May 5, 2005; Accepted November 10, 2005.Google Scholar
- Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. Google Scholar
Digital Library
- Kristina Scherbaum, Tobias Ritschel, Matthias Hullin, Thorsten Thormählen, Volker Blanz, and Hans Seidel. 2011. Computer-suggested facial makeup. Comput. Graph. Forum 30, 2.Google Scholar
Cross Ref
- Behjat Siddiquie, Rogério Schmidt Feris, and Larry Davis. 2011. Image ranking and retrieval based on multi-attribute queries. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google Scholar
Digital Library
- Wai Tong, Chi Tang, Michael Brown, and Ying Xu. 2007. Example-based cosmetic transfer. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition.Google Scholar
Cross Ref
- Martin J. Wainwright and Michael I. Jordan. 2008. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning. Google Scholar
Digital Library
- Nan Wang, Haizhou Ai, and Feng Tang. 2012. What are good parts for hair shape modeling? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google Scholar
Digital Library
- Yang Wang and Greg Mori. 2010. A discriminative latent model of object classes and attributes. In Proceedings of the European Conference on Computer Vision. Google Scholar
Digital Library
- John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. 2009. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Proceedings of the Neural Information Processing Systems Conference.Google Scholar
- Yi Yang and Deva Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google Scholar
Digital Library
Index Terms
“Wow! You Are So Beautiful Today!”
Recommendations
"Wow! you are so beautiful today!"
MM '13: Proceedings of the 21st ACM international conference on MultimediaBeauty e-Experts, a fully automatic system for hairstyle and facial makeup recommendation and synthesis, is developed in this work. Given a user-provided frontal face image with short/bound hair and no/light makeup, the Beauty e-Experts system can not ...
"Wow! you are so beautiful today!"
MM '13: Proceedings of the 21st ACM international conference on MultimediaIn this demo, we present Beauty e-Experts, a fully automatic system for hairstyle and facial makeup recommendation and synthesis. Given a user-provided frontal facial image with short/bound hair and no/light makeup, the Beauty e-Experts system can not ...
The Beautiful Encounters
ARTECH 2021: 10th International Conference on Digital and Interactive ArtsThis paper introduces an art project called “The Beautiful Encounters” that resonates with the surrealist painter Rene Magritte's work and aims to introduce an interactive scenario as a case study that applies AI technology. The article's focus lies on ...






Comments