skip to main content
research-article

“Wow! You Are So Beautiful Today!”

Published:01 October 2014Publication History
Skip Abstract Section

Abstract

Beauty e-Experts, a fully automatic system for makeover recommendation and synthesis, is developed in this work. The makeover recommendation and synthesis system simultaneously considers many kinds of makeover items on hairstyle and makeup. Given a user-provided frontal face image with short/bound hair and no/light makeup, the Beauty e-Experts system not only recommends the most suitable hairdo and makeup, but also synthesizes the virtual hairdo and makeup effects. To acquire enough knowledge for beauty modeling, we built the Beauty e-Experts Database, which contains 1,505 female photos with a variety of attributes annotated with different discrete values. We organize these attributes into two different categories, beauty attributes and beauty-related attributes. Beauty attributes refer to those values that are changeable during the makeover process and thus need to be recommended by the system. Beauty-related attributes are those values that cannot be changed during the makeup process but can help the system to perform recommendation. Based on this Beauty e-Experts Dataset, two problems are addressed for the Beauty e-Experts system: what to recommend and how to wear it, which describes a similar process of selecting hairstyle and cosmetics in daily life. For the what-to-recommend problem, we propose a multiple tree-structured supergraph model to explore the complex relationships among high-level beauty attributes, mid-level beauty-related attributes, and low-level image features. Based on this model, the most compatible beauty attributes for a given facial image can be efficiently inferred. For the how-to-wear-it problem, an effective and efficient facial image synthesis module is designed to seamlessly synthesize the recommended makeovers into the user facial image. We have conducted extensive experiments on testing images of various conditions to evaluate and analyze the proposed system. The experimental results well demonstrate the effectiveness and efficiency of the proposed system.

References

  1. Timo Ahonen, Abdenour Hadid, and Matti. Pietikainen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. K. Aucoin. 2000. Face Forward. Little, Brown Company.Google ScholarGoogle Scholar
  3. Serge Belongie, Jitendra Malik, and Jan Puzicha. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mark Berg, Otfried Cheong, Marc van Kreveld, and Mark Overmars. 2008. Computational Geometry: Algorithms and Applications (3rd ed.). Springer-Verlag. Google ScholarGoogle ScholarCross RefCross Ref
  5. Fred Bookstein. 1989. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yuri Boykov and Olga Veksler. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Fangmei Chen and David Zhang. 2010. A benchmark for geometric facial beauty study. In Proceedings of the International Conference on Medical Biometrics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Chow and C. Liu. 1968. Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Timothy Cootes, Christopher Taylor, David Cooper, and Jim Graham. 1995. Active shape models-their training and application. Comput. Vision Image Understand. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ardeshir Goshtasby. 1986. Piecewise linear mapping functions for image registration. Pattern Recogn. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Dong Guo and Terence Sim. 2009. Digital face makeup by example. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  16. Simon Haykin. 1999. Neural Networks. Prentice Hall.Google ScholarGoogle Scholar
  17. Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Guided image filtering. In Proceedings of the European Conference on Computer Vision. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Gary Huang, Manu Ramesh, Tamara Berg, and Erik Learned. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report. University of Massachusetts.Google ScholarGoogle Scholar
  19. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ian Jolliffe. 2002. Principal Component Analysis. In Encyclopedia of Statistics in Behavioral Science, Springer.Google ScholarGoogle Scholar
  21. Michael J. Jones and James M. Rehg. 1999. Statistical color models with application to skin detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  22. Daphne Koller and Nir Friedman. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Anat Levin, Alex Rav-Acha, and Dani Lischinski. 2007. Spectral matting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle ScholarCross RefCross Ref
  24. Luoqi Liu, Hui Xu, Junliang Xing, Si Liu, Xi Zhou, and Shuicheng. Yan. 2013. “Wow! You are so beautiful today!” In Proceedings of the 21st ACM International Conference on ACM Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Si Liu, Jiashi Feng, Zheng Song, Tianzhu Zhang, Hanqing Lu, Changsheng Xu, and Shuicheng Yan. 2012. “Hi, magic closet, tell me what to wear”. In Proceedings of the 20th ACM International Conference on ACM Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. T. Mensink, J. Verbeek, and G. Csurka. 2013. Tree-structured CRF Models for interactive image labeling. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Yuki Nagai, Kuniko Ushiro, Yoshiro Matsunami, Tsuyoshi Hashimoto, and Yuusuke Kojima. 2005. Hairstyle suggesting system, hairstyle suggesting method, and computer program product. U.S. Patent US20050251463 A1. Filed May 5, 2005; Accepted November 10, 2005.Google ScholarGoogle Scholar
  28. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kristina Scherbaum, Tobias Ritschel, Matthias Hullin, Thorsten Thormählen, Volker Blanz, and Hans Seidel. 2011. Computer-suggested facial makeup. Comput. Graph. Forum 30, 2.Google ScholarGoogle ScholarCross RefCross Ref
  30. Behjat Siddiquie, Rogério Schmidt Feris, and Larry Davis. 2011. Image ranking and retrieval based on multi-attribute queries. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Wai Tong, Chi Tang, Michael Brown, and Ying Xu. 2007. Example-based cosmetic transfer. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition.Google ScholarGoogle ScholarCross RefCross Ref
  32. Martin J. Wainwright and Michael I. Jordan. 2008. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Nan Wang, Haizhou Ai, and Feng Tang. 2012. What are good parts for hair shape modeling? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Yang Wang and Greg Mori. 2010. A discriminative latent model of object classes and attributes. In Proceedings of the European Conference on Computer Vision. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. 2009. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Proceedings of the Neural Information Processing Systems Conference.Google ScholarGoogle Scholar
  36. Yi Yang and Deva Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. “Wow! You Are So Beautiful Today!”

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!