skip to main content
research-article

Virtual Portraitist: An Intelligent Tool for Taking Well-Posed Selfies

Published:24 January 2019Publication History
Skip Abstract Section

Abstract

Smart photography carries the promise of quality improvement and functionality extension in making aesthetically appealing pictures. In this article, we focus on self-portrait photographs and introduce new methods that guide a user in how to best pose while taking a selfie. While most of the current solutions use a post processing procedure to beautify a picture, the developed tool enables a novel function of recommending a good look before the photo is captured. Given an input face image, the tool automatically estimates the pose-based aesthetic score, finds the most attractive angle of the face, and suggests how the pose should be adjusted. The recommendation results are determined adaptively to the appearance and initial pose of the input face. We apply a data mining approach to find distinctive, frequent itemsets and association rules from online profile pictures, upon which the aesthetic estimation and pose recommendation methods are developed. A simulated and a real image set are used for experimental evaluation. The results show the proposed aesthetic estimation method can effectively select user-favorable photos. Moreover, the recommendation performance for the vertical adjustment is moderately related to the degree of conformity among the professional photographers’ recommendations. This study echoes the trend of instant photo sharing, in which a user takes a picture and then immediately shares it on a social network without engaging in tedious editing.

References

  1. R. Agrawaland R. Srikant. 1994. Fast algorithms for mining association rules in large databases. In International Conference on Very Large Data Bases. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Agrawal, T. Imielinski, and A. N. Swami. 1993. Mining association rules between sets of items in large databases. In ACM International Conference on SIGMOD. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. Bottino and A. Laurentini. 2010. The analysis of facial beauty: An emerging area of research in pattern analysis. Image Analysis and Recognition 6111 (2010), 425--435. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman. 2018. VGGFace2: A dataset for recognising faces across pose and age. In IEEE Conference on Automatic Face and Gesture Recognition.Google ScholarGoogle Scholar
  5. Y. Eisenthal, G. Dror, and E. Ruppin. 2006. Facial attractiveness: Beauty and the machine. Neural Computing 18 (2006), 119--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. J. L. Fleiss and J. Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement 33 (1973), 613--619.Google ScholarGoogle ScholarCross RefCross Ref
  7. B. J. Frey and D. Dueck. 2007. Clustering by passing messages between data points. Science 315, 5814 (2007), 972--976.Google ScholarGoogle Scholar
  8. P. Frosh. 2015. The gestural image: The selfie, photography theory, and kinesthetic sociability. International Journal of Communication 9 (2015). https://ijoc.org/index.php/ijoc/rt/captureCite/3146/1388/BibtexCitationPlugin.Google ScholarGoogle Scholar
  9. M. Gygli, H. Grabner, H. Riemenschneider, and L. V. Gool. 2014. Creating summaries from user videos. In European Conference on Computer Vision.Google ScholarGoogle Scholar
  10. T. Horprasert, Y. Yacoob, and L. Davis. 1996. Computing 3-D head orientation from a monocular image sequence. In IEEE International Conference on Automatic Face and Gesture Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Y. T. Hsieh and M. Yeh. 2017. Head pose recommendation for taking good selfies. In ACM International Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Y. Hu, L. Manikonda, and S. Kambhampati. 2014. What We Instagram: A first analysis of Instagram photo content and user types. In International AAAI Conference on Web and Social Media.Google ScholarGoogle Scholar
  13. Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49. University of Massachusetts, Amherst.Google ScholarGoogle Scholar
  14. A. Kagian, G. Dror, T. Leyvand, I. Meilijson, D. Cohen-Or, and E. Ruppin. 2008. A machine learning predictor of facial attractiveness revealing human-like psychophysical biases. Vision Research 48, 2 (2008), 235--243.Google ScholarGoogle ScholarCross RefCross Ref
  15. M. M. Kalayeh, M. Seifu, W. LaLanne, and M. Shah. 2015. How to take a good selfie? In ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Khosla, R. Hamid, C. Lin, and N. Sundaresan. 2013. Large-scale video summarization using web-image priors. In IEEE International Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. K. Koscinski. 2009. Current status and future directions of research on facial attractiveness. Anthropological Review 72 (2009), 45--75.Google ScholarGoogle ScholarCross RefCross Ref
  18. A. Laurentini and A. Bottino. 2014. Computer analysis of face beauty: A survey. Computer Vision and Image Understanding 125 (2014), 184--199.Google ScholarGoogle ScholarCross RefCross Ref
  19. Q. Li and D. Vogel. 2017. Guided selfies using models of portrait aesthetics. In ACM Conference on Designing Interactive Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. H. B. Mann and D. R. Whitney. 1947. On a Test of whether One of Two Random Variables is Stochastically Larger than the Other. Annals of Mathematical Statistics.Google ScholarGoogle ScholarCross RefCross Ref
  21. E. Murphy-Chutorian and M. M. Trivedi. 2009. Head pose estimation in computer vision: A survey. IEEE Trans. Pattern Analysis and Machine Intelligence 31, 4 (2009), 607--626. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. T. V. Nguyen, S. Liu, B. Ni, J. Tan, Y. Rui, and S. Yan. 2013. Towards decrypting attractiveness via multi-modality cues. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 4 (2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. P. M. Pallett, S. Link, and K. Lee. 2010. New golden ratios for facial beauty. Vision Research 50, 2 (2010), 149--154.Google ScholarGoogle ScholarCross RefCross Ref
  24. T. Quack, V. Ferrari, B. Leibe, and L. V. Gool. 2007. Efficient mining of frequent and distinctive feature configurations. In IEEE International Conference on Computer Vision.Google ScholarGoogle Scholar
  25. G. Rhodes. 2006. The evolutionary psychology of facial beauty. Annual Review of Psychology 57 (2006), 199--226.Google ScholarGoogle ScholarCross RefCross Ref
  26. K. Sayood. 2012. Introduction to Data Compression, 4th Edition. Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. G. Tzimiropoulos and M. Pantic. 2013. Optimization problems for fast AAM fitting in-the-wild. In IEEE International Conference on Computer Vision. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jessica M. Utts and Robert F. Heckard. 2006. Statistical Ideas and Methods. Thomson Brooks/Cole.Google ScholarGoogle Scholar
  29. D. R. Valenzano, A. Mennucci, G. Tartarelli, and A. Cellerino. 2006. Shape analysis of female facial attractiveness. Vision Research 46 (2006), 1282--1291.Google ScholarGoogle ScholarCross RefCross Ref
  30. Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, and Keying Ye. 2010. Probability and Statistics for Engineers and Scientists. Prentice Hall.Google ScholarGoogle Scholar
  31. S. Wang, M. Shao, and Y. Fu. 2014. Attractive or not? Beauty prediction with attractiveness-aware encoders and robust late fusion. In ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics Bulletin 1, 6 (1945), 80--83.Google ScholarGoogle ScholarCross RefCross Ref
  33. X. Xiong and F. De la Torre. 2013. Supervised descent method and its applications to face alignment. In IEEE International Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Yeh and H. W. Lin. 2014. Virtual portraitist: Aesthetic evaluation of selfies based on angle. In ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Virtual Portraitist: An Intelligent Tool for Taking Well-Posed Selfies

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 1s
          Special Section on Deep Learning for Intelligent Multimedia Analytics and Special Section on Multi-Modal Understanding of Social, Affective and Subjective Attributes of Data
          January 2019
          265 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3309769
          Issue’s Table of Contents

          Copyright © 2019 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 24 January 2019
          • Revised: 1 October 2018
          • Accepted: 1 October 2018
          • Received: 1 March 2018
          Published in tomm Volume 15, Issue 1s

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!