Human Perception of Visual Realism for Photo and Computer-Generated Face Images

Abstract

Computer-generated (CG) face images are common in video games, advertisements, and other media. CG faces vary in their degree of realism, a factor that impacts viewer reactions. Therefore, efficient control of visual realism of face images is important. Efficient control is enabled by a deep understanding of visual realism perception: the extent to which viewers judge an image as a real photograph rather than a CG image. Across two experiments, we explored the processes involved in visual realism perception of face images. In Experiment 1, participants made visual realism judgments on original face images, inverted face images, and images of faces that had the top and bottom halves misaligned. In Experiment 2, participants made visual realism judgments on original face images, scrambled faces, and images that showed different parts of faces. Our findings indicate that both holistic and piecemeal processing are involved in visual realism perception of faces, with holistic processing becoming more dominant when resolution is lower. Our results also suggest that shading information is more important than color for holistic processing, and that inversion makes visual realism judgments harder for realistic images but not for unrealistic images. Furthermore, we found that eyes are the most influential face part for visual realism, and face context is critical for evaluating realism of face parts. To the best of our knowledge, this work is a first realism-centric study attempting to bridge the human perception of visual realism on face images with general face perception tasks.

References

1. Rama Amishav and Ruth Kimchi. 2010. Perceptual integrality of componential and configural information in faces. Psychonomic Bulletin and Review 17, 5, 743--748.
2. Rosemary A. Bailey. 2008. Design of Comparative Experiments. Vol. 25. Cambridge University Press.
3. Michael J. Bernstein, Steven G. Young, and Kurt Hugenberg. 2007. The cross-category effect mere social categorization is sufficient to elicit an own-group bias in face recognition. Psychological Science 18, 8, 706--712.
4. Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99). 187--194.
5. Elizabeth Brown and D. Perrett. 1993. What gives a face its gender. Perception 22, 829--840.
6. Vicki Bruce and Andy Young. 2012. Face Perception. Psychology Press.
7. Cindy M. Bukach, Isabel Gauthier, and Michael J. Tarr. 2006. Beyond faces and modularity: The power of an expertise framework. Trends in Cognitive Sciences 10, 4, 159--166.
8. Roberto Cabeza and Takashi Kato. 2000. Features are also important: Contributions of featural and configural processing to face recognition. Psychological Science 11, 5, 429--433.
9. Marco Cecchini, Paola Aceto, Daniela Altavilla, Letizia Palumbo, and Carlo Lai. 2013. The role of the eyes in processing an intact face and its scrambled image: A dense array ERP and low-resolution electromagnetic tomography (sLORETA) study. Social Neuroscience 8, 4, 314--325.
10. Stephan M. Collishaw and Graham J. Hole. 2000. Featural and configurational processes in the recognition of faces of different familiarity. Perception 29, 8, 893--910.
11. Kate Crookes, Simone Favelle, and William G. Hayward. 2013. Holistic processing for other-race faces in Chinese participants occurs for upright but not inverted faces. Frontiers in Psychology 4, 29.
12. Zhigang Deng and Xiaohan Ma. 2008. Perceptually guided expressive facial animation. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 67--76.
13. Karine Durand, Mathieu Gallay, Alix Seigneuric, Fabrice Robichon, and Jean-Yves Baudouin. 2007. The development of facial emotion recognition: The role of configural information. Journal of Experimental Child Psychology 97, 1, 14--27.
14. Shaojing Fan, Tian-Tsong Ng, Jonathan S. Herberg, Bryan L. Koenig, and Shiqing Xin. 2012. Real or Fake?: Human judgments about photographs and computer-generated images of faces. In SIGGRAPH Asia 2012 Technical Briefs (SA'12). Article 17.
15. Martha J. Farah. 1996. Is face recognition special? Evidence from neuropsychology. Behavioural Brain Research 76, 1, 181--189.
16. Martha J. Farah, Kevin D. Wilson, Maxwell Drain, and James N. Tanaka. 1998. What is “special” about face perception? Psychological Review 105, 3, 482.
17. Hany Farid and Mary J. Bravo. 2012. Perceptual discrimination of computer generated and photographic faces. Digital Investigation 8, 3, 226--235.
18. Simone Favelle and Stephen Palmisano. 2012. The face inversion effect following pitch and yaw rotations: Investigating the boundaries of holistic processing. Frontiers in Psychology 3, 563.
19. Francis Galton. 1883. Inquiries into Human Faculty and Its Development. Macmillan and Company.
20. Zaifeng Gao and Shlomo Bentin. 2011. Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay. Journal of Experimental Psychology: Human Perception and Performance 37, 4, 1051.
21. Elena Garces, Adolfo Munoz, Jorge Lopez-Moreno, and Diego Gutierrez. 2012. Intrinsic images by clustering. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 1415--1424.
22. Isabel Gauthier and Michael J. Tarr. 1997. Becoming a “Greeble” expert: Exploring mechanisms for face recognition. Vision Research 37, 12, 1673--1682.
23. Sharon Gilad, Ming Meng, and Pawan Sinha. 2009. Role of ordinal contrast relationships in face encoding. Proceedings of the National Academy of Sciences 106, 13, 5353--5358.
24. Valérie Goffaux, Judith Peters, Julie Haubrechts, Christine Schiltz, Bernadette Jansma, and Rainer Goebel. 2011. From coarse to fine? Spatial and temporal dynamics of cortical face processing. Cerebral Cortex 21, 2, 467--476.
25. Valérie Goffaux and Bruno Rossion. 2006. Faces are “spatial”--holistic face perception is supported by low spatial frequencies. Journal of Experimental Psychology: Human Perception and Performance 32, 4, 1023.
26. Valérie Goffaux, Christine Schiltz, Marieke Mur, and Rainer Goebel. 2013. Local discriminability determines the strength of holistic processing for faces in the fusiform face area. Frontiers in Psychology 3, 604.
27. Alexandra J. Golby, John D. E. Gabrieli, Joan Y. Chiao, and Jennifer L. Eberhardt. 2001. Differential responses in the fusiform region to same-race and other-race faces. Nature Neuroscience 4, 8, 845--850.
28. Jason M. Gold, Patrick J. Mundy, and Bosco S. Tjan. 2012. The perception of a face is no more than the sum of its parts. Psychological Science 23, 4, 427--434.
29. Alexander N. Gorban, Elena V. Smirnova, and Tatiana A. Tyukina. 2010. Correlations, risk and crisis: From physiology to finance. Physica A: Statistical Mechanics and Its Applications 389, 16, 3193--3217.
30. Roger Grosse, Micah K. Johnson, Edward H. Adelson, and William T. Freeman. 2009. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In Proceedings of the IEEE International Conference on Computer Vision. 2335--2342.
31. Peter J. B. Hancock, Vicki Bruce, and A. Mike Burton. 2000. Recognition of unfamiliar faces. Trends in Cognitive Sciences 4, 9, 330--337.
32. James V. Haxby, Elizabeth A. Hoffman, and M. Ida Gobbini. 2000. The distributed human neural system for face perception. Trends in Cognitive Sciences 4, 6, 223--233.
33. William G. Hayward, Gillian Rhodes, and Adrian Schwaninger. 2008. An own-race advantage for components as well as configurations in face recognition. Cognition 106, 2, 1017--1027.
34. Peter J. Hills, David A. Ross, and Michael B. Lewis. 2011. Attention misplaced: The role of diagnostic features in the face-inversion effect. Journal of Experimental Psychology: Human Perception and Performance 37, 5, 1396.
35. Peter J. Hills, Anthony J. Sullivan, and J. Michael Pake. 2012. Aberrant first fixations when looking at inverted faces in various poses: The result of the centre-of-gravity effect? British Journal of Psychology 103, 4, 520--538.
36. Erin M. Ingvalson and Michael J. Wenger. 2005. A strong test of the dual-mode hypothesis. Perception and Psychophysics 67, 1, 14--35.
37. Roxane J. Itier, Claude Alain, Katherine Sedore, and Anthony R. McIntosh. 2007. Early face processing specificity: It's in the eyes! Journal of Cognitive Neuroscience 19, 11, 1815--1826.
38. Roxane J. Itier and Margot J. Taylor. 2002. Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: A repetition study using ERPs. Neuroimage 15, 2, 353--372.
39. Rachael E. Jack, Caroline Blais, Christoph Scheepers, Philippe G. Schyns, and Roberto Caldara. 2009. Cultural confusions show that facial expressions are not universal. Current Biology 19, 18, 1543--1548.
40. Nancy Kanwisher. 2000. Domain specificity in face perception. Nature Neuroscience 3, 759--763.
41. Nancy Kanwisher and Galit Yovel. 2006. The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society B: Biological Sciences 361, 1476, 2109--2128.
42. Andrzej Kasinski, Andrzej Florek, and Adam Schmidt. 2008. The PUT face database. Image Processing and Communications 13, 3--4, 59--64.
43. Ruth Kimchi and Rama Amishav. 2010. Faces as perceptual wholes: The interplay between component and configural properties in face processing. Visual Cognition 18, 7, 1034--1062.
44. Renaud Laguesse and Bruno Rossion. 2013. Face perception is whole or none: Disentangling the role of spatial contiguity and interfeature distances in the composite face illusion. Perception 42, 1013--1026.
45. Michéal Larkin and Carol O'Sullivan. 2011. Perception of simplification artifacts for animated characters. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization. 93--100.
46. Richard Le Grand, Catherine J. Mondloch, Daphne Maurer, and Henry P. Brent. 2004. Impairment in holistic face processing following early visual deprivation. Psychological Science 15, 11, 762--768.
47. Helmut Leder and Vicki Bruce. 2000. When inverted faces are recognized: The role of configural information in face recognition. Quarterly Journal of Experimental Psychology: Section A 53, 2, 513--536.
48. Chang Hong Liu and James Ward. 2006. The use of 3D information in face recognition. Vision Research 46, 6, 768--773.
49. Janek S. Lobmaier and Fred W. Mast. 2007. Perception of novel faces: The parts have it! Perception 36, 11, 1660.
50. Karl F. MacDorman, Robert D. Green, Chin-Chang Ho, and Clinton T. Koch. 2009. Too real for comfort? Uncanny responses to computer generated faces. Computers in Human Behavior 25, 3, 695--710.
51. Karl F. MacDorman and Hiroshi Ishiguro. 2006. The uncanny advantage of using androids in cognitive and social science research. Interaction Studies 7, 3, 297--337.
52. Daphne Maurer, Richard Le Grand, and Catherine J. Mondloch. 2002. The many faces of configural processing. Trends in Cognitive Sciences 6, 6, 255--260.
53. Gregory McCarthy, Aina Puce, John C. Gore, and Truett Allison. 1997. Face-specific processing in the human fusiform gyrus. Journal of Cognitive Neuroscience 9, 5, 605--610.
54. Rachel McDonnell, Martin Breidt, and Heinrich Bülthoff. 2012. Render me real?: Investigating the effect of render style on the perception of animated virtual humans. ACM Transactions on Graphics 31, 4 (July 2012), 91:1--91:11.
55. Thomas J. McKeeff, Rankin W. McGugin, Frank Tong, and Isabel Gauthier. 2010. Expertise increases the functional overlap between face and object perception. Cognition 117, 3, 355--360.
56. Elinor McKone, Anne Aimola Davies, Hayley Darke, Kate Crookes, Tushara Wickramariyaratne, Stephanie Zappia, Chiara Fiorentini, Simone Favelle, Mary Broughton, and Dinusha Fernando. 2013. Importance of the inverted control in measuring holistic face processing with the composite effect and part-whole effect. Frontiers in Psychology 4, 33.
57. Elinor McKone, Nancy Kanwisher, and Bradley C. Duchaine. 2007. Can generic expertise explain special processing for faces? Trends in Cognitive Sciences 11, 1, 8--15.
58. Ann M. McNamara. 2005. Exploring perceptual equivalence between real and simulated imagery. In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization (APGV'05). 123--128.
59. Christian A. Meissner and John C. Brigham. 2001. Thirty years of investigating the own-race bias in memory for faces: A meta-analytic review. Psychology, Public Policy, and Law 7, 1, 3.
60. Gary W. Meyer, Holly E. Rushmeier, Michael F. Cohen, Donald P. Greenberg, and Kenneth E. Torrance. 1986. An experimental evaluation of computer graphics imagery. ACM Transactions on Graphics 5, 1, 30--50.
61. Caroline Michel, Bruno Rossion, Jaehyun Han, Chan-Sup Chung, and Roberto Caldara. 2006. Holistic processing is finely tuned for faces of one's own race. Psychological Science 17, 7, 608--615.
62. Masahiro Mori. 1970. The uncanny valley. Energy 7, 4, 33--35.
63. Morris Moscovitch, Gordon Winocur, and Marlene Behrmann. 1997. What is special about face recognition? Nineteen experiments on a person with visual object agnosia and dyslexia but normal face recognition. Journal of Cognitive Neuroscience 9, 5, 555--604.
64. Vaidehi Natu, David Raboy, and Alice J. O'Toole. 2011. Neural correlates of own-and other-race face perception: Spatial and temporal response differences. NeuroImage 54, 3, 2547--2555.
65. Charles A. Nelson. 2001. The development and neural bases of face recognition. Infant and Child Development 10, 1--2, 3--18.
66. Tian-Tsong Ng and Shih-Fu Chang. 2013. Discrimination of computer synthesized or recaptured images from real images. In Digital Image Forensics. Springer, 275--309.
67. Kristine L. Nowak, Marina Krcmar, and Kirstie M. Farrar. 2008. The causes and consequences of presence: Considering the influence of violent video games on presence and aggression. Presence: Teleoperators and Virtual Environments 17, 3, 256--268.
68. Jens-Rainer Ohm. 2004. Multimedia Communication Technology: Representation, Transmission and Identification of Multimedia Signals. Springer Verlag.
69. Akinyinka Omigbodun and Garrison Cottrell. 2013. Evidence for holistic facial expression processing with a neurocomputational model. Journal of Vision 13, 9, Article 99.
70. Gabriele Paolacci, Jesse Chandler, and Panagiotis Ipeirotis. 2010. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 5, 411--419.
71. Daniel Piepers and Rachel Robbins. 2012. A review and clarification of the terms “holistic,” “configural,” and “relational” in the face perception literature. Frontiers in Psychology 3, 559.
72. Paul Rademacher, Jed Lengyel, Edward Cutrell, and Turner Whitted. 2001. Measuring the perception of visual realism in images. In Proceedings of the Eurographics Workshop on Rendering Techniques. 235--248.
73. Ganesh Ramanarayanan, James Ferwerda, Bruce Walter, and Kavita Bala. 2007. Visual equivalence: Towards a new standard for image fidelity. ACM Transactions on Graphics 26, 3, 76.
74. Gillian Rhodes, William G. Hayward, and Christopher Winkler. 2006. Expert face coding: Configural and component coding of own-race and other-race faces. Psychonomic Bulletin and Review 13, 3, 499--505.
75. Jennifer J. Richler, Thomas J. Palmeri, and Isabel Gauthier. 2012. Meanings, mechanisms, and measures of holistic processing. Frontiers in Psychology 3, 553.
76. Maximilian Riesenhuber and Tomaso Poggio. 1999. Hierarchical models of object recognition in cortex. Nature Neuroscience 2, 11, 1019--1025.
77. Bruno Rossion. 2013. The composite face illusion: A whole window into our understanding of holistic face perception. Visual Cognition 21, 2, 139--253.
78. Bruno Rossion, I. Gauthier, Valérie Goffaux, M. J. Tarr, and M. Crommelinck. 2002. Expertise training with novel objects leads to left-lateralized facelike electrophysiological responses. Psychological Science 13, 3, 250--257.
79. Bruno Rossion, Bernard Hanseeuw, and Laurence Dricot. 2012. Defining face perception areas in the human brain: A large-scale factorial fMRI face localizer analysis. Brain and Cognition 79, 2, 138--157.
80. Kaitlin F. Ryan and Noah Z. Schwartz. 2013. Face recognition in emotional scenes: Observers remember the eye shape but forget the nose. Perception 42, 330--340.
81. Javid Sadrô, Izzat Jarudi, and Pawan Sinhaô. 2003. The role of eyebrows in face recognition. Perception 32, 285--293.
82. Adrian Schwaninger, Claus-Christian Carbon, and Helmut Leder. 2003. Expert face processing: Specialization and constraints. In Development of Face Processing. Hogrefe, Gottingen, Germany, 81--97.
83. Adrian Schwaninger, Janek S. Lobmaier, Christian Wallraven, and Stephan Collishaw. 2009. Two routes to face perception: Evidence from psychophysics and computational modeling. Cognitive Science 33, 8, 1413--1440.
84. Adrian Schwaninger, Stefan Ryf, and Franziska Hofer. 2003. Configural information is processed differently in perception and recognition of faces. Vision Research 43, 14, 1501--1505.
85. Adrian Schwaninger and Jisien Yang. 2011. The application of 3D representations in face recognition. Vision Research 51, 9, 969--977.
86. Justine Sergent. 1984. An investigation into component and configural processes underlying face perception. British Journal of Psychology 75, 2, 221--242.
87. Pawan Sinha, Benjamin Balas, Yuri Ostrovsky, and Richard Russell. 2006. Face recognition by humans: Nineteen results all computer vision researchers should know about. Proceedings of the IEEE 94, 11, 1948--1962.
88. Yiying Song, Yu L. L. Luo, Xueting Li, Miao Xu, and Jia Liu. 2013. Representation of contextually related multiple objects in the human ventral visual pathway. Journal of Cognitive Neuroscience 25, 8, 1261--1269.
89. Cheston Tan and Tomaso Poggio. 2013. Faces as a “model category” for visual object recognition. Technical Report MIT-CSAIL-TR-2013-004, CBCL-311, Massachusetts Institute of Technology, Cambridge, MA.
90. James W. Tanaka and Martha J. Farah. 1993. Parts and wholes in face recognition. Journal of Experimental Psychology 46, 2, 225--245.
91. Tim Valentine. 1988. Upside-down faces: A review of the effect of inversion upon face recognition. British Journal of Psychology 79, 4, 471--491.
92. Goedele Van Belle, Peter De Graef, Karl Verfaillie, Bruno Rossion, and Philippe Lefèvre. 2010. Face inversion impairs holistic perception: Evidence from gaze-contingent stimulation. Journal of Vision 10, 5, 10.
93. Peter Vangorp, Christian Richardt, Emily A. Cooper, Gaurav Chaurasia, Martin S. Banks, and George Drettakis. 2013. Perception of perspective distortions in image-based rendering. ACM Transactions on Graphics 32, 4, Article 58.
94. Christian Wallraven, Martin Breidt, Douglas W. Cunningham, and Heinrich H. Bülthoff. 2008. Evaluating the perceptual realism of animated facial expressions. ACM Transactions on Applied Perception (TAP) 4, 4, 4.
95. Ruosi Wang, Jingguang Li, Huizhen Fang, Moqian Tian, and Jia Liu. 2012. Individual differences in holistic processing predict face recognition ability. Psychological Science 23, 2, 169--177.
96. Tamara L. Watson. 2013. Implications of holistic face processing in autism and schizophrenia. Frontiers in Psychology 4, 414.
97. Thomas D. Wickens. 2001. Elementary Signal Detection Theory. Oxford University Press.
98. Alan C.-N. Wong, Cindy M. Bukach, Janet Hsiao, Emma Greenspon, Emily Ahern, Yiran Duan, and Kelvin F. H. Lui. 2012a. Holistic processing as a hallmark of perceptual expertise for nonface categories including Chinese characters. Journal of Vision 12, 13, 7.
99. Yetta K. Wong, Jonathan R. Folstein, and Isabel Gauthier. 2012b. The nature of experience determines object representations in the visual system. Journal of Experimental Psychology: General 141, 4, 682.
100. Yaoda Xu. 2005. Revisiting the role of the fusiform face area in visual expertise. Cerebral Cortex 15, 8, 1234--1242.
101. Su Xue, Aseem Agarwala, Julie Dorsey, and Holly Rushmeier. 2012. Understanding and improving the realism of image composites. ACM Transactions on Graphics 31, 4, 84:1--84:10.
102. Robert K. Yin. 1969. Looking at upside-down faces. Journal of Experimental Psychology 81, 1, 141.
103. Andrew W. Young, Deborah Hellawell, and Dennis C. Hay. 1987. Configurational information in face perception. Perception 16, 6, 747--759.

Index Terms

• Applied computing

  • Law, social and behavioral sciences

• Computing methodologies

  • Computer graphics

    • Image manipulation

      • Texturing

    • Rendering

      • Reflectance modeling