Gloss perception in painterly and cartoon rendering

Abstract

Depictions with traditional media such as painting and drawing represent scene content in a stylized manner. It is unclear, however, how well stylized images depict scene properties like shape, material, and lighting. In this article, we describe the first study of material perception in stylized images (specifically painting and cartoon) and use nonphotorealistic rendering algorithms to evaluate how such stylization alters the perception of gloss. Our study reveals a compression of the range of representable gloss in stylized images so that shiny materials appear more diffuse in painterly rendering, while diffuse materials appear shinier in cartoon images. From our measurements we estimate the function that maps realistic gloss parameters to their perception in a stylized rendering. This mapping allows users of NPR algorithms to predict the perception of gloss in their images. The inverse of this function exaggerates gloss properties to make the contrast between materials in a stylized image more faithful. We have conducted our experiment both in a lab and on a crowdsourcing Web site. While crowdsourcing allows us to quickly design our pilot study, a lab experiment provides more control on how subjects perform the task. We provide a detailed comparison of the results obtained with the two approaches and discuss their advantages and drawbacks for studies like ours.

References

1. Baird, J. and Noma, E. 1978. Fundamentals of Scaling and Psychophysics. Wiley Series in Behavior. Wiley.
2. Berzhanskaya, J., Swaminathan, G., Beck, J., and Mingolla, E. 2002. Highlights and surface gloss perception. J. Vis. 2, 3.
3. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? ACM Trans. Graph. 28.
4. Cooke, H. L. 1967. Painting Lessons from the Great Masters. Watson-Guptill Publications.
5. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-Generated watercolor. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'97).
6. DeCarlo, D. and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3, 769--776.
7. Dror, R. O., Adelson, E. H., and Willsky, A. S. 2001. Recognition of surface reflectance properties from a single image under unknown real-world illumination. In Proceedings of the IEEE Workshop on Identifying Objects across Variations in Lighting.
8. Ferwerda, J. A., Pellacini, F., and Greenberg, D. P. 2001. A psychophysically-based model of surface gloss perception. In Proceedings of the SPIE Conference on Human Vision and Electronic Imaging.
9. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2001. How do humans determine reflectance properties under unknown illumination? In Proceedings of the IEEE Workshop on Identifying Objects across Variations in Lighting.
10. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-World illumination and the perception of surface reflectance properties. J. Vis. 3, 5, 347--368.
11. Gooch, A., Gooch, B., Shirley, P., and Cohen, E. 1998. A non-photorealistic lighting model for automatic technical illustration. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). 447--452.
12. Gooch, B., Reinhard, E., and Gooch, A. 2004. Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23, 1.
13. Haeberli, P. 1990. Paint by numbers: Abstract image representations. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'90). 207--214.
14. Hays, J. and Essa, I. 2004. Image and video based painterly animation. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR'04). 113--120.
15. Heer, J. and Bostock, M. 2010. Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. In Proceedings of the 28^th International Conference on Human Factors in Computing Systems (CHI'10).
16. Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). 453--460.
17. Johnson, C. 1992. Creating Textures in Watercolor. North Light Books.
18. Kang, H., Lee, S., and Chui, C. K. 2007. Coherent line drawing. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR'07). 43--50.
19. Kozlowski, O. and Kautz, J. 2007. Is accurate occlusion of glossy reflections necessary? In Proceedings of the ACM Symposium on Applied Perception in Graphics and Visualization (APGV'07). 91--98.
20. Krivanek, J., Ferwerda, J. A., and Bala, K. 2010. Effects of global illumination approximations on material appearance. ACM Trans. Graph. 29, 3.
21. Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'97). 407--414.
22. Meier, B. J. 1996. Painterly rendering for animation. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96).
23. Nishida, S. and Shinya, M. 1998. Use of image-based information in judgments of surface-reflectance properties. J. Opt. Soc. Amer. A: Opt. Image Sci. Vis. 15, 12.
24. Ott, J. and Kuseno, Y. 2005. Let's Draw Manga: Using Color. Digital Manga Publishing.
25. Pellacini, F., Ferwerda, J. A., and Greenberg, D. P. 2000. Toward a psychophysically-based light reflection model for image synthesis. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH' 00). 55--64.
26. Pharr, M. and Humphreys, G. 2004. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, San Fransisco, CA.
27. Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: Towards a new standard for image fidelity. ACM Trans. Graph. 26, 3.
28. Smith, K., Soler, C., Luft, T., Deussen, O., and Thollot, J. 2010. Automatic pen-and-ink illustration of tone, gloss, and texture. Res. rep. RR-7194, INRIA.
29. Vangorp, P. and Dutre, P. 2008. Shape-Dependent gloss correction. In Proceedings of the ACM Symposium on Applied Perception in Graphics and Visualization (APGV'08). 123--130.
30. Vangorp, P., Laurijssen, J., and Dutre, P. 2007. The influence of shape on the perception of material reflectance. ACM Trans. Graph. 26, 3, 77.
31. Wallraven, C., Bulthoff, H. H., Cunningham, D. W., Fischer, J., and Bartz, D. 2007. Evaluation of real-world and computer-generated stylized facial expressions. ACM Trans. Appl. Percept. 4, 3, 16.
32. Ward, G. J. 1992. Measuring and modeling anisotropic reflection. ACM Trans. Graph. 26, 2, 265--272.
33. Wills, J., Agarwal, S., Kriegman, D., and Belongie, S. 2009. Toward a perceptual space for gloss. ACM Trans. Graph. 28, 4.
34. Winkenbach, G. and Salesin, D. H. 1996. Rendering parametric surfaces in pen and ink. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). 469--476.
35. Winnemoeller, H., Olsen, S. C., and Gooch, B. 2006. Real-Time video abstraction. ACM Trans. Graph. 25, 3.
36. Winnemoeller, H., Feng, D., Gooch, B., and Suzuki, S. 2007. Using npr to evaluate perceptual shape cues in dynamic environments. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR'07). 85--92.
37. Xue, S., Chen, X., Dorsey, J., and Rushmeier, H. E. 2010. Printed patterns for enhanced shape perception of papercraft models. Comput. Graph. Forum 29, 2, 625--634.
38. Zeng, K., Zhao, M., Xiong, C., and Zhu, S.-C. 2009. From image parsing to painterly rendering. ACM Trans. Graph. 29, 1.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces