Analysis of human faces using a measurement-based skin reflectance model

Abstract

We have measured 3D face geometry, skin reflectance, and subsurface scattering using custom-built devices for 149 subjects of varying age, gender, and race. We developed a novel skin reflectance model whose parameters can be estimated from measurements. The model decomposes the large amount of measured skin data into a spatially-varying analytic BRDF, a diffuse albedo map, and diffuse subsurface scattering. Our model is intuitive, physically plausible, and -- since we do not use the original measured data -- easy to edit as well. High-quality renderings come close to reproducing real photographs. The analysis of the model parameters for our sample population reveals variations according to subject age, gender, skin type, and external factors (e.g., sweat, cold, or makeup). Using our statistics, a user can edit the overall appearance of a face (e.g., changing skin type and age) or change small-scale features using texture synthesis (e.g., adding moles and freckles). We are making the collected statistics publicly available to the research community for applications in face synthesis and analysis.

References

1. Angelopoulou, E., Molana, R., and Daniilidis, K. 2001. Multispectral skin color modeling. In IEEE Conf. on Comp. Vision and Pattern Rec. (CVPR), 635--642.
2. Barsky, S., and Petrou, M. 2001. Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Eighth IEEE International Conference on Computer Vision, vol. 2, 600--605.
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. Computer Graphics 33, Annual Conference Series, 187--194.
4. Blinn, J. 1977. Models of light reflection for computer synthesized pictures. Computer Graphics 11, Annual Conference Series, 192--198.
5. Borshukov, G., and Lewis, J. 2003. Realistic human face rendering for the matix reloaded. In ACM SIGGRAPH 2003 Conference Abstracts and Applications (Sketch).
6. Cula, O., and Dana, K. 2002. Image-based skin analysis. In Texture 2002, The Second International Workshop on Texture Analysis and Synthesis, 35--42.
7. Cula, O., Dana, K., Murphy, F., and Rao, B. 2004. Bidirectional imaging and modeling of skin texture. IEEE Transactions on Biomedical Engineering 51, 12 (Dec.), 2148--2159.
8. Cula, O., Dana, K., Murphy, F., and Rao, B. 2005. Skin texture modeling. International Journal of Computer Vision 62, 1--2 (April--May), 97--119.
9. Dana, K. J., van Ginneken, B., Nayar, S. K., and Koen-Derink, J. J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 1, 18, 1--34.
10. Debevec, P., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Computer Graphics, SIGGRAPH 97 Proceedings, 369--378.
11. Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Computer Graphics, SIGGRAPH 96 Proceedings, 11--20.
12. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Computer Graphics, SIGGRAPH 2000 Proceedings, 145--156.
13. Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., and Hawkins, T. 2002. A lighting reproduction approach to live-action compositing. ACM Transactions on Graphics (SIGGRAPH 2002) 21, 3 (July), 547--556.
14. Donner, C., and Jensen, H. W. 2005. Light diffusion in multi-layered translucent materials. ACM Transactions on Graphics 24, 3 (July), 1032--1039.
15. Fitzpatrick, T. 1988. The validity and practicality of sun-reactive skin types i through vi. Arch. Dermatology 124, 6 (June), 869--871.
16. Fuchs, C., Goesele, M., Chen, T., and Seidel, H.-P. 2005. An empirical model for heterogeneous translucent objects. Tech. Rep. MPI-I-2005-4-006, Max-Planck Institute für Informatik.
17. Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2005. Reflectance from images: A model-based approach for human faces. Research Report MPI-I-2005-4-001, Max-Planck-Institut für Informatik, Stuhlsatzenhausweg 85, 66123 Saarbrücken, Germany. Accepted for publication in IEEE TVCG.
18. Georghiades, A., Belhumeur, P., and Kriegman, D. 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47--54.
19. Georghiades, A. 2003. Recovering 3-D shape and reflectance from a small number of photographs. In Rendering Techniques, 230--240.
20. Goesele, M., Lensch, H., Lang, J., Fuchs, C., and Seidel, H. 2004. Disco -- acquisition of translucent objects. ACM Transactions on Graphics 24, 3, 835--844.
21. Green, P., Halbert, M., and Robinson, P. 1966. Canonical analysis: An exposition and illustrative application. Marketing Research 3 (Feb.), 32--39.
22. Hanrahan, P., and Krueger, W. 1993. Reflection from layered surfaces due to subsurface scattering. In Computer Graphics, SIGGRAPH 93 Proceedings, 165--174.
23. Hawkins, T., Wenger, A., Tchou, C., Goransson, F., and Debevec, P. 2004. Animatable facial reflectance fields. In Rendering Techniques '04 (Proceedings of the Second Eurographics Symposium on Rendering).
24. Heeger, D., and Bergen, J. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 229--238.
25. Hery, C., 2005. Face cloning at ILM. Digital Face Cloning, Siggraph Course Notes.
26. Igarashi, T., Nishino, K., and Nayar, S. 2005. The appearance of human skin. Tech. Rep. CUCS-024-05, Department of Computer Science, Columbia University, June.
27. Jensen, H. W., and Buhler, J. 2002. A rapid hierarchical rendering technique for translucent materials. In Computer Graphics, SIGGRAPH 2002 Proceedings, 576--581.
28. Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. 2001. A practical model for subsurface light transport. In Computer Graphics, SIGGRAPH 2001 Proceedings, 511--518.
29. Jones, A., Gardner, A., Bolas, M., McDowall, I., and Debevec, P. 2005. Performance geometry capture for spatially varying relighting. In ACM SIGGRAPH 2005 Conference Abstracts and Applications (Sketch).
30. Koenderink, J., and Pont, S. 2003. The secret of velvety skin. Machine Vision and Application, 14, 260--268. Special Issue on Human Modeling, Analysis, and Synthesis.
31. Krishnaswamy, A., and Baranoski, G. 2004. A biophysically-based spectral model of light interaction with human skin. Computer Graphics Forum 23, 3 (Sept.), 331--340.
32. Lafortune, E., Foo, S.-C., Torrance, K., and Greenberg, D. 1997. Non-linear approximation of reflectance functions. Computer Graphics 31, Annual Conference Series, 117--126.
33. Larsson, M., Nilsson, H., and Strömberg, T. 2003. In vivo determination of local skin optical properties and photon path length by use of spatially resolved diffuse reflectance with applications in laser doppler flowmetry. Applied Optics 42, 1 (Jan.), 124--134.
34. Lawrence, J., Rusinkiewicz, S., and Ramamoorthi, R. 2004. Efficient brdf importance sampling using a factored representation. ACM Transactions on Graphics 23, 3, 496--505.
35. Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2001. Image-based reconstruction of spatially varying materials. In Proceedings of the 12th Eurographics Workshop on Rendering, 104--115.
36. Loop, C. 1987. Smooth Subdivision Surfaces based on Triangles. Master's thesis, Department of Mathematics, University of Utah.
37. Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-based BRDF measurement including human skin. In Proceedings of the 10th Eurographics Workshop on Rendering, 139--152.
38. Marschner, S., Guneter, B., and Raghupathy, S. 2000. Modeling and rendering for realistic facial animation. In 11th Eurographics Workshop on Rendering, 231--242.
39. Marschner, S., Westin, S., Lafortune, E., and Torrance, K. 2000. Image-based measurement of the Bidirectional Reflectance Distribution Function. Applied Optics 39, 16 (June), 2592--2600.
40. Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics 24, 3, 787--794.
41. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3, 536--543.
42. Ng, C., and Li, L. 2001. A multi-layered reflection model of natural human skin. In Computer Graphics International, 249256.
43. Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of brdf models. In Eurographics Symposium on Rendering, 117--126.
44. Nickell, S., Hermann, M., Essenpreis, M., Farrell, T. J., Kramer, U., and Patterson, M. S. 2000. Anisotropy of light propagation in human skin. Phys. Med. Biol. 45, 2873--2886.
45. Nicodemus, F., Richmond, J., Hsia, J., Ginsberg, I., and Limperis, T. 1977. Geometric considerations and nomenclature for reflectance. Monograph 160, National Bureau of Standards (US), October.
46. Paris, S., Sillion, F., and Quan, L. 2003. Lightweight face relighting. In Proceedings of Pacific Graphics, 41--50.
47. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. 1998. Synthesizing realistic facial expressions from photographs. In Computer Graphics, vol. 32 of SIGGRAPH 98 Proceedings, 75--84.
48. Sander, P., Gosselin, D., and Mitchell, J. 2004. Real-time skin rendering on graphics hardware. SIGGRAPH 2004 Sketch.
49. Sheffer, A., Levy, B., Mogilnitsky, M., and Bogomyakov, A. 2005. ABF++: Fast and robust angle based flattening. ACM Transactions on Graphics 24, 2, 311--330.
50. Stam, J. 2001. An illumination model for a skin layer bounded by rough surfaces. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques, Springer, Wien, Vienna, Austria, 39--52.
51. Tong, X., Wang, J., Lin, S., Guo, B., and Shum, H.-Y. 2005. Modeling and rendering of quasi-homogeneous materials. ACM Transactions on Graphics 24, 3, 1054--1061.
52. Torrance, K. E., and Sparrow, E. M. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America 57, 1104--1114.
53. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Transactions on Graphics 22, 3, 770--779.
54. Tuchin, V. 2000. Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. SPIE Press.
55. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3, 756--764.
56. Weyrich, T., Matusik, W., Pfister, H., Ngan, A., and Gross, M. 2005. Measuring skin reflectance and subsurface scattering. Tech. Rep. TR2005-046, Mitsubishi Electric Research Laboratories (MERL).
57. Williams, L. 2005. Case study: The gemini man. In SIGGRAPH 2005 Course 'Digital Face Cloning'.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision tasks

        • Scene understanding