Inverse shade trees for non-parametric material representation and editing

Abstract

Recent progress in the measurement of surface reflectance has created a demand for non-parametric appearance representations that are accurate, compact, and easy to use for rendering. Another crucial goal, which has so far received little attention, is editability: for practical use, we must be able to change both the directional and spatial behavior of surface reflectance (e.g., making one material shinier, another more anisotropic, and changing the spatial "texture maps" indicating where each material appears). We introduce an Inverse Shade Tree framework that provides a general approach to estimating the "leaves" of a user-specified shade tree from high-dimensional measured datasets of appearance. These leaves are sampled 1- and 2-dimensional functions that capture both the directional behavior of individual materials and their spatial mixing patterns. In order to compute these shade trees automatically, we map the problem to matrix factorization and introduce a flexible new algorithm that allows for constraints such as non-negativity, sparsity, and energy conservation. Although we cannot infer every type of shade tree, we demonstrate the ability to reduce multi-gigabyte measured datasets of the Spatially-Varying Bidirectional Reflectance Distribution Function (SVBRDF) into a compact representation that may be edited in real time.

References

1. Ashikhmin, M., Premože, S., and Shirley, P. 2000. A microfacet-based BRDF generator. In Proceedings of ACM SIGGRAPH 2000.
2. Chen, W.-C., Bouguet, J.-Y., Chu, M. H., and Grzeszczuk, R. 2002. Light field mapping: efficient representation and hardware rendering of surface light fields. In ACM Transactions on Graphics (SIGGRAPH 2002).
3. Cook, R. L. 1984. Shade trees. In Computer Graphics (Proceedings of ACM SIGGRAPH 1984), 223--231.
4. Dana, K., van Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics 18, 1, 1--34.
5. Furukawa, R., Kawasaki, H., Ikeuchi, K., and Sakauchi, M. 2002. Appearance based object modeling using texture database: acquisition, compression and rendering. In Eurographics Workshop on Rendering, 257--266.
6. Gardner, A., Tchou, C., Hawkins, T., and Debevec, P. 2003. Linear light source reflectometry. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 749--758.
7. Gill, P., Murray, W., Saunders, M., and Wright, M. 1984. Procedures for optimization problems with a mixture of bounds and general linear constraints. In ACM Transactions on Mathematical Software.
8. Goldman, D. B., Curless, B., Hertzmann, A., and Seitz, S. M. 2005. Shape and spatially-varying BRDFs from photometric stereo. In IEEE International Conference on Computer Vision.
9. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In Proceedings of ACM SIGGRAPH 1996.
10. Han, J. Y., and Perlin, K. 2003. Measuring bidirectional texture reflectance with a kaleidoscope. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 741--748.
11. Hartigan, J. A., and Wong, M. A. 1979. A k-means clustering algorithm. Applied Statistics 28, 100--108.
12. Heidrich, W., and Seidel, H.-P. 1999. Realistic, hardware-accelerated shading and lighting. In Proceedings of ACM SIGGRAPH 1999.
13. Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence.
14. Hoyer, P. O. 2002. Non-negative sparse coding. In IEEE Workshop on Neural Networks for Signal Processing, 557--565.
15. Jaroszkiewicz, R., and McCool, M. D. 2003. Fast extraction of BRDFs and material maps from images. In Graphcs Interface.
16. Kautz, J., and McCool, M. 1999. Interactive rendering with arbitrary BRDFs using separable approximations. In Eurographics Workshop on Rendering, 247--260.
17. Lafortune, E. P. F., Foo, S.-C., Torrance, K. E., and Green-Berg, D. P. 1997. Non-linear approximation of reflectance functions. In Proceedings of ACM SIGGRAPH 1997, 117--126.
18. Lawrence, J., Rusinkiewicz, S., and Ramamoorthi, R. 2004. Efficient BRDF importance sampling using a factored representation. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3.
19. Lee, D., and Seung, H. S. 2000. Algorithms for non-negative matrix factorization. In Proceedings of Neural Information Processing Systems, 556--562.
20. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Transactions on Graphics 22, 2.
21. Leung, T., and Malik, J. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision 43, 1, 29--44.
22. Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-Based BRDF measurement including human skin. In Eurographics Workshop on Rendering, 139--152.
23. Marschner, S. R., Westin, S. H., Arbree, A., and Moon, J. T. 2005. Measuring and modeling the appearance of finished wood. ACM Transactions on Graphics (SIGGRAPH 2005) 24, 3, 727--734.
24. Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 759--769.
25. McAllister, D. 2002. A Generalized Surface Appearance Representation for Computer Graphics. PhD thesis, UNC.
26. McCool, M. D., Ang, J., and Ahmad, A. 2001. Homomorphic factorization of BRDFs for high-performance rendering. In Proceedings of ACM SIGGRAPH 2001, 185--194.
27. Nag, 2005. Numerical Algorithms Group C Library.
28. Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of BRDF models. In Proceedings of the Eurographics Symposium on Rendering, 117--226.
29. Olshausen, B. A., and Field, D. J. 2002. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607--609.
30. Peers, P., Vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., and Dutré, P. 2006. A compact factored representation of heterogeneous subsurface scattering. ACM Transactions on Graphics (SIGGRAPH 2006) 25, 3.
31. Rusinkiewicz, S. 1998. A new change of variables for efficient BRDF representation. In Eurographics Workshop on Rendering, 11--22.
32. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 770--779.
33. Vasilescu, M. A., and Terzopoulos, D. 2004. TensorTextures: Multilinear image-based rendering. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3.
34. Ward, G. J. 1992. Measuring and modeling anisotropic reflection. In Computer Graphics (Proceedings of ACM SIGGRAPH 1992), 265--272.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

      • Texturing

    • Shape modeling

      • Parametric curve and surface models

      • Volumetric models

• Theory of computation

  • Randomness, geometry and discrete structures