Kernel Nyström method for light transport

Abstract

We propose a kernel Nyström method for reconstructing the light transport matrix from a relatively small number of acquired images. Our work is based on the generalized Nyström method for low rank matrices. We introduce the light transport kernel and incorporate it into the Nyström method to exploit the nonlinear coherence of the light transport matrix. We also develop an adaptive scheme for efficiently capturing the sparsely sampled images from the scene. Our experiments indicate that the kernel Nyström method can achieve good reconstruction of the light transport matrix with a few hundred images and produce high quality relighting results. The kernel Nyström method is effective for modeling scenes with complex lighting effects and occlusions which have been challenging for existing techniques.

References

1. An, X., and Pellacini, F. 2008. Appprop: All-pairs appearance-space edit propagation. ACM Transactions on Graphics 27, 3 (Aug.), 40:1--40:9.
2. Chuang, Y.-Y., Zongker, D. E., Hindorff, J., Curless, B., Salesin, D. H., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 121--130.
3. Cristianini, N., and Shawe-Taylor, J. 2000. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press.
4. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 369--378.
5. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 145--156.
6. Fazel, M. 2002. Matrix rank minimization with applications. PhD thesis, Stanford University.
7. Fuchs, M., Blanz, V., Lensch, H. P. A., and Seidel, H.-P. 2007. Adaptive sampling of reflectance fields. ACM Transactions on Graphics 26, 2 (June), 10:1--10:18.
8. Garg, G., Talvala, E.-V., Levoy, M., and Lensch, H. P. A. 2006. Symmetric photography: Exploiting data-sparseness in reflectance fields. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 251--262.
9. Goesele, M., Lensch, H. P. A., Lang, J., Fuchs, C., and Seidel, H.-P. 2004. Disco: acquisition of translucent objects. ACM Transactions on Graphics 23, 3 (Aug.), 835--844.
10. Goreinov, S., Tyrtyshnikov, E. E., and Zamarashkin, N. L. 1997. A theory of pseudo-skeleton approximations. Linear Algeabra and Applications 261, 1--21.
11. Hašan, M., Pellacini, F., and Bala, K. 2007. Matrix rowcolumn sampling for the many-light problem. ACM Transactions on Graphics 26, 3 (July), 26:1--26:10.
12. Hašan, M., Velazquez-Armendariz, E., Pellacini, F., and Bala, K. 2008. Tensor clustering for rendering manylight animations. Computer Graphics Forum (Proc. Eurographics Rendering 2008) 27, 4, 1105--1114.
13. Hawkins, T., Einarsson, P., and Debevec, P. 2005. A dual light stage. In Rendering Techniques 2005: 16th Eurographics Workshop on Rendering, 91--98.
14. Levoy, M., and Hanrahan, P. M. 1996. Light field rendering. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 31--42.
15. Lin, Z., Wong, T.-T., and Shum, H.-Y. Relighting with the reflected irradiance field: Representation, sampling and reconstruction. International Journal of Computer Vision 49, 2.
16. Mahajan, D., Shlizerman, I. K., Ramamoorthi, R., and Belhumeur, P. 2007. A theory of locally low dimensional light transport. ACM Transactions on Graphics 26, 3 (July), 62:1--62:10.
17. Masselus, V., Peers, P., Dutré, P., and Willems, Y. D. 2003. Relighting with 4d incident light fields. ACM Transactions on Graphics 22, 3 (July), 613--620.
18. Masselus, V., Peers, P., Dutr0108, P., and Willems, Y. D. 2004. Smooth reconstruction and compact representation of reflectance functions for image-based relighting. In Rendering Techniques 2004: 15th Eurographics Workshop on Rendering, 287--298.
19. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., and McMillan, L. 2002. Image-based 3D photography using opacity hulls. ACM Transactions on Graphics 21, 3 (July), 427--437.
20. Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-refined reflectance functions from natural illumination. In Rendering Techniques 2004: 15th Eurographics Workshop on Rendering, 299--308.
21. Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. Allfrequency shadows using non-linear wavelet lighting approximation. ACM Transactions on Graphics 22, 3 (July), 376--381.
22. Peers, P., and Dutré, P. 2005. Inferring reflectance functions from wavelet noise. In Rendering Techniques 2005: 16th Eurographics Workshop on Rendering, 173--182.
23. Peers, P., Vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., and Dutré, P. 2006. A compact factored representation of heterogeneous subsurface scattering. ACM Transactions on Graphics 25, 3 (July), 746--753.
24. Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive light transport sensing. ACM Transactions on Graphics 28, 1 (Jan.), 3:1--3:18.
25. Platt, J. C. 2005. Fastmap, metricmap, and landmark mds are all nyström algorithms. In 10th International Workshop on Artificial Intelligence and Statistics, 261--268.
26. Press, W. H., et al. 1992. Numerical recipes in C (second edition). Cambridge University Press.
27. Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 2, 609--618.
28. Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual photography. ACM Transactions on Graphics 24, 3, 745--755.
29. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3 (Aug.), 756--764.
30. Williams, C., and Seeger, M. 2000. Using the nyström method to speed up kernel machines. Advances in Neural Information Processing Systems 13, 682--688.
31. Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment matting and compositing. In Proceedings of SIGGRAPH 99, Computer Graphics Proceedings, Annual Conference Series, 205--214.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Reconstruction

      • Image and video acquisition

  • Modeling and simulation

    • Model development and analysis

      • Modeling methodologies

• Mathematics of computing

  • Mathematical analysis

    • Numerical analysis

      • Computations on matrices