5D Covariance tracing for efficient defocus and motion blur

Abstract

The rendering of effects such as motion blur and depth-of-field requires costly 5D integrals. We accelerate their computation through adaptive sampling and reconstruction based on the prediction of the anisotropy and bandwidth of the integrand. For this, we develop a new frequency analysis of the 5D temporal light-field, and show that first-order motion can be handled through simple changes of coordinates in 5D. We further introduce a compact representation of the spectrum using the covariance matrix and Gaussian approximations. We derive update equations for the 5 × 5 covariance matrices for each atomic light transport event, such as transport, occlusion, BRDF, texture, lens, and motion. The focus on atomic operations makes our work general, and removes the need for special-case formulas. We present a new rendering algorithm that computes 5D covariance matrices on the image plane by tracing paths through the scene, focusing on the single-bounce case. This allows us to reduce sampling rates when appropriate and perform reconstruction of images with complex depth-of-field and motion blur effects.

References

1. Arikan, O., Forsyth, D. A., and O'Brien, J. F. 2005. Fast and detailed approximate global illumination by irradiance decomposition. ACM Trans. Graph. 24, 3, 1108--1114.
2. Bagher, M., M., Soler, C., Subr, K., Belcour, L., and Holzschuch, N. 2012. Interactive rendering of acquired materials on dynamic geometry using bandwidth prediction. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D'12). M. Garland and R. Wang, Eds., ACM Press, New York, 127--134.
3. Barsky, B. A., Horn, D. R., Klein, S. A., Pang, J. A., and Yu, M. 2003. Camera models and optical systems used in computer graphics: Part II, image based techniques. In Proceedings of the International Conference on Computational Science and its Applications.
4. Barsky, B. A. and Kosloff, T. J. 2008. Algorithms for rendering depth of field effects in computer graphics. In Proceedings of the 12^th WSEAS International Conference on Computers. 999--1010.
5. Bolin, M. R. and Meyer, G. W. 1995. A frequency based ray tracer. In Proceedings of the Annual Conference on Computer Graphics (SIGGRAPH'95). 409--418.
6. Bolin, M. R. and Meyer, G. W. 1998. A perceptually based adaptive sampling algorithm. In Proceedings of the Annual Conference on Computer Graphics (SIGGRAPH'98). 299--309.
7. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In Proceedings of the Annual Conference on Computer Graphics (SIGGRAPH'00). 307--318.
8. Chen, M. and Arvo, J. 2000. Theory and application of specular path perturbation. ACM Trans. Graph. 19, 4, 246--278.
9. Cook, R. L. 1986. Stochastic sampling in computer graphics. ACM Trans. Graph. 5, 51--72.
10. Cook, R. L., Carpenter, L., and Catmull, E. 1987. The reyes image rendering architecture. ACM SIGGRAPH Comput. Graph. 21, 95--102.
11. Cook, R. L., Porter, T., and Carpenter, L. 1984. Distributed ray tracing. ACM SIGGRAPH Comput. Graph. 19, 3, 137--145.
12. Dippé, M. A. Z. and Wold, E. H. 1985. Antialiasing through stochastic sampling. ACM SIGGRAPH Comput. Graph. 19, 3, 69--78.
13. Durand, F., Holzschuch, N., Soler, C., Chan, E., and Sillion, F. X. 2005. A frequency analysis of light transport. ACM Trans. Graph. 24, 3, 1115--1126.
14. Egan, K., Durand, F., and Ramamoorthi, R. 2011a. Practical filtering for efficient ray-traced directional occlusion. ACM Trans. Graph. 30, 6.
15. Egan, K., Hecht, F., Durand, F., and Ramamoorthi, R. 2011b. Frequency analysis and sheared filtering for shadow light fields of complex occluders. ACM Trans. Graph. 30, 2, 1--13.
16. Egan, K., Tseng, Y.-T., Holzschuch, N., Durand, F., and Ramamoorthi, R. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. 28, 3.
17. Fatahalian, K., Luong, E., Boulos, S., Akeley, K., Mark, W. R., and Hanrahan, P. 2009. Data-parallel rasterization of micropolygons with defocus and motion blur. In Proceedings of the Conference on High Performance Graphics (HPG'09). ACM Press, New York, 59--68.
18. Ferwerda, J. A., Shirley, P., Pattanaik, S. N., and Greenberg, D. P. 1997. A model of visual masking for computer graphics. In Proceedings of the Annual Conference on Computer Graphics (SIGGRAPH'97). 143--152.
19. Gerrard, A. and Burch, J. 1975. Introduction to Matrix Methods in Optics. Wiley Series in Pure and Applied Optics. John Wiley and Sons.
20. Hachisuka, T., Jarosz, W., Weistroffer, R. P., Dale, K., Humphreys, G., Zwicker, M., and Jensen, H. W. 2008. Multidimensional adaptive sampling and reconstruction for ray tracing. ACM Trans. Graph. 27, 3, 33:1--33:10.
21. Hasan, M., Pellacini, F., and Bala, K. 2007. Matrix row-column sampling for the many-light problem. ACM Trans. Graph. 26, 3.
22. Heckbert, P. S. 1989. Fundamentals of texture mapping and image warping, Master's thesis, UCB/CSD 89/516CS, UC Berkeley. University of Californiya.
23. Hou, Q., Qin, H., Li, W., Guo, B., and Zhou, K. 2010. Micropolygon ray tracing with defocus and motion blur. In ACM SIGGRAPH'10 Papers. ACM Press, New York, 64:1--64:10.
24. Igehy, H. 1999. Tracing ray differentials. In Proceedings of the 26^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'99). ACM Press/Addison-Wesley Publishing Co., New York, 179--186.
25. Kirk, D. and Arvo, J. 1991. Unbiased sampling techniques for image synthesis. SIGGRAPH Comput. Graph. 25, 4, 153--156.
26. Kolb, C., Hanrahan, P. M., and Mitchell, D. 1995. A realistic camera model for computer graphics. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95). 317--324.
27. Kraus, M. and Strengert, M. 2007. Depth-of-field rendering by pyramidal image processing. Comput. Graph. Forum 26, 3, 645--654.
28. Lee, S., Eisemann, E., and Seidel, H.-P. 2009. Depth-of-field rendering with multiview synthesis. ACM Trans. Graph. 28, 5, 134:1--134:6.
29. Lee, S., Eisemann, E., and Seidel, H.-P. 2010. Real-time lens blur effects and focus control. ACM Trans. Graph. 29, 4, 65:1--65:7.
30. Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal light field reconstruction for rendering distribution effects. ACM Trans. Graph. 30, 4.
31. Lessig, C. and Fiume, E. 2010. On the effective dimension of light transport. Comput. Graph. Forum 29, 4, 1399--1403.
32. Levin, A., Hasinoff, S. W., Green, P., Durand, F., and Freeman, W. T. 2009. 4D frequency analysis of computational cameras for depth of field extension. ACM Trans. Graph. 28, 3, 97:1--97:10.
33. Levin, A., Sand, P., Cho, T. S., Durand, F., and Freeman, W. T. 2008. Motion-invariant photography. ACM Trans. Graph. 27, 3, 71:1--71:10.
34. Loviscach, J. 2005. Motion blur for textures by means of anisotropic filtering. In Proceedings of the Eurographics Symposium on Rendering. 105--110.
35. Mahajan, D., Shlizerman, I. K., Ramamoorthi, R., and Belhumeur, P. 2007. A theory of locally low dimensional light transport. ACM Trans. Graph. 26, 3.
36. Max, N. L. and Lerner, D. M. 1985. A two-and-a-half-d motion-blur algorithm. In Proceedings of the 12^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'85). ACM Press, New York, 85--93.
37. Mitchell, D. 1987. Generating antialiased images at low sampling densities. In Proceedings of the 14^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'87).
38. Mitchell, D. P. 1991. Spectrally optimal sampling for distribution ray tracing. In Proceedings of the 18^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'91). 157--164.
39. Myszkowski, K. 1998. The visible differences predictor: applications to global illumination problems. In Proceedings of the Eurographics Workshop on Rendering Techniques (EG'98). 223--236.
40. Ng, R. 2005. Fourier slice photography. ACM Trans. Graph. 24, 3, 735--744.
41. Overbeck, R. S., Donner, C., and Ramamoorthi, R. 2009. Adaptive wavelet rendering. ACM Trans. Graph. 28, 5, 140:1--140:12.
42. Potmesil, M. and Chakravarty, I. 1981. A lens and aperture camera model for synthetic image generation. SIGGRAPH Comput. Graph. 15, 3, 297--305.
43. Potmesil, M. and Chakravarty, I. 1983. Modeling motion blur in computer-generated images. SIGGRAPH Comput. Graph. 17, 3, 389--399.
44. Quinn, J. A., Langbein, F. C., and Martin, R. R. 2007. Lowdiscrepancy point sampling of meshes for rendering. In Proceedings of the Eurographics Symposium on Point-Based Graphics. 19--28.
45. Ramamoorthi, R. and Hanrahan, P. 2001. A signal-processing framework for inverse rendering. In Proceedings of the 28^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). 117--128.
46. Ramamoorthi, R., Koudelka, M., and Belhumeur, P. 2005. A fourier theory for cast shadows. IEEE Trans. Pattern Anal. Mach. Intell. 27, 2, 288--295.
47. Ramamoorthi, R., Mahajan, D., and Belhumeur, P. 2007. A first-order analysis of lighting, shading, and shadows. ACM Trans. Graph. 26, 1.
48. Rousselle, F., Knaus, C., and Zwicker, M. 2011. Adaptive sampling and reconstruction using greedy error minimization. In Proceedings of the ACM SIGGRAPH Asia. ACM Press, New York.
49. Sen, P. and Darabi, S. 2011a. Compressive rendering: A rendering application of compressed sensing. IEEE Trans. Visual. Comput. Graph. 17, 4, 487--499.
50. Sen, P. and Darabi, S. 2011b. On fltering the noise from the random parameters in monte carlo rendering. ACM Trans. Graph.
51. Sloan, P.-P., Hall, J., Hart, J., and Snyder, J. 2003. Clustered principal components for precomputed radiance transfer. ACM Trans. Graph. 22, 3, 382--391.
52. Soler, C., Subr, K., Durand, F., Holzschuch, N., and Sillion, F. X. 2009. Fourier depth of field. ACM Trans. Graph. 28, 2.
53. Stokes, W. A., Ferwerda, J. A., Walter, B., and Greenberg, D. P. 2004. Perceptual illumination components: a new approach to efficient, high quality global illumination rendering. ACM Trans. Graph. 23, 3, 742--749.
54. Suykens, F. and Willems, Y. 2001. Path differentials and applications. In Proceedings of the Rendering Techniques'01 (EG Workshop on Rendering). 257--268.
55. Walter, B., Arbree, A., Bala, K., and Greenberg, D. P. 2006. Multidimensional lightcuts. ACM Trans. Graph. 25, 3, 1081--1088.
56. Ward, G. J. and Heckbert, P. 1992. Irradiance gradients. In Proceedings of the Workshop on Rendering (EG'92). 85--98.
57. Whitted, T. 1980. An improved illumination model for shaded display. Comm. ACM 23, 6, 343--349.
58. Zhou, T., Chen, J., and Pullen, M. 2007. Accurate depth of field simulation in real time. Comput. Graph. Forum 26, 1, 15--23.

Index Terms

• Computing methodologies

  • Computer graphics

    • Rendering

      • Ray tracing