Learning based compression for real-time rendering of surface light fields

ABSTRACT

Photo-realistic rendering in real-time is a key challenge in computer graphics. A number of techniques where the light transport in a scene is pre-computed, compressed and used for real-time image synthesis have been proposed, e.g. [Ramamoorthi 2009]. We extend on this idea and present a technique where the radiance distribution in a scene, including arbitrarily complex materials and light sources, is pre-computed and stored as surface light fields (SLF) at each surface. An SLF describes the full appearance of each surface in a scene as a 4D function over the spatial and angular domains. An SLF is a complex data set with a large memory footprint often in the order of several GB per object in the scene. The key contribution in this work is a novel approach for compression of SLFs enabling real-time rendering of complex scenes. Our learning-based compression technique is based on exemplar orthogonal bases (EOB) [Gurumoorthy et al. 2010], and trains a compact dictionary of full-rank orthogonal basis pairs with sparse coefficients. Our results outperform the widely used CPCA method [Miandji et al. 2011] in terms of storage cost, visual quality and rendering speed. Compared to PRT techniques for real-time global illumination, our approach is limited to static scenes but can represent high frequency materials and any type of light source in a unified framework.

References

1. Gurumoorthy, K., Rajwade, A., Banerjee, A., and Rangarajan, A. 2010. A method for compact image representation using sparse matrix and tensor projections onto exemplar orthonormal bases. Image Processing, IEEE Transactions on 19, 2 (feb.) 322--334.
2. Miandji, E., Kronander, J., and Unger, J. 2011. Geometry independent surface light fields for real time rendering of precomputed global illumination. In SIGRAD 2011, Linkping University Electronic Press, 27--34.
3. Ramamoorthi R. 2009. Precomputation-based rendering. Found. Trends. Comput. Graph. Vis. 3, 4 (Apr.), 281--369.