Near-field illumination for mixed reality with delta radiance fields

ABSTRACT

Apart from geometric registration, fusing synthetic objects with a real context requires believable interaction of real and virtual light. Where shadows provide the visual cues to locate a synthetic object in a real scene and transferred light from real light sources let it appear in harmony with its surroundings, the mutual indirect interaction of illumination is a necessary detail to convince an observer that the rendered result is not merely augmented but part of the scene. Such a mixed reality (MR) system has applications in movie production, advertisement of unfinished products or cultural heritage visualization. While there are a range of relighting tools available for offline renderers or static scenes (e.g. photos), interactive MR systems usually disregard proper lighting entirely or greatly simplify shading. A method often employed is to merge virtual light and shadows cast from synthetic objects with a real background with Differential Rendering, leaving out any other light interaction such as indirect light bounces. Attempts have been made to resolve this issue with Instant Radiosity [Knecht et al. 2010; Lensing and Broll 2012]. To suppress flickering, a large number of virtual point lights (VPL) is necessary, drastically taxing execution speed. We propose to model all light, virtual and real, as a unified radiance field instead to avoid performance issues from oversampling and to maintain temporal coherence.

References

1. Crassin, C., Neyret, F., Sainz, M., Green, S., and Eisemann, E. 2011. Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum.
2. Kaplanyan, A., and Dachsbacher, C. 2010. Cascaded light propagation volumes for real-time indirect illumination. In Proc. of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, I3D '10, 99--107.
3. Knecht, M., Traxler, C., Mattausch, O., Purgathofer, W., and Wimmer, M. 2010. Differential Instant Radiosity for Mixed Reality. In 2010 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 99--107.
4. Lensing, P., and Broll, W. 2012. Instant indirect illumination for dynamic mixed reality scenes. 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 109--118.