Adaptive image synthesis for compressive displays

Abstract

Recent years have seen proposals for exciting new computational display technologies that are compressive in the sense that they generate high resolution images or light fields with relatively few display parameters. Image synthesis for these types of displays involves two major tasks: sampling and rendering high-dimensional target imagery, such as light fields or time-varying light fields, as well as optimizing the display parameters to provide a good approximation of the target content.

In this paper, we introduce an adaptive optimization framework for compressive displays that generates high quality images and light fields using only a fraction of the total plenoptic samples. We demonstrate the framework for a large set of display technologies, including several types of auto-stereoscopic displays, high dynamic range displays, and high-resolution displays. We achieve significant performance gains, and in some cases are able to process data that would be infeasible with existing methods.

References

1. Adelson, E. H., and Bergen, J. R. 1991. The plenoptic function and the elements of early vision. In Comp. Models of Visual Processing, 3--20.
2. Bertsekas, D. 1997. A New Class of Incremental Gradient Methods for Least Squares Problems. SIAM Journal on Optimization 7, 4, 913--926.
3. Cook, R., Porter, T., and Carpenter, L. 1984. Distributed ray tracing. In Proc. SIGGRAPH, 137--145.
4. Cossairt, O. S., Napoli, J., Hill, S. L., Dorval, R. K., and Favalora, G. E. 2007. Occlusion-Capable Multiview Volumetric Three-Dimensional Display. Applied Optics 46, 8, 1244--1250.
5. Didyk, P., Eisemann, E., Ritschel, T., Myszkowski, K., and Seidel, H.-P. 2010. Apparent Display Resolution Enhancement for Moving Images. ACM Trans. Graph. (SIGGRAPH) 29, 4, 113:1--113:8.
6. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., and Seidel, H.-P. 2011. A Perceptual Model for Disparity. ACM Trans. Graph. (SIGGRAPH) 30, 4, 96:1--96:10.
7. Egan, K., Tseng, Y.-T., Holzschuch, N., Durand, F., and Ramamoorthi, R. 2009. Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur. ACM Trans. Graph. (SIGGRAPH) 28, 3, 93:1--93:13.
8. Egan, K., Hecht, F., Durand, F., and Ramamoorthi, R. 2011. Frequency Analysis and Sheared Filtering for Shadow Light Fields of Complex Occluders. ACM Trans. Graph. 30, 2, 9:1--9:13.
9. Friedlander, M., and Schmidt, M. 2012. Hybrid deterministic-stochastic methods for data fitting. SIAM Journal on Scientific Computing 34, 3, 1380--1405.
10. Glassner, A. S., Fishkin, K. P., Marimont, D. H., and Stone, M. C. 1995. Device-Directed Rendering. ACM Trans. Graph. 14, 1, 58--76.
11. Gotoda, H. 2011. Reduction of Image Blurring in an Autostereoscopic Multilayer Liquid Crystal Display. In Proc. SPIE Stereoscopic Displays and Applications XXII, vol. 7863, 21:1--21:7.
12. Gregson, J., Krimerman, M., Hullin, M. B., and Heidrich, W. 2012. Stochastic Tomography and its Applications in 3D Imaging of Mixing Fluids. ACM Trans. Graph. (SIGGRAPH) 31, 4, 52:1--52:10.
13. Grosse, M., Wetzstein, G., Grundhöfer, A., and Bimber, O. 2010. Coded Aperture Projection. ACM Trans. Graph. 29, 3, 22:1--22:12.
14. Hastings, W. 1970. Monte carlo sampling methods using markov chains and their applications. Biometrika 57, 1, 97--109.
15. Ives, F. E., 1903. Parallax Stereogram and Process of Making Same. U.S. Patent 725,567.
16. Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. (SIGGRAPH) 26, 40:1--40:10.
17. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-Adaptive Parallax Barriers: Optimizing Dual-Layer 3D Displays using Low-Rank Light Field Factorization. ACM Trans. Graph. (SIGGRAPH Asia) 29, 163:1--163:10.
18. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 30, 186:1--186:9.
19. Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal Light Field Reconstruction for Rendering Distribution Effects. ACM Trans. Graph. (SIGGRAPH) 30, 4, 55:1--55:12.
20. Lehtinen, J., Aila, T., Laine, S., and Durand, F. 2012. Reconstructing the Indirect Light Field for Global Illumination. ACM Trans. Graph. (SIGGRAPH) 31, 4, 51:1--51:10.
21. Li, T.-M., Wu, Y.-T., and Chuang, Y.-Y. 2012. SURE-based Optimization for Adaptive Sampling and Reconstruction. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 194:1--194:9.
22. Lippmann, G. 1908. Épreuves réversibles donnant la sensation du relief. Journal of Physics 7, 4, 821--825.
23. Mantiuk, R., Kim, K. J., Rempel, A. G., and Heidrich, W. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in all Luminance Conditions. ACM Trans. Graph. (SIGGRAPH) 30, 4, 40:1--40:14.
24. Marwah, K., Wetzstein, G., Bando, Y., and Raskar, R. 2013. Compressive Ligth Field Photography using Over-complete Dictionaries and Optimized Projections. ACM Trans. Graph. (SIGGRAPH) 32, 4, 1--11.
25. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, E. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics 21, 1087--1092.
26. Pharr, M., and Humphreys, G. 2010. Physically based rendering: From theory to implementation. Morgan Kaufmann.
27. Sajadi, B., Gopi, M., and Majumder, A. 2012. Edge-guided Resolution Enhancement in Projectors via Optical Pixel Sharing. ACM Trans. Graph. (SIGGRAPH) 31, 4, 79:1--79:122.
28. Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., Whitehead, L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High Dynamic Range Display Systems. ACM Trans. Graph. (SIGGRAPH) 23, 3, 760--768.
29. Sen, P., and Darabi, S. 2011. Compressive Rendering: A Rendering Application of Compressed Sensing. IEEE TVCG 17, 4, 487--499.
30. Sen, P., Darabi, S., and Xiao, L. 2011. Compressive Rendering of Multidimensional Scenes. In LNCS "Video Processing and Computational Video", 152--183.
31. Soler, C., Subr, K., Durand, F., Holzschuch, N., and Sillion, F. 2009. Fourier Depth of Field. ACM Trans. Graph. 28, 2, 18:1--18:12.
32. Trentacoste, M., Heidrich, W., Whitehead, L., Seetzen, H., and Ward, G. 2007. Photometric Image Processing for High Dynamic Range Displays. JVCIR 18, 5, 439--451.
33. Veach, E., and Guibas, L. J. 1997. Metropolis Light Transport. In Proc. SIGGRAPH, 65--76.
34. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. (SIGGRAPH) 30, 95:1--95:11.
35. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 80:1--80:11.
36. Widrow, B., and Stearns, S. 1985. Adaptive signal processing, vol. 1. Prentice-Hall.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering