Abstract
Light field images, for example, taken with plenoptic cameras, offer interesting post-processing opportunities, including depth-of-field management, depth estimation, viewpoint selection, and 3D image synthesis. Like most capture devices, however, plenoptic cameras have a limited dynamic range, so that over- and under-exposed areas in plenoptic images are commonplace. We therefore present a straightforward and robust plenoptic reconstruction technique based on the observation that vignetting causes peripheral views to receive less light than central views. Thus, corresponding pixels in different views can be used to reconstruct illumination, especially in areas where information missing in one view is present in another. Our algorithm accurately reconstructs under- and over-exposed regions (known as declipping), additionally affording an increase in peak luminance by up to two f-stops, and a comparable lowering of the noise floor. The key advantages of this approach are that no hardware modifications are necessary to improve the dynamic range, that no multiple exposure techniques are required, and therefore that no ghosting or other artifacts are introduced.
- Mekides Assefa Abebe, Tania Pouli, Jonathan Kervec, and Chaker Larabi. 2015. Color clipping and over-exposure correction. In Eurographics Symposium on Rendering—Experimental Ideas 8 Implementations, Jaakko Lehtinen and Derek Nowrouzezahrai (eds.). The Eurographics Association.Google Scholar
- Edward H. Adelson and John Y. A. Wang. 1992. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2 (1992), 99--106. Google Scholar
Digital Library
- Kurt Barton Akeley, Brian Cabral, Colvin Pitts, Chia-Kai Liang, Bennett Wilburn, Timothy James Knight, and Yi-Ren Ng. 2014. Compensation for sensor saturation and microlens modulation during light-field image processing. (2014). Patent No. US 9386288 B2, filed December 15, 2014, issued July 5, 2016.Google Scholar
- Tom E. Bishop and Paolo Favaro. 2012. The light field camera: Extended depth of field, aliasing, and superresolution. IEEE Trans. Pattern Anal. Mach. Intell. 34, 5 (2012), 972--986. Google Scholar
Digital Library
- Tom E. Bishop, Sara Zanetti, and Paolo Favaro. 2009. Light field superresolution. In Proceedings of the IEEE International Conference on Computational Photography. 1--9.Google Scholar
Cross Ref
- D. Cho, M. Lee, S. Kim, and Y.-W. Tai. 2013. Modeling the calibration pipeline of the Lytro camera for high quality light-field image reconstruction. In IEEE International Conference on Computer Vision. 3280--3287. Google Scholar
Digital Library
- F. R. Chung. 1997. Spectral Graph Theory. American Mathematical Society.Google Scholar
Digital Library
- Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13, 9 (2004), 1200--1212. Google Scholar
Digital Library
- Donald Dansereau, Oscar Pizarro, and Stefan Williams. 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1027--1034. Google Scholar
Digital Library
- Paul Debevec and Jitendra Malik. 1997. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH’97: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. 369--378. Google Scholar
Digital Library
- Alexei A. Efros and Thomas K. Leung. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, Vol. 2. 1033--1038. Google Scholar
Digital Library
- Juliet Fiss, Brian Curless, and Richard Szeliski. 2014. Refocussing plenoptic images using depth-adaptive splatting. In Proceedings of the IEEE International Conference on Computational Photography. 1--9.Google Scholar
- Orazio Gallo, N. Gelfandz, Wei-Chao Chen, Marius Tico, and Kari Pulli. 2009. Artifact-free high dynamic range imaging. In Proceedings of the 2009 IEEE International Conference on Computational Photography (ICCP’09). IEEE, 1--7.Google Scholar
Cross Ref
- Elena Garces, Jose I. Echevarria, Wen Zhang, Hongzhi Wu, Kun Zhou, and Diego Gutierrez. 2016. Intrinsic light fields. https://arxiv.org/abs/1608.04342 (2016).Google Scholar
- Todor Georgiev and Andrew Lumsdaine. 2010. Rich image capture with plenoptic cameras. In Proceedings of the IEEE International Conference on Computational Photography. 1--8.Google Scholar
Cross Ref
- Todor Georgiev, Andrew Lumsdaine, and Sergio Goma. 2009. High dynamic range image capture with plenoptic 2.0 camera. In Proceedings of Signal Recovery and Synthesis. Optical Society of America, SWA7P.Google Scholar
Cross Ref
- Todor Georgiev, Ke Colin Zheng, Brian Curless, David Salesin, Shree Nayar, and Chintan Intwala. 2006. Spatio-angular resolution tradeoff in integral photography. In Proceedings of the Eurographics Symposium on Rendering. 263--272. Google Scholar
Digital Library
- Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. 1996. The lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 43--54. Google Scholar
Digital Library
- Miguel Granados, Hans-Peter Seidel, and Hendrik P. A. Lensch. 2008. Background estimation from non-time sequence images. In Proceedings of Graphics Interface 2008. 33--40. Google Scholar
Digital Library
- Christopher Hahne, Amar Aggoun, Vladan Velisavljevic, Susanne Fiebig, and Matthias Pesch. 2016. Refocusing distance of a standard plenoptic camera. Optics Express 24, 19 (2016), 21521--21540.Google Scholar
Cross Ref
- Roarke Horstmeyer, Gary Euliss, Ravindra Athale, and Marc Levoy. 2009. Flexible multimodal camera using a light field architecture. In Proceedings of the IEEE Conference on Computational Photography. 1--8.Google Scholar
Cross Ref
- W. Hu, G. Cheung, and M. Kazui. 2016. Graph-based dequantization of block-compressed piecewise smooth images. IEEE Signal Process. Lett. 23, 2 (2016), 242--246.Google Scholar
Cross Ref
- W. Hu, X. Li, G. Cheung, and O. Au. 2013. Depth map denoising using graph-based transform and group sparsity. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing.Google Scholar
- Wei Hu, Mozhdeh Seifi, and Erik Reinhard. 2017. Optical center estimation for lenslet-based plenoptic cameras. In 19th IEEE International Workshop on Multimedia Signal Processing (MMSP’17).Google Scholar
Cross Ref
- Ivo Ihrke, Timo Stich, Heiko Gottschlich, Marcus Magnor, and Hans-Peter Seidel. 2008. Fast incident light field acquisition and rendering. J. WSCG 16, 1 (2008), 25--32.Google Scholar
- Aaron Isaksen, Leonard McMillan, and Steven J Gortler. 2000. Dynamically reparameterized light fields. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. 297--306. Google Scholar
Digital Library
- Frederic E. Ives. 1903. Parallax Stereogram and Process of Making Same. (Filed Sept. 25, 1902, issued Apr. 14, 1903). U.S. Patent US725567 A.Google Scholar
- E. A. Khan, A. O. Akyuz, and E. Reinhard. 2006. Ghost removal in high dynamic range images. In IEEE International Conference on Image Processing. 2005--2008.Google Scholar
- Seon Joo Kim and Marc Pollefeys. 2008. Robust radiometric calibration and vignetting correction. IEEE Trans. Pattern Anal. Mach. Intell. 30, 4 (2008), 562--576. Google Scholar
Digital Library
- Marc Levoy and Pat Hanrahan. 1996. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 31--42. Google Scholar
Digital Library
- Marc Levoy, Ren Ng, Andrew Adams, Matthew Footer, and Mark Horowitz. 2006. Light field microscopy. ACM Trans. Graph. 25, 3 (2006), 924--934. Google Scholar
Digital Library
- Chia-Kai Liang and Ravi Ramamoorthi. 2015. A light transport framework for lenslet light field cameras. ACM Trans. Graph. 34, 2 (March 2015), 16:1--16:19. Google Scholar
Digital Library
- Andrew Lumsdaine and Todor Georgiev. 2008. Full Resolution Lightfield Rendering.Technical report, Indiana University and Adobe Systems.Google Scholar
- Andrew Lumsdaine and Todor Georgiev. 2009. The focused plenoptic camera. In IEEE International Conference on Computational Photography (ICCP’09). 1--8.Google Scholar
Cross Ref
- U. Luxburg. 2007. A tutorial on spectral clustering. In Statistics and Computing, Vol. 17, No. 4. 395--416. Google Scholar
Digital Library
- Henrique S. Malvar, Li-Wei He, and Ross Cutler. 2004. High-quality linear interpolation for demosaicking of bayer-patterned color images. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 3. iii--485.Google Scholar
- Alkhazur Manakov, John Restrepo, Oliver Klehm, Ramon Hegedus, Elmar Eisemann, Hans-Peter Seidel, and Ivo Ihrke. 2013. A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging. ACM Trans. Graph. 32, 4 (2013), 47--1. Google Scholar
Digital Library
- S. Mann and R. W. Picard. 1995. On being ‘undigital’ with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proceedings of IS&T 48th Annual Conference. 422--428.Google Scholar
- Syed Z. Masood, Jiejie Zhu, and Marshall F. Tappen. 2009. Automatic correction of saturated regions in photographs using cross-channel correlation. Comput. Graph. Forum 28, 7 (2009), 1861--1869.Google Scholar
Cross Ref
- Vincent Masselus, Pieter Peers, Philip Dutré, and Yves D. Willems. 2003. Relighting with 4D incident light fields. ACM Trans. Graph. 22, 3 (2003), 613--620. Google Scholar
Digital Library
- Peyman Milanfar. 2013. A tour of modern image filtering: New insights and methods, both practical and theoretical. IEEE Signal Process. Mag. 30, 1 (2013), 106--128.Google Scholar
Cross Ref
- T. Mitsunaga and S. K. Nayar. 1999. Radiometric self calibration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. 374--380.Google Scholar
- Shree K. Nayar and Srinivasa G. Narasimhan. 2002. Assorted pixels: Multisampled imaging with structural models. In Proceedings of the European Conference on Computer Vision. 636--652. Google Scholar
Digital Library
- Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, and Pat Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. Comput. Sci. Tech. Rep. 2, 11 (2005), 1--11.Google Scholar
- Christian Perwass and Lennart Wietzke. 2012. Single lens 3D-camera with extended depth-of-field. In IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, 829108--829108.Google Scholar
- Colvin Pitts, Timothy James Knight, Chia-Kai Liang, and Yi-Ren Ng. 2015. Generating dolly zoom effect using light field image data. (2015). Patent No. US 8971625 B2, filed June 23, 2014, issued March 3, 2015.Google Scholar
- Ramesh Raskar, Amit Agrawal, Cyrus A. Wilson, and Ashok Veeraraghavan. 2008. Glare aware photography: 4D Ray sampling for reducing glare effects of camera lenses. ACM Trans. Graph. 27, 3 (2008), 56:1--56:10. Google Scholar
Digital Library
- E. Reinhard and K. Devlin. 2005. Dynamic range reduction inspired by photoreceptor physiology. IEEE Trans. Vis. Comput. Graph. 11, 1 (2005), 13--24. Google Scholar
Digital Library
- E. Reinhard, W. Heidrich, P. Debevec, S. Pattanaik, G. Ward, and K. Myszkowski. 2010. High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting (2nd ed.). Morgan Kaufmann.Google Scholar
- Erik Reinhard, Erum Arif Khan, Ahmet Oğuz Akyüz, and Garrett M. Johnson. 2008. Color Imaging: Fundamentals and Applications. A K Peters, Wellesley. Google Scholar
Digital Library
- Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. 2002. Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 3 (2002), 267--276. Google Scholar
Digital Library
- M. A. Robertson, S. Borman, and R. L. Stevenson. 2003. Estimation-theoretic approach to dynamic range enhancement using multiple exposures. Journal of Electronic Imaging 12 (2003), 219--228. Issue 2.Google Scholar
Cross Ref
- Neus Sabater, Mozhdeh Seifi, Valter Drazic, Gustavo Sandri, and Patrick Pérez. 2014. Accurate disparity estimation for plenoptic images. In Computer Vision — ECCV 2014 Workshops. Springer, 548--560.Google Scholar
- Yoav Y. Schechner and Shree K. Nayar. 2003. Generalized mosaicing: High dynamic range in a wide field of view. International Journal of Computer Vision 53, 3 (2003), 245--267. Google Scholar
Digital Library
- David C. Schedl, Clemens Birklbauer, and Oliver Bimber. 2014. Coded exposure HDR light-field video recording. Computer Graphics Forum 33, 2 (2014), 33--42. Google Scholar
Digital Library
- Pradeep Sen, Nima Khademi Kalantari, Maziar Yaesoubi, Soheil Darabi, Dan B. Goldman, and Eli Shechtman. 2012. Robust patch-based HDR reconstruction of dynamic scenes. ACM Trans. Graph. 31, 6 (2012), 203:1--203:11. Google Scholar
Digital Library
- D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst. 2013. The emerging field of signal processing on graphs. IEEE Signal Processing Magazine 30, 3 (2013), 83--98.Google Scholar
Cross Ref
- Thiow Keng Tan, Choong Seng Boon, and Yuya Suzuki. 2006. Intra prediction by template matching. In Proceedings of the IEEE International Conference on Image Processing. 1693--1696.Google Scholar
Cross Ref
- Michael Tao, Sunil Hadap, Jitendra Malik, and Ravi Ramamoorthi. 2013. Depth from combining defocus and correspondence using light-field cameras. In Proceedings of the IEEE International Conference on Computer Vision. 673--680. Google Scholar
Digital Library
- J. Unger, S. Gustavson, P. Larsson, and A. Ynnerman. 2008. Free form incident light fields. Computer Graphics Forum 27, 4 (2008), 1293--1301. Google Scholar
Digital Library
- J. Unger, A. Wenger, T. Hawtins, A. Gardner, and P. Debevec. 2003. Capturing and rendering with incident light fields. In Proceedings of the Eurographics Symposium on Rendering. 141--149. Google Scholar
Digital Library
- Ashok Veeraraghavan, Ramesh Raskar, Amit Agrawal, Ankit Mohan, and Jack Tumblin. 2007. Dappled photography: Mask enhanced cameras for hyterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26, 3 (2007), 69. Google Scholar
Digital Library
- Lvdi Wang, Li-Yi Wei, Kun Zhou, Baining Guo, and Heung-Yeung Shum. 2007. High dynamic range image hallucination. In Proceedings of the 18th Eurographics conference on Rendering Techniques. Eurographics Association, 321--326. Google Scholar
Digital Library
- Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A Efros, and Ravi Ramamoorthi. 2016. A 4D light-field dataset and CNN architecture for material recognition. In Proceedings of the European Conference on Computer Vision. 121--138.Google Scholar
Cross Ref
- Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600--612. Google Scholar
Digital Library
- Sven Wanner and Bastian Goldluecke. 2014. Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 3 (2014), 606--619. Google Scholar
Digital Library
- Greg Ward. 2003. Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures. J. Graph. Tools 8, 2 (2003), 17--30.Google Scholar
Cross Ref
- Li-Yi Wei and Marc Levoy. 2001. Texture synthesis over arbitrary manifold surfaces. In SIGGRAPH’01: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. 355--360. Google Scholar
Digital Library
- Z. Yu, X. Guo, H. Ling, A. Lumsdaine, and J. Yu. 2013. Line assisted light field triangulation and stereo matching. In Proceedings of the IEEE International Conference on Computer Vision. 2792--2799. Google Scholar
Digital Library
- Zhan Yu, Jingji Yu, Andrew Lumsdaine, and Todor Georgiev. 2012. An analysis of color demosaicing in plenoptic cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 901--908. Google Scholar
Digital Library
- Yuanjie Zheng, Stephen Lin, Chandra Kambhamettu, Jingyi Yu, and Sing Bing Kang. 2009. Single-image vignetting correction. IEEE Trans. Pattern Anal. Mach. Intell. 31, 12 (2009), 2243--2256. Google Scholar
Digital Library
- Shan Zhu and Kai-Kuang Ma. 2000. A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9, 2 (2000), 287--290. Google Scholar
Digital Library
Index Terms
Over- and Under-Exposure Reconstruction of a Single Plenoptic Capture
Recommendations
RII: Renovating the irregular illumination of digital image archives
Digitization is critical for preserving valuable culture archives such as texts and images. Considering the physical characters of digital cameras or scanners and the artificial negligence, some distortions such as under-exposure or over-exposure often ...
Layered Light Field Reconstruction for Defocus Blur
We present a novel algorithm for reconstructing high-quality defocus blur from a sparsely sampled light field. Our algorithm builds upon recent developments in the area of sheared reconstruction filters and significantly improves reconstruction quality ...
Single-Shot Specular Surface Reconstruction with Gonio-Plenoptic Imaging
ICCV '15: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)We present a gonio-plenoptic imaging system that realizes a single-shot shape measurement for specular surfaces. The system is comprised of a collimated illumination source and a plenoptic camera. Unlike a conventional plenoptic camera, our system ...






Comments