High-quality computational imaging through simple lenses

Abstract

Modern imaging optics are highly complex systems consisting of up to two dozen individual optical elements. This complexity is required in order to compensate for the geometric and chromatic aberrations of a single lens, including geometric distortion, field curvature, wavelength-dependent blur, and color fringing.

In this article, we propose a set of computational photography techniques that remove these artifacts, and thus allow for postcapture correction of images captured through uncompensated, simple optics which are lighter and significantly less expensive. Specifically, we estimate per-channel, spatially varying point spread functions, and perform nonblind deconvolution with a novel cross-channel term that is designed to specifically eliminate color fringing.

References

1. Beck, A. and Teboulle, M. 2009. A fast iterative shrinkage-thresholding alogorithm for linear inverse problems. SIAM J. Image Sci. 2, 182--202.
2. Brauers, J., Seiler, C., and Aach, T. 2010. Direct psf estimation using a random noise target. In Proceedings of the International Society for Optical Engineering (SPIE'10). 75370B.
3. Chambolle, A. and Pock, T. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120--145.
4. Chan, S., Khoshabeh, R., Gibson K., Gill, P., and Nguyen, T. 2011. IEEE Trans. Image Proc. 20, 11, 3097--3111.
5. Cho, T. S., Joshi, N., Zitnick, C. L., Kang, S. B., Szeliski, R., and FREEMAN, W. T. 2010. A content-aware image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 169--176.
6. Cho, T. S., Zitnick, C. L., Joshi, N., Kang, S. B., Szeliski, R., and Freeman, W. T. 2012. Image restoration by matching gradient distributions. IEEE Trans. Pattern Anal. Mach. Intell. 34, 4, 683--694.
7. Chung, S. W., Kim, B.-K., and Song, W.-J. 2009. Detecting and eliminating chromatic aberration in digital images. In Proceedings of the 16^th IEEE International Conference on Image Processing (ICIP'12). 3905--3908.
8. Combettes, B. L. and Pesquet, J.-C. 2011. Proximal splitting methods in signal processing. In Fixed Point Algorithms for Inverse Problems in Science and Engineering. Springer, 185--212.
9. Cossarit, O. and Nayar, S. 2010. Spectral focal sweep. Extended depth of field from chromactic aberrations sweep. In Proceedings of the IEEE Conference on Computational Photography (ICCP'10). 1--8.
10. Friedlander, M. P. 2011. Sparse optimization minitutorial. http://www. ca.ubc./∼mpf/papers/-tutorial-siopt11.pdf
11. Guass, C. 1841. Dioptrische Untersuchungen. Dieterich.
12. Gibson, J. D. and Bovik, A., Eds. 2000. Handbook of Image and Video Processing. Academic Press.
13. Goldluecke, B. and Cremers, D. 2011. Introducing total curvature for image processing. In Proceedings of the International Conference on Computer Vision. 1267--1274.
14. Goldstein, T. and Osher, S. 2009. The split bregman method for l1-regularized problems. SIAM J. Image Sci. 2, 323--343.
15. Guichard, F., Nguyen, H. P., Tessires, R., Pyanet, M., Tarchouna, I., and Cao, F. 2009. Extended depths-of-field using sharpness transport across color channels. In Proceeding of the International Society for Optical Engineering (SPIE'09).
16. Joshi, N., Lawrence, C., Zitnick, R., Szeliski, D., and Kriegman, J. 2009. Imagr deblurring and denoising using color priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1550--1557.
17. Joshi, N., Szeliski, R., and Kriegman, J. 2008. Psf estimation using sharp edge prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
18. Kang, S. B. 2007. Automatic removal of chromatic aberration from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
19. Kee, E., Paris, S., Chen, S., and Wang, J. 2011. Modeling and removing spatially varying optical blur. In Proceedings of the IEEE Conference on Computational Photography (ICCP'11). 1--8.
20. Krishnan, D. and Fergus, R. 2009. Fast image deconvolution using hyper-laplacian priors. Adv. Neural Inf. Process. Syst. 22, 1--9.
21. Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coated aperature. ACM Trans. Graph. 26, 3.
22. Lucy, L. 1974. An iterative technique for the rectification of observed distributions. The Astronom. J. 79, 745--.
23. Mahajan, V. N. 1991. Aberration Theory Made Simple. SPIE.
24. Nesterov, Y. 2005. Smooth minimization of non smooth functions. Math. Program. 103, 127--152.
25. Rashed, R. 1990. A pioneer in anaclastics: Iben sahl on burning mirrors and lenses. Isis 81, 3, 464--491.
26. Richardson, W. H. 1972. Bayesian-based iterative method of image restoration. J. Optical Soc. Amer. 62, 55--59.
27. Rowland, T. 2012. Operator norm. http://mathworld.wolfram.com/OperatorNorm.html.
28. Rump, M. and Klein, R. 2010. Spectralization: Reconstructing spectra from spare data. In Proceedings of the 21^st Eurographics Conference on Rendering Techniques, J. Lawrence and M. Stamminger, Eds. Eurographics Association, 1347--1354.
29. Schuler, C. J., Hirsch, M., Harmeling, S., and Scholkopf, B. 2011. Non-stationary correction of optical abberations. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'11). 659--666.
30. Seidel, L. 1857. Uber die theorie der fehler, mit welchen die durch optische instrumente gesehenen bilder behaftet sind, und uber die mathe matischen bedingungen ihrer aufhebung. Abhandlungen der Naturwissenschaftlich-Technischen Commission bei der Konigl. Bayerischen Akademie der Wissen schaften in Munchen. Cotta.
31. Shih, Y., Guenter, V., and Joshi, N. 2012. Image enhancement using calibrated lens simulations. In Proceedings of the European Conference on Computer Vision (ECCV'12), A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, Eds. Lecture Notes in Computer Science, vol. 7575. Springer, 42--56.
32. Trimeche, M., Paliy, D., Vehvilainen, M., and Katkovnik, V. 2005. Multichannel image deblurring of raw color components. In Proceedings of the SPIE Conference on Computational Imaging. 169--178.
33. Wiener, N. 1964. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. The MIT Press.
34. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2007. Image deblurring with blurred noisy image pairs. ACM Trans. Graph. 26, 3, 1.
35. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2008. Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Trans. Graph. 27, 3, 74:1--74:10.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Document scanning

      • Graphics recognition and interpretation

• Computing methodologies

  • Computer graphics

    • Image manipulation