Image deblurring with blurred/noisy image pairs

ABSTRACT

Taking satisfactory photos under dim lighting conditions using a hand-held camera is challenging. If the camera is set to a long exposure time, the image is blurred due to camera shake. On the other hand, the image is dark and noisy if it is taken with a short exposure time but with a high camera gain. By combining information extracted from both blurred and noisy images, however, we show in this paper how to produce a high quality image that cannot be obtained by simply denoising the noisy image, or deblurring the blurred image alone.

Our approach is image deblurring with the help of the noisy image. First, both images are used to estimate an accurate blur kernel, which otherwise is difficult to obtain from a single blurred image. Second, and again using both images, a residual deconvolution is proposed to significantly reduce ringing artifacts inherent to image deconvolution. Third, the remaining ringing artifacts in smooth image regions are further suppressed by a gain-controlled deconvolution process. We demonstrate the effectiveness of our approach using a number of indoor and outdoor images taken by off-the-shelf hand-held cameras in poor lighting environments.

References

1. Bardsley, J., Jefferies, S., Nagy, J., and Plemmons, R. 2006. Blind iterative restoration of images with spatially-varying blur. In Optics Express, 1767--1782.
2. Bascle, B., Blake, A., and Zisserman, A. 1996. Motion deblurring and super-resolution from an image sequence. In Processings of ECCV, vol. II, 573--582.
3. Ben-Ezra, M., and Nayar, S. K. 2003. Motion deblurring using hybrid imaging. In Processings of CVPR, vol. I, 657--664.
4. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. Graph. 24, 3, 845--852.
5. Buades, A., Coll, B., and Morel, J. M. 2005. A non-local algorithm for image denoising. In Proceedings of CVPR, vol. II, 60--65.
6. Canny, J. 1986. A computational approach to edge detection. IEEE Trans. on PAMI. 8, 6, 679--698.
7. Caron, J. N., M., N. N., and J., R. C. 2002. Noniterative blind data restoration by use of an extracted filter function. Applied optics (Appl. opt.) 41, 32, 68--84.
8. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH, 369--378.
9. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In Proceedings of SIGGRAPH, 257--266.
10. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 3, 673--678.
11. Engl, H. W., Hanke, M., and Neubauer, A. 2000. Regularization of Inverse Problems. Kluwer Academic.
12. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proceedings of SIGGRAPH, 249--256.
13. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing camera shake from a single photograph. In ACM Trans. Graph., vol. 25, 787--794.
14. Geman, D., and Reynolds, G. 1992. Constrained restoration and the recovery of discontinuities. IEEE Trans. on PAMI. 14, 3, 367--383.
15. H. Richardson, W. 1972. Bayesian-based iterative method of image restoration. JOSA, A 62, 1, 55--59.
16. Jalobeanu, A., Blanc-Feraud, L., and Zerubia, J. 2002. Estimation of blur and noise parameters in remote sensing. In Proceedings of ICASSP, 249--256.
17. Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Bayesian correction of image intensity with spatial consideration. In Proceedings of ECCV, 342--354.
18. Kundur, D., and Hatzinakos, D. 1996. Blind image deconvolution. IEEE Signal Processing Magazine. 13, 3, 43--64.
19. Levin, A. 2006. Blind motion deblurring using image statistics. In Advances in Neural Information Processing Systems (NIPS).
20. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. 24, 3, 836--844.
21. Lim, S. H., and Silverstein, D. A. 2006. Method for deblurring an image. US Patent Application, Pub. No. US2006/0187308 A1, Aug 24, 2006.
22. Liu, X., and Gamal, A. 2001. Simultaneous image formation and motion blur restoration via multiple capture. Proceedings of ICASSP..
23. Liu, C., Freeman, W., Szeliski, R., and Kang, S. 2006. Noise estimation from a single image. In Proceedings of CVPR, vol. I, 901--908.
24. Neelamani, R., Choi, H., and Baraniuk, R. 2004. ForWaRd: Fourier-wavelet regularized deconvolution for ill-conditioned systems. IEEE Trans. on Signal Processing 52, 2, 418--433.
25. Nikon. 2005. http://www.nikon.co.jp/main/eng/portfolio/about/technology/nikon_technology/vr_e/index.htm.
26. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on PAMI 12, 7, 629--639.
27. Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., and Toyama., K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 3, 664--672.
28. Portilla, J., Strela, V., Wainwright, M., and Simoncelli., E. P. 2003. Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans. on Image Processing 12, 11, 1338--1351.
29. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3, 795--804.
30. Rav-Acha, A., and Peleg, S. 2000. Restoration of multiple images with motion blur in different directions. IEEE Workshop on Applications of Computer Vision.
31. Rav-Acha, A., and Peleg, S. 2005. Two motion-blurred images are better than one. Pattern Recogn. Lett. 26, 3, 311--317.
32. Reeves, S. J., and Mersereau, R. M. 1992. Blur identification by the method of generalized cross-validation. IEEE Trans. on Image Processing. 1, 3, 301--311.
33. Roth, S., and Black, M. J. 2005. Fields of experts: A framework for learning image priors. In Proceedings of CVPR, vol. II, 860--867.
34. Rudin, L., Osher, S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Phys. D. 60, 259--268.
35. Simoncelli, E. P., and Adelson, E. H. 1996. Noise removal via bayesian wavelet coring. In Proceedings of ICIP, vol. I, 379--382.
36. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proceedings of ICCV, 839--846.
37. Y. Yitzhaky, I. Mor, A. L., and Kopeika., N. 1998. Direct method for restoration of motion blurred images. J. Opt. Soc. Am., A 15, 6, 1512--1519.
38. Zarowin, C. B. 1994. Robust, noniterative, and computationally efficient modification of vab cittert deconvolution optical figuring. JOSA, A 11, 10, 2571--2583.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Interest point and salient region detections

  • Computer graphics

    • Image manipulation

  • Machine learning