Edge-guided resolution enhancement in projectors via optical pixel sharing

Abstract

Digital projection technology has improved significantly in recent years. But, the relationship of cost with respect to available resolution in projectors is still super-linear. In this paper, we present a method that uses projector light modulator panels (e.g. LCD or DMD panels) of resolution n X n to create a perceptually close match to a target higher resolution cn X cn image, where c is a small integer greater than 1. This is achieved by enhancing the resolution using smaller pixels at specific regions of interest like edges.

A target high resolution image (cn X cn) is first decomposed into (a) a high resolution (cn X cn) but sparse edge image, and (b) a complementary lower resolution (n X n) non-edge image. These images are then projected in a time sequential manner at a high frame rate to create an edge-enhanced image -- an image where the pixel density is not uniform but changes spatially. In 3D ready projectors with readily available refresh rate of 120Hz, such a temporal multiplexing is imperceptible to the user and the edge-enhanced image is perceptually almost identical to the target high resolution image.

To create the higher resolution edge image, we introduce the concept of optical pixel sharing. This reduces the projected pixel size by a factor of 1/c² while increasing the pixel density by c² at the edges enabling true higher resolution edges. Due to the sparsity of the edge pixels in an image we are able to choose a sufficiently large subset of these to be displayed at the higher resolution using perceptual parameters. We present a statistical analysis quantifying the expected number of pixels that will be reproduced at the higher resolution and verify it for different types of images.

References

1. Agrawal, A., and Raskar, R. 2007. Resolving objects at higher resolution using a single motion blurred image. IEEE CVPR.
2. Aliaga, D., Yeung, Y. H., Law, A. J., Sajadi, B., and Majumder, A. 2011. Fast high-resolution appearance editing using superimposed projections. ACM TOG.
3. Allen, W., and Ulichney, R. 2005. Wobulation: Doubling the addressed resolution of projection displays. SID.
4. Babu, R. S., and Murthy, K. E. S. 2011. A survey on the methods of super-resolution image reconstruction. IJCV 15, 2.
5. Baker, S., and Nayar, S. K. 1999. A theory of single-viewpoint catadioptric image formation. IJCV 35, 2.
6. Bala, K., Walter, B., and Greenberg, D. P. 2003. Combining edges and points for interactive high-quality rendering. ACM Transactions of Graphics (Siggraph).
7. Bhasker, E., Juang, R., and Majumder, A. 2007. Registration techniques for using imperfect and partially calibrated devices in planar multi-projector displays. IEEE TVCG 13, 6.
8. Chen, H., Sukthankar, R., Wallace, G., and Li, K. 2002. Scalable alignment of large-format multi-projector displays using camera homography trees. IEEE Vis.
9. Cole, F., and Finkelstein, A. 2010. Two fast methods for high-quality line visibility. IEEE TVCG 16, 5.
10. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? ACM TOG 28, 3.
11. C.Tomasi, and Manduchi, R. 1998. Bilateral filtering for gray and color images. ICCV.
12. Damera-Venkata, N., and Chang, N. L. 2009. Display supersampling. ACM TOG 28, 1.
13. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. ACM Siggraph.
14. DeCarlo, D., Finkelstein, A., and Rusinkiewicz, S. 2004. Interactive rendering of suggestive contours with temporal coherence. NPAR.
15. Dijk, J., van Grinkel, M., van Asselt, R. J., van Vliet, L., and Verbeek, P. W. 2003. A new sharpness measure based on gaussian lines and edges. CAIP.
16. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM TOG 21, 3.
17. Fattal, R. 2007. Image upsampling via imposed edges statistics. ACM TOG 26, 3.
18. Goldstein, E. B. 2001. Sensation and Perception. Wadsworth Publishing Company.
19. Hirsch, M., Lanman, D., Holtzman, H., and Raskar, R. 2009. BiDi screen: A thin, depth-sensing LCD for 3D interaction using lights fields. ACM TOG 28, 5.
20. Jaynes, C., and Ramakrishnan, D. 2003. Super-resolution composition in multi-projector displays. PROCAMS.
21. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM TOG.
22. Kuthirummal, S., and Nayar, S. K. 2006. Multiview radial catadioptric imaging for scene capture. ACM TOG.
23. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: optimizing dual-layer 3d displays using low-rank light field factorization. ACM TOG.
24. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization fields: Dynamic light field display using multi-layer LCDs. ACM TOG 30, 6.
25. Lazarev, A., and Palto, S. 2009. Materials for light efficient LCD. Society for Information Display (SID).
26. Levin, A., Fergus, R., Durand, F., and Freeman, B. 2007. Image and depth from a conventional camera with a coded aperture. ACM TOG 26, 3.
27. Levin, A., Sand, P., Cho, T. S., Durand, F., and Freeman, W. T. 2008. Motion-invariant photography. ACM TOG 27, 3.
28. Liang, C., Lin, T. H., Wong, B. Y., Liu, C., and Chen, H. H. 2008. Programmable aperture photography:multiplexed light field acquisition. ACM TOG 27, 3.
29. Lin, W. S., Gai, Y. L., and Kassim, A. A. 2006. Perceptual impact of edge sharpness in images. IEEE Proceedings on Vision, Image and Signal Processing.
30. Majumder, A., and Irani, S. 2007. Perception based contrast enhancement of images. ACM Transactions on Applied Perception 4, 3.
31. Majumder, A., and Stevens, R. 2005. Perceptual photometric seamlessness in tiled projection-based displays. ACM TOG 24.
32. Majumder, A., Brown, R., and Ghoroury, H. E. 2010. Display gamut reshaping for color emulation and balancing. IEEE CVPR Workshop on Projector Camera Systems (PROCAMS).
33. Majumder, A. 2005. Is spatial super-resolution possible with multiple overlapping projectors? ICASSP.
34. Nayar, S. K., Krishnan, G., Grossberg, M. D., and Raskar, R. 2006. Fast separation of direct and global components of a scene using high frequency illumination. ACM TOG 25.
35. Ran, X., and Farvardin, N. 1995. A perceptually motivated three component image model: Part i. IEEE TIP 4, 4.
36. Raskar, R., and Cohen, M. F. 1999. Image precision silhouette edges. ACM I3D.
37. Raskar, R., h Tan, K., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: Depth edge detection and stylized rendering using multi-flash imaging. ACM TOG 23, 3.
38. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: Motion deblurring using fluttered shutter. ACM TOG 25, 3.
39. Stupp, E. H., and Brennesholtz, M. S. 1999. Projection Displays. Wiley.
40. Sun, T., and Kelly, K. 2009. Compressive sensing hyperspectral imager. Computational Optical Sensing and Imaging, Optical Society of America.
41. Valois, R. L. D., and Valois, K. K. D. 1990. Spatial Vision. Oxford University Press.
42. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM TOG 26, 3.
43. Veeraraghavan, A., Reddy, D., and Raskar, R. 2010. Coded strobing photography: Compressive sensing of high-speed periodic events. IEEE PAMI.
44. Wakin, M., Laska, J., Duarte, M., Baron, D., Sarvotham, S., Takhar, D., Kelly, K., and Baraniuk, R. 2006. An architecture for compressive imaging. ICIP.
45. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM TOG 30, 4.
46. Wilburn, B., Joshi, N., Vaish, V., Talvala, E. V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. ACM TOG 24, 3.
47. Winkler, S. 2001. Visual fidelity and perceived quality: towards comprehensive metrics. Proceedings of SPIE 4299.

Index Terms

• Computing methodologies

  • Computer graphics

    • Rendering