skip to main content
research-article

Real-time Apparent Resolution Enhancement for Head-mounted Displays

Authors Info & Claims
Published:25 July 2018Publication History
Skip Abstract Section

Abstract

The insufficient pixel density of current head-mounted displays is one of the major obstacles in achieving immersive and fully engaging experiences. It is possible to overcome this physical limitation for moving content using software techniques. To this end, previous techniques utilized high-framerate displays and optimized for low-resolution images that, when shown on a display, can significantly increase the apparent spatial resolution. However, so far, all the proposed techniques require expensive optimization, which makes the techniques unsuitable for real-time applications. To overcome this problem, we present a novel method that can improve apparent resolution of such displays in real time. We replace expensive optimizations with a two-step filtering approach. Due to the efficiency of our technique, we can account not only for the motion in the scene but also for any motion in the perceived image introduced by movement of a user. This greatly extends the range of situations where the resolution enhancement can be achieved. In this paper, we present the derivation of the motion-flow-dependent filters and how they can be applied to increase the perceived resolution. To evaluate the performance of our technique, we conducted a user experiment which compares our method to alternative solutions regarding perceived resolution as well as overall quality and demonstrates the advantages of our technique.

References

  1. Will Allen and Robert Ulichney. 2005. 47.4: Invited Paper: Wobulation: Doubling the addressed resolution of projection displays. SID Symposium Digest of Technical Papers 36, 1 (2005), 1514--1517.Google ScholarGoogle Scholar
  2. Floraine Berthouzoz and Raanan Fattal. 2012a. Apparent resolution enhancement for motion videos. In Proceedings of the ACM Symposium on Applied Perception. ACM, 91--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Floraine Berthouzoz and Raanan Fattal. 2012b. Resolution enhancement by vibrating displays. ACM Transactions on Graphics (TOG) 31, 2 (2012), 15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Christine A Curcio, Kenneth R Sloan, Robert E Kalina, and Anita E Hendrickson. 1990. Human photoreceptor topography. Journal of Comparative Neurology 292, 4 (1990), 497--523.Google ScholarGoogle ScholarCross RefCross Ref
  5. Niranjan Damera-Venkata and Nelson L Chang. 2009. Display supersampling. ACM Transactions on Graphics (TOG) 28, 1 (2009), 9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Piotr Didyk Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2010. Apparent display resolution enhancement for moving images. ACM Transactions on Graphics (TOG) 29, 4 (2010), 113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Thomas Engelhardt, Thorsten-Walther Schmidt, Jan Kautz, and Carsten Dachsbacher. 2014. Low-cost subpixel rendering for diverse displays. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 199--209. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Felix Heide, Douglas Lanman, Dikpal Reddy, Jan Kautz, Kari Pulli, and David Luebke. 2014. Cascaded displays: Spatiotemporal superresolution using offset pixel layers. ACM Transactions on Graphics (TOG) 33, 4 (2014), 60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Benjamin Kading and Jeremy Straub. 2015. Enhancing head and helmet-mounted displays using a virtual pixel technology. In SPIE Defense and Security. International Society for Optics and Photonics, 947011--947011.Google ScholarGoogle Scholar
  10. Michael Kalloniatis and Charles Luu. 2007. Temporal resolution. Webvision: The Organization of the Retina and Visual System (2007).Google ScholarGoogle Scholar
  11. Michiel A Klompenhouwer and Leo Jan Velthoven. 2004. Motion blur reduction for liquid crystal displays: Motion-compensated inverse filtering. In Proceedings of SPIE, Vol. 5308. 690--699.Google ScholarGoogle Scholar
  12. Belen Masia, Gordon Wetzstein, Piotr Didyk, and Diego Gutierrez. 2013. A survey on computational displays: Pushing the boundaries of optics, computation, and perception. Computers 8 Graphics 37, 8 (2013), 1012--1038. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Dean S Messing and Louis J Kerofsky. 2006. Using optimal rendering to visually mask defective subpixels. In Proceedings of SPIE, Vol. 6057. 236--247.Google ScholarGoogle Scholar
  14. Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Trans Graph (Proc SIGGRAPH Asia) 35, 6 (2016), 179. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. John C Platt. 2000. Optimal filtering for patterned displays. IEEE Signal Processing Letters 7, 7 (2000), 179--181. Road to VR. 2017. Understanding pixel density 8 retinal resolution, and why it's important for AR/VR headsets. (2017). https://www.roadtovr.com/Google ScholarGoogle ScholarCross RefCross Ref
  16. Michael Stengel, Martin Eisemann, Stephan Wenger, Benjamin Hell, and Marcus Magnor. 2013. Optimizing apparent display resolution enhancement for arbitrary videos. IEEE Transactions on Image Processing 22, 9 (2013), 3604--3613. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Krzysztof Templin, Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. Apparent resolution enhancement for animations. In Proceedings of the 27th Spring Conference on Computer Graphics. ACM, 57--64. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Real-time Apparent Resolution Enhancement for Head-mounted Displays

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Article Metrics

              • Downloads (Last 12 months)10
              • Downloads (Last 6 weeks)0

              Other Metrics

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!