skip to main content
research-article

PROPANE: A Progressive Panorama Streaming Protocol to Support Interactive 3D Virtual Environment Exploration on Graphics-Constrained Devices

Published:04 September 2014Publication History
Skip Abstract Section

Abstract

Image-Based Rendering (IBR) has become widely known by its relatively low requirements for generating new scenes based on a sequence of reference images. This characteristic of IBR shows a remarkable potential impact in rendering complex 3D virtual environments on graphics-constrained devices, such as head-mounted displays, set-top boxes, media streaming devices, and so on. If well exploited, IBR coupled with remote rendering would enable the exploration of complex virtual environments on these devices. However, remote rendering requires the transmission of a large volume of images. In addition, existing solutions consider limited and/or deterministic navigation schemes as a means of decreasing the volume of streamed data. This article proposes the PROgressive PANorama StrEaming protocol (PROPANE) to offer users a smoother virtual navigation experience by prestreaming the imagery data required to generate new views as the user wanders within a 3D environment. PROPANE is based on a very simple yet effective trigonometry model and uses a strafe (lateral movement) technique to minimize the delay between image updates at the client end. This article introduces the concept of key partial panoramas, namely panorama segments that cover movements in any direction by simply strafing from an appropriate key partial panorama and streaming the amount of lost pixels. Therefore, PROPANE can provide a constrained device with sufficient imagery data to cover a future user's viewpoints, thereby minimizing the impact of transmission delay and jitter. PROPANE has been implemented and compared to two baseline remote rendering schemes. The evaluation results show that the proposed technique outperforms the selected and closely related existing schemes by minimizing the response time while not limiting the user to predefined paths as opposed to previous protocols.

References

  1. T. E. Akihiro, A. Katayama, H. Tamura, M. Hirose, T. Tanikawa, and M. Saito. 1998. Image-based walk-through system for large-scale scenes. In roceedings of the 4th International Conference on Virtual Systems and Multimedia (VSMM'98). 269--274.Google ScholarGoogle Scholar
  2. D. Anguelov, C. Dulong, D. Filip, C. Frueh, S. Lafon, R. Lyon, A. Ogale, L. Vincent, and J. Weaver. 2010. Google street view: Capturing the world at street level. Comput. 43, 6, 32--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. K. Bouatouch, G. Point, and G. Thomas. 2005. A client-server approach to image-based rendering on mobile terminals. Rapport de recherche RR-5447, INRIA. http://www.inria.fr/rrrt/rr-5447.htmlGoogle ScholarGoogle Scholar
  4. A. Boukerche, R. Jarrar, and R. Pazzi. 2009. A novel interactive streaming protocol for image-based 3d virtual environment navigation. In Proceedings of the IEEE International Conference on Communications (ICC'09). 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. Boukerche and R. W. N. Pazzi. 2006. Remote rendering and streaming of progressive panoramas for mobile devices. In Proceedings of the 14th Annual International Conference on Multimedia (MULTIMEDIA'06). ACM Press, New York, 691--694. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. A. Boukerche, R. W. N. Pazzi, and J. Feng. 2008. An end-to-end virtual environment streaming technique for thin mobile devices over heterogeneous networks. Comput. Comm. 31, 11, 2716--2725. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. A. Boukerche, R. Werner, and N. Pazzi. 2007. A peer-to-peer approach for remote rendering and image streaming in walkthrough applications. In Proceedings of the IEEE International Conference on Communications (ICC'07). 1692--1697.Google ScholarGoogle Scholar
  8. D. Bradley, A. Brunton, M. Fiala, and G. Roth. 2005. Image-based navigation in real environments using panoramas. In Proceedings of the IEEE International Workshop on Haptic Audio Visual Environments and their Applications.Google ScholarGoogle Scholar
  9. H. Chi, Q. Zhang., J. Jia, and X. Shen. 2007. Efficient search and scheduling in p2p-based media-on-demand streaming service. IEEE J. Selected Areas Comm. 25, 1, 119--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gaikai. 2013. Gaikai® desktop. http://www.gaikai.com/.Google ScholarGoogle Scholar
  11. Z. Jiang, Y. Mao, Q. Jia, N. Jiang, J. Tao, X. Fang, and H. Bao. 2006. Panowalk: A remote image-based rendering system for mobile devices. In Proceedings of the 7th Pacific Rim Conference on Advances in Multimedia Information Processing (PCM'06). Springer, 641--649. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Kilner, J. Starck, A. Hilton, and O. Grau. 2007. Dual-mode deformable models for free-viewpoint video of sports events. In Proceedings of the 6th International Conference on 3D Digital Imaging and Modeling (3DIM'07). IEEE Computer Society, 177--184. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. H. Kimata, D. Ochi, A. Kameda, H. Noto, K. Fukazawa, and A. Kojima. 2012. Mobile and multi-device interactive panorama video distribution system. In Proceedings of the 1st IEEE Global Conference on Consumer Electronics (GCCE'12). 574--578.Google ScholarGoogle Scholar
  14. R. Leadbetter. 2013. Secrets of the wii u gamepad. http://www.eurogamer.net/articles/digitalfoundry-secrets-of-the-wii-u-gamepad.Google ScholarGoogle Scholar
  15. Y. Lei, Z. Jiang, D. Chen, and H. Bao. 2004. Image-based walkthrough over internet on mobile devices. In Proceedings of the International Workshop on Grid and Cooperative Computing (GCC'04). Lecture Notes in Computer Science, vol. 3252, Springer, 728--735.Google ScholarGoogle ScholarCross RefCross Ref
  16. E. Lengyel. 2002. Mathematics for 3D Game Programming and Computer Graphics. Charles River Media. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J.-G. Luo, Q. Zhang, Y. Tang, and S.-Q. Yang. 2009. A trace-driven approach to evaluate the scalability of p2p-based video-on-demand service. IEEE Trans. Parallel Distrib. Syst. 20, 1, 59--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. H. Maamar, A. Boukerche, and E. Petriu. 2013. Streaming 3d meshes over thin mobile devices. IEEE Wirel. Comm. 20, 3, 136--142.Google ScholarGoogle ScholarCross RefCross Ref
  19. Onlive. 2013. Onlive® desktop. http://desktop.onlive.com/about.Google ScholarGoogle Scholar
  20. S. M. Seitz and C. R. Dyer. 1996. View morphing. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). 21--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. Seo, S. Kim, H. Park, and H. Ko. 2013. Real-time panoramic video streaming system with overlaid interface concept for social media. Multimedia Syst. 19, 1--13.Google ScholarGoogle Scholar
  22. R. Shea, J. Liu, E.-H. Ngai, and Y. Cui. 2013. Cloud gaming: Architecture and performance. IEEE Netw. 27, 4, 16--21.Google ScholarGoogle ScholarCross RefCross Ref
  23. H. Shum, S. Chan, and S. Kang. 2012. Image-Based Rendering. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Tanimoto. 2009. Overview of ftv (free-viewpoint television). In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'09). 1552--1553. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. L. Yang and R. Crawfis. 2002. Rail-track viewer: An image-based virtual walkthrough system. In Proceedings of the Workshop on Virtual Environments (EGVE'02). Eurographics Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Q. Zhao, L. Wan, W. Feng, J. Zhang, and T.-T. Wong. 2013. Cube2video: Navigate between cubic panoramas in real-time. IEEE Trans. Multimedia 15, 8, 1745--1754. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. PROPANE: A Progressive Panorama Streaming Protocol to Support Interactive 3D Virtual Environment Exploration on Graphics-Constrained Devices

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!