Abstract
Image-Based Rendering (IBR) has become widely known by its relatively low requirements for generating new scenes based on a sequence of reference images. This characteristic of IBR shows a remarkable potential impact in rendering complex 3D virtual environments on graphics-constrained devices, such as head-mounted displays, set-top boxes, media streaming devices, and so on. If well exploited, IBR coupled with remote rendering would enable the exploration of complex virtual environments on these devices. However, remote rendering requires the transmission of a large volume of images. In addition, existing solutions consider limited and/or deterministic navigation schemes as a means of decreasing the volume of streamed data. This article proposes the PROgressive PANorama StrEaming protocol (PROPANE) to offer users a smoother virtual navigation experience by prestreaming the imagery data required to generate new views as the user wanders within a 3D environment. PROPANE is based on a very simple yet effective trigonometry model and uses a strafe (lateral movement) technique to minimize the delay between image updates at the client end. This article introduces the concept of key partial panoramas, namely panorama segments that cover movements in any direction by simply strafing from an appropriate key partial panorama and streaming the amount of lost pixels. Therefore, PROPANE can provide a constrained device with sufficient imagery data to cover a future user's viewpoints, thereby minimizing the impact of transmission delay and jitter. PROPANE has been implemented and compared to two baseline remote rendering schemes. The evaluation results show that the proposed technique outperforms the selected and closely related existing schemes by minimizing the response time while not limiting the user to predefined paths as opposed to previous protocols.
- T. E. Akihiro, A. Katayama, H. Tamura, M. Hirose, T. Tanikawa, and M. Saito. 1998. Image-based walk-through system for large-scale scenes. In roceedings of the 4th International Conference on Virtual Systems and Multimedia (VSMM'98). 269--274.Google Scholar
- D. Anguelov, C. Dulong, D. Filip, C. Frueh, S. Lafon, R. Lyon, A. Ogale, L. Vincent, and J. Weaver. 2010. Google street view: Capturing the world at street level. Comput. 43, 6, 32--38. Google Scholar
Digital Library
- K. Bouatouch, G. Point, and G. Thomas. 2005. A client-server approach to image-based rendering on mobile terminals. Rapport de recherche RR-5447, INRIA. http://www.inria.fr/rrrt/rr-5447.htmlGoogle Scholar
- A. Boukerche, R. Jarrar, and R. Pazzi. 2009. A novel interactive streaming protocol for image-based 3d virtual environment navigation. In Proceedings of the IEEE International Conference on Communications (ICC'09). 1--6. Google Scholar
Digital Library
- A. Boukerche and R. W. N. Pazzi. 2006. Remote rendering and streaming of progressive panoramas for mobile devices. In Proceedings of the 14th Annual International Conference on Multimedia (MULTIMEDIA'06). ACM Press, New York, 691--694. Google Scholar
Digital Library
- A. Boukerche, R. W. N. Pazzi, and J. Feng. 2008. An end-to-end virtual environment streaming technique for thin mobile devices over heterogeneous networks. Comput. Comm. 31, 11, 2716--2725. Google Scholar
Digital Library
- A. Boukerche, R. Werner, and N. Pazzi. 2007. A peer-to-peer approach for remote rendering and image streaming in walkthrough applications. In Proceedings of the IEEE International Conference on Communications (ICC'07). 1692--1697.Google Scholar
- D. Bradley, A. Brunton, M. Fiala, and G. Roth. 2005. Image-based navigation in real environments using panoramas. In Proceedings of the IEEE International Workshop on Haptic Audio Visual Environments and their Applications.Google Scholar
- H. Chi, Q. Zhang., J. Jia, and X. Shen. 2007. Efficient search and scheduling in p2p-based media-on-demand streaming service. IEEE J. Selected Areas Comm. 25, 1, 119--130. Google Scholar
Digital Library
- Gaikai. 2013. Gaikai® desktop. http://www.gaikai.com/.Google Scholar
- Z. Jiang, Y. Mao, Q. Jia, N. Jiang, J. Tao, X. Fang, and H. Bao. 2006. Panowalk: A remote image-based rendering system for mobile devices. In Proceedings of the 7th Pacific Rim Conference on Advances in Multimedia Information Processing (PCM'06). Springer, 641--649. Google Scholar
Digital Library
- J. Kilner, J. Starck, A. Hilton, and O. Grau. 2007. Dual-mode deformable models for free-viewpoint video of sports events. In Proceedings of the 6th International Conference on 3D Digital Imaging and Modeling (3DIM'07). IEEE Computer Society, 177--184. Google Scholar
Digital Library
- H. Kimata, D. Ochi, A. Kameda, H. Noto, K. Fukazawa, and A. Kojima. 2012. Mobile and multi-device interactive panorama video distribution system. In Proceedings of the 1st IEEE Global Conference on Consumer Electronics (GCCE'12). 574--578.Google Scholar
- R. Leadbetter. 2013. Secrets of the wii u gamepad. http://www.eurogamer.net/articles/digitalfoundry-secrets-of-the-wii-u-gamepad.Google Scholar
- Y. Lei, Z. Jiang, D. Chen, and H. Bao. 2004. Image-based walkthrough over internet on mobile devices. In Proceedings of the International Workshop on Grid and Cooperative Computing (GCC'04). Lecture Notes in Computer Science, vol. 3252, Springer, 728--735.Google Scholar
Cross Ref
- E. Lengyel. 2002. Mathematics for 3D Game Programming and Computer Graphics. Charles River Media. Google Scholar
Digital Library
- J.-G. Luo, Q. Zhang, Y. Tang, and S.-Q. Yang. 2009. A trace-driven approach to evaluate the scalability of p2p-based video-on-demand service. IEEE Trans. Parallel Distrib. Syst. 20, 1, 59--70. Google Scholar
Digital Library
- H. Maamar, A. Boukerche, and E. Petriu. 2013. Streaming 3d meshes over thin mobile devices. IEEE Wirel. Comm. 20, 3, 136--142.Google Scholar
Cross Ref
- Onlive. 2013. Onlive® desktop. http://desktop.onlive.com/about.Google Scholar
- S. M. Seitz and C. R. Dyer. 1996. View morphing. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). 21--30. Google Scholar
Digital Library
- D. Seo, S. Kim, H. Park, and H. Ko. 2013. Real-time panoramic video streaming system with overlaid interface concept for social media. Multimedia Syst. 19, 1--13.Google Scholar
- R. Shea, J. Liu, E.-H. Ngai, and Y. Cui. 2013. Cloud gaming: Architecture and performance. IEEE Netw. 27, 4, 16--21.Google Scholar
Cross Ref
- H. Shum, S. Chan, and S. Kang. 2012. Image-Based Rendering. Springer. Google Scholar
Digital Library
- M. Tanimoto. 2009. Overview of ftv (free-viewpoint television). In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'09). 1552--1553. Google Scholar
Digital Library
- L. Yang and R. Crawfis. 2002. Rail-track viewer: An image-based virtual walkthrough system. In Proceedings of the Workshop on Virtual Environments (EGVE'02). Eurographics Association. Google Scholar
Digital Library
- Q. Zhao, L. Wan, W. Feng, J. Zhang, and T.-T. Wong. 2013. Cube2video: Navigate between cubic panoramas in real-time. IEEE Trans. Multimedia 15, 8, 1745--1754. Google Scholar
Digital Library
Index Terms
PROPANE: A Progressive Panorama Streaming Protocol to Support Interactive 3D Virtual Environment Exploration on Graphics-Constrained Devices
Recommendations
Free-viewpoint video rendering for mobile devices
MIRAGE '13: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and ApplicationsFree-viewpoint video renderers (FVVR) allow a user to view captured video footage from any position and direction. Despite the obvious appeal of such systems, they have yet to make a major impact on digital entertainment. Current FVVR implementations ...
Accelerating voxel-based terrain rendering with keyframe-free image-based rendering
VG'01: Proceedings of the 2001 Eurographics conference on Volume GraphicsWe propose a voxel-based terrain rendering method which incorporates a novel keyframe-free image-based rendering algorithm and a new heuristic ray coherence raycasting algorithm. The current image is generated by warping the previous image with a ...
Image-based rendering of diffuse, specular and glossy surfaces from a single image
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniquesIn this paper, we present a new method to recover an approximation of the bidirectional reflectance distribution function (BRDF) of the surfaces present in a real scene. This is done from a single photograph and a 3D geometric model of the scene. The ...






Comments