skip to main content
research-article

Cloud Baking: Collaborative Scene Illumination for Dynamic Web3D Scenes

Published:15 June 2018Publication History
Skip Abstract Section

Abstract

We propose Cloud Baking, a collaborative rendering architecture for dynamic Web3D scenes. In our architecture, the cloud renderer renders the scene with the global illumination (GI) information in a GI map; the web-based client renderer renders the scene with ambient lighting only and blends it with the GI map received from the cloud for the final scene. This approach allows the users to interact with the web scene and change the scene dynamically through the web interface end, yet move the computationally heavy tasks of global illumination computation to the cloud. A challenge we face is the interaction delay that causes the frames rendered on the cloud and the client to go out of sync. We propose to use 3D warping and a hole-filling algorithm designed for GI map to predict the late GI map. We show both quantitatively and visually the quality of the GI map produced using our method. Our prediction algorithm allows us to further reduce the frequency at which the GI map is computed and sent from the server, reducing both computational needs and bandwidth usage.

References

  1. P. Bao and D. Gourlay. 2004. Remote walkthrough over mobile networks using 3-D image warping and streaming. IEE Proceedings-Vision, Image and Signal Processing 151, 4 (2004), 329--336.Google ScholarGoogle ScholarCross RefCross Ref
  2. Ken W. Brodlie, David A. Duce, Julian R. Gallop, J. P. R. B. Walton, and J. D. Wood. 2004. Distributed and collaborative visualization. Computer Graphics Forum 23, 2 (2004), 223--251.Google ScholarGoogle ScholarCross RefCross Ref
  3. Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the latency of cloud gaming systems. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, Scottsdale, AZ, 1269--1272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Luca Chittaro and Roberto Ranon. 2007. Web3D technologies in learning, education and training: Motivations, issues, opportunities. Computers 8 Education 49, 1 (2007), 3--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Sharon Choy, Bernard Wong, Gwendal Simon, and Catherine Rosenberg. 2012. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games. IEEE Press, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Mark Claypool and Kajal Claypool. 2006. Latency and player actions in online games. Commun. ACM 49, 11 (2006), 40--45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cyril Crassin, David Luebke, Michael Mara, Morgan McGuire, Brent Oster, Peter Shirley, Peter-Pike Sloan, and Chris Wyman. 2015. CloudLight: A system for amortizing indirect lighting in real-time rendering. Journal of Computer Graphics Techniques 4, 4 (2015), 1--27.Google ScholarGoogle Scholar
  8. Cyril Crassin, Fabrice Neyret, Miguel Sainz, Simon Green, and Elmar Eisemann. 2011. Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum 30, 7 (2011), 1921--1930.Google ScholarGoogle ScholarCross RefCross Ref
  9. Carsten Dachsbacher and Marc Stamminger. 2005. Reflective shadow maps. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games. ACM, 203--231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Jos Dirksen. 2013. Learning Three.js: the JavaScript 3D Library for WebGL. Packt Publishing Ltd.Google ScholarGoogle Scholar
  11. Florent Duguet and George Drettakis. 2004. Flexible point-based rendering on mobile devices. IEEE Computer Graphics and Applications 24, 4 (2004), 57--63. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Christoph Fehn. 2004. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. Proceedings of SPIE 5291, 2 (2004), 93--104.Google ScholarGoogle ScholarCross RefCross Ref
  13. Keith Gladstien. 2013. Flash Game Development In a Social, Mobile and 3D World. Cengage Learning. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Liwei Hsu. 2012. Web 3D simulation-based application in tourism education: A case study with Second Life. Journal of Hospitality, Leisure, Sport 8 Tourism Education 11, 2 (2012), 113--124.Google ScholarGoogle ScholarCross RefCross Ref
  15. Anton Kaplanyan and Carsten Dachsbacher. 2010. Cascaded light propagation volumes for real-time indirect illumination. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 99--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. David Koller, Michael Turitzin, Marc Levoy, Marco Tarini, Giuseppe Croccia, Paolo Cignoni, and Roberto Scopigno. 2004. Protected interactive 3D graphics via remote rendering. ACM Transactions on Graphics 23, 3 (2004), 695--703. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 151--165. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Javier Lluch, Rafael Gaitán, Emilio Camahort, and Roberto Vivó. 2005. Interactive three-dimensional rendering on mobile computer devices. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. ACM, 254--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proceedings of the 1997 Symposium on Interactive 3D Graphics. ACM, 7--ff. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Leonard McMillan Jr. 1997. An Image-Based Approach to Three-Dimensional Computer Graphics. Ph.D. Dissertation. University of North Carolina at Chapel Hill. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Yuji Mori, Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto. 2009. View generation with 3D warping using depth information for FTV. Signal Processing: Image Communication 24, 1 (2009), 65--72. Special issue on advances in three-dimensional television and video. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Fiona Fui-Hoon Nah. 2004. A study on tolerable waiting time: How long are web users willing to wait? Behaviour 8 Information Technology 23, 3 (2004), 153--163.Google ScholarGoogle Scholar
  23. Brian Paul, Sean Ahern, Wes Bethel, Eric Brugger, Rich Cook, Jamison Daniel, Ken Lewis, Jens Owen, and Dale Southard. 2008. Chromium renderserver: Scalable and open remote rendering infrastructure. IEEE Transactions on Visualization and Computer Graphics 14, 3 (2008), 627--639. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered depth images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 231--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Shu Shi, Klara Nahrstedt, and Roy Campbell. 2012. A real-time remote rendering system for interactive mobile graphics. ACM Transactions on Multimedia Computing, Communications, and Applications 8, 3s (2012), 46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Aljoscha Smolic, Karsten Muller, Kristina Dix, Philipp Merkle, Peter Kauff, and Thomas Wiegand. 2008. Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. In 2008 15th IEEE International Conference on Image Processing. 2448--2451.Google ScholarGoogle ScholarCross RefCross Ref
  27. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Laixiang Wen, Jinyuan Jia, and Shuang Liang. 2014. LPM: Lightweight progressive meshes towards smooth transmission of Web3D media over internet. In Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 95--103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Laixiang Wen, Ning Xie, and Jinyuan Jia. 2016. Client-driven strategy of large-scale scene streaming. In Proceedings of the International Conference on Multimedia Modeling. Springer, 93--103.Google ScholarGoogle ScholarCross RefCross Ref
  30. Minhui Zhu, Sebastien Mondet, Géraldine Morin, Wei Tsang Ooi, and Wei Cheng. 2011. Towards peer-assisted rendering in networked virtual environments. In Proceedings of the 19th International Conference on Multimedia, November 28--December 1. Scottsdale, AZ, 183--192. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Thomas Zinner, Thomas Zinner, Osama Abboud, and Tobias Hossfeld. 2010. Impact of frame rate and resolution on objective QoE metrics. In The 2nd International Workshop on Quality of Multimedia Experience (QoMEX’10). 29--34.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Cloud Baking: Collaborative Scene Illumination for Dynamic Web3D Scenes

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 3s
          Special Section on Delay-Sensitive Video Computing in the Cloud and Special Section on Extended MMSys-NOSSDAV Best Papers
          June 2018
          317 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3233173
          Issue’s Table of Contents

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 15 June 2018
          • Accepted: 1 April 2018
          • Revised: 1 February 2018
          • Received: 1 August 2017
          Published in tomm Volume 14, Issue 3s

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!