skip to main content
research-article

A High-Fidelity and Low-Interaction-Delay Screen Sharing System

Authors Info & Claims
Published:20 May 2016Publication History
Skip Abstract Section

Abstract

The pervasive computing environment and wide network bandwidth provide users more opportunities to share screen content among multiple devices. In this article, we introduce a remote display system to enable screen sharing among multiple devices with high fidelity and responsive interaction. In the developed system, the frame-level screen content is compressed and transmitted to the client side for screen sharing, and the instant control inputs are simultaneously transmitted to the server side for interaction. Even if the screen responds immediately to the control messages and updates at a high frame rate on the server side, it is difficult to update the screen content with low delay and high frame rate in the client side due to non-negligible time consumption on the whole screen frame compression, transmission, and display buffer updating. To address this critical problem, we propose a layered structure for screen coding and rendering to deliver diverse screen content to the client side with an adaptive frame rate. More specifically, the interaction content with small region screen update is compressed by a blockwise screen codec and rendered at a high frame rate to achieve smooth interaction, while the natural video screen content is compressed by standard video codec and rendered at a regular frame rate for a smooth video display. Experimental results with real applications demonstrate that the proposed system can successfully reduce transmission bandwidth cost and interaction delay during screen sharing. Especially for user interaction in small regions, the proposed system can achieve a higher frame rate than most previous counterparts.

References

  1. Apple. 2016. Apple Airplay. Retrieved from http://www.apple.com/appletv/airplay/.Google ScholarGoogle Scholar
  2. AnyDesk. 2016a. Homepage. Retrieved from http://anydesk.com/remote-desktop.Google ScholarGoogle Scholar
  3. AnyDesk. 2016b. Remote Desktop Software Benchmark. Retrieved from http://anydesk.com/benchmark/anydesk-benchmark.pdf.Google ScholarGoogle Scholar
  4. Ricardo A. Baratto, Leonard N. Kim, and Jason Nieh. 2005. THINC: A virtual display architecture for thin-client computing. ACM SIGOPS Operat. Syst. Rev. 39, 5 (2005), 277--290. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Surendar Chandra, Jacob T. Biehl, John Boreczky, Scott Carter, and Lawrence A. Rowe. 2012. Understanding screen contents for building a high performance, real time screen sharing system. In Proceedings of the 20th ACM International Conference on Multimedia. ACM, New York, NY, 389--398. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yu-Chun Chang, Po-Han Tseng, Kuan-Ta Chen, and Chin-Laung Lei. 2011. Understanding the performance of thin-client gaming. In Proceedings of the 2011 IEEE International Workshop Technical Committee on Communications Quality and Reliability (CQR). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  7. Kuan-Ta Chen, Yu-Chun Chang, Hwai-Jung Hsu, De-Yu Chen, Chun-Ying Huang, and Cheng-Hsin Hsu. 2014. On the quality of service of cloud gaming systems. IEEE Trans. Multimed. 16, 2 (2014), 480--495. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the latency of cloud gaming systems. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, New York, NY, 1269--1272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Tihao Chiang and Ya-Qin Zhang. 1997. A new rate control scheme using quadratic rate distortion model. IEEE Trans. Circuits Syst. Video Technol. 7, 1 (1997), 246--250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Sharon Choy, Bernard Wong, Gwendal Simon, and Catherine Rosenberg. 2012. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games. IEEE, 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Bernd Oliver Christiansen and Klaus Erik Schauser. 2002. Fast motion detection for thin client compression. In Proceedings of the 2002 Data Compression Conference. IEEE, 332--341. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chromecast. 2016. Homepage. Retrieved from http://www.google.com/chrome/devices/chromecast/.Google ScholarGoogle Scholar
  13. Lien Deboosere, Jeroen De Wachter, Pieter Simoens, Filip De Turck, Bart Dhoedt, and Piet Demeester. 2007. Thin client computing solutions in low-and high-motion scenarios. In Proceedings of the 3rd International Conference on Networking and Services, 2007. IEEE, 38--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Wenpeng Ding, Yan Lu, and Feng Wu. 2007. Enable efficient compound image compression in H. 264/AVC intra coding. In Proceedings of the IEEE International Conference on Image Processing (ICIP) 2007, Vol. 2. IEEE, 337--340.Google ScholarGoogle ScholarCross RefCross Ref
  15. DisplayCastSourcecode. 2016. Homepage. Retrieved from https://github.com/displayCast/.Google ScholarGoogle Scholar
  16. FFmpeg. 2016. Homepage. Retrieved from http://ffmpeg.org/.Google ScholarGoogle Scholar
  17. GamingAnywhere. 2016. GamingAnywhere: An Open Cloud Gaming System. Retrieved from https://github.com/chunying/gaminganywhere.Google ScholarGoogle Scholar
  18. Patrick Haffner, Paul G. Howard, Patrice Simard, Yoshua Bengio, Yann Lecun, and others. 1998. High quality document image compression with DjVu. J. Electron. Imag. 7, 3 (1998), 410--425.Google ScholarGoogle ScholarCross RefCross Ref
  19. Mark Handley, Sally Floyd, Jitendra Padhye, and Jörg Widmer. 2002. TCP Friendly Rate Control (TFRC): Protocol Specification. Technical Report. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Chih-Fan Hsu, Tsung-Han Tsai, Chun-Ying Huang, Cheng-Hsin Hsu, and Kuan-Ta Chen. 2015. Screencast dissected: Performance measurements and design considerations. In Proceedings of the 6th ACM Multimedia Systems Conference. ACM, New York, NY, 177--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Chun-Ying Huang, Cheng-Hsin Hsu, Yu-Chun Chang, and Kuan-Ta Chen. 2013. GamingAnywhere: An open cloud gaming system. In Proceedings of the 4th ACM Multimedia Systems Conference. ACM, New York, NY, 36--47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Michael Jarschel, Daniel Schlosser, Sven Scheuring, and Tobias Hofeld. 2011. An evaluation of QoE in cloud gaming based on subjective tests. In Proceedings of the 2011 5th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS). IEEE, 330--335. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. LogMeIn. 2016. Homepage. Retrieved from https://secure.logmein.com/.Google ScholarGoogle Scholar
  24. Yan Lu, Shipeng Li, and Huifeng Shen. 2011. Virtualized screen: A third element for cloud-mobile convergence. IEEE Trans. Multimed. 18, 2 (2011), 4--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Dan Miao, Jingjing Fu, Yan Lu, Shipeng Li, and Chang Wen Chen. 2013. Layered screen video coding leveraging hardware video codec. In Proceedings of the 2013 IEEE International Conference on Multimedia and Expo. IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  26. Dan Miao, Jingjing Fu, Yan Lu, Shipeng Li, and Chang Wen Chen. 2014. High frame rate screen video coding for screen sharing applications. In Proceedings of the 2014 IEEE International Symposium on Circuits and Systems. IEEE, 2157--2160.Google ScholarGoogle ScholarCross RefCross Ref
  27. Microsoft. 2016a. Remote Desktop Protocol (RDP). Retrieved from http://msdn.microsoft.com/en-us/library/aa383015(v=vs.85).aspx.Google ScholarGoogle Scholar
  28. Microsoft. 2016b. Using Performance Monitor. Retrieved from https://technet.microsoft.com/en-us/library/cc749115.aspx.Google ScholarGoogle Scholar
  29. NCast. 2016. Homepage. Retrieved from http://www.ncast.com/.Google ScholarGoogle Scholar
  30. OnLive. 2016. Homepage. Retrieved from http://www.onlive.com/.Google ScholarGoogle Scholar
  31. Zhaotai Pan, Huifeng Shen, Yan Lu, Shipeng Li, and Nenghai Yu. 2013. A low-complexity screen compression scheme for interactive screen sharing. IEEE Trans. Circuits Syst. Video Technol. 23, 6 (2013), 949--960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Tristan Richardson, Quentin Stafford-Fraser, Kenneth R. Wood, and Andy Hopper. 1998. Virtual network computing. IEEE Trans. Internet Comput. 2, 1 (1998), 33--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Robert W. Scheifler and Jim Gettys. 1986. The X window system. ACM Trans. Graphics 5, 2 (1986), 79--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Huifeng Shen, Yan Lu, Feng Wu, and Shipeng Li. 2009. A high-performanance remote computing platform. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications, 2009. IEEE, 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Pieter Simoens, Paul Praet, Bert Vankeirsbilck, Jeroen De Wachter, Lien Deboosere, Filip De Turck, Bart Dhoedt, and Piet Demeester. 2008. Design and implementation of a hybrid remote display protocol to optimize multimedia experience on thin client devices. In Proceedings of the Australian Telecommunication Networks and Applications Conference, 2008 (ATNAC’08). IEEE, 391--396.Google ScholarGoogle ScholarCross RefCross Ref
  36. Software Systems Laboratory. 2016. THINC. Retrieved from http://systems.cs.columbia.edu/projects/thinc/.Google ScholarGoogle Scholar
  37. StreamMyGame. 2016. Homepage. Retrieved from http://streammygame.com/smg/index.php.Google ScholarGoogle Scholar
  38. StreamMyGame. 2012. Homepage. Retrieved from http://streammygame.com/smg/index.php.Google ScholarGoogle Scholar
  39. Kheng-Joo Tan, Jia-Wei Gong, Bing-Tsung Wu, Dou-Cheng Chang, Hsin-Yi Li, Yi-Mao Hsiao, Yung-Chung Chen, Shi-Wu Lo, Yuan-Sun Chu, and Jiun-In Guo. 2010. A remote thin client system for real time multimedia streaming over VNC. In Proceedings of the 2010 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 992--997.Google ScholarGoogle ScholarCross RefCross Ref
  40. VideoLAN. 2016. x264. Retrieved from http://www.videolan.org/developers/x264.html.Google ScholarGoogle Scholar
  41. TeamViewer. 2016. Homepage. Retrieved from http://www.teamviewer.com/.Google ScholarGoogle Scholar
  42. TightVNC. 2016. Homepage. Retrieved from http://www.tightvnc.com/.Google ScholarGoogle Scholar
  43. Shiqi Wang, Jingjing Fu, Yan Lu, Shipeng Li, and Wen Gao. 2012. Content-aware layered compound video compression. In Proceedings of the 2012 IEEE International Symposium on Circuits and Systems. IEEE, 145--148.Google ScholarGoogle ScholarCross RefCross Ref
  44. Wireshark. 2016. Homepage. Retrieved from https://www.wireshark.org/.Google ScholarGoogle Scholar
  45. Wi-Fi Alliance. 2016. Wi-Fi CERTIFIED Miracast. Retrieved from http://www.wi-fi.org/discover-wi-fi/wi-fi-certified-miracast.Google ScholarGoogle Scholar
  46. Jiyan Wu, Bo Cheng, Chau Yuen, Yanlei Shang, and Junliang Chen. 2015a. Distortion-aware concurrent multipath transfer for mobile video streaming in heterogeneous wireless networks. IEEE Trans. Mobile Comput. 14, 4 (2015), 688--701.Google ScholarGoogle ScholarCross RefCross Ref
  47. Jiyan Wu, Chau Yuen, Ngai-Man Cheung, Junliang Chen, and Chang Wen Chen. 2015b. Enabling adaptive high-frame-rate video streaming in mobile cloud gaming applications. IEEE Trans. Circuits Syst. Video Technol. 25, 12 (2015), 1988--2001.Google ScholarGoogle ScholarCross RefCross Ref
  48. Tao Zhang, Xun Guo, Yan Lu, Shipeng Li, Siwei Ma, and Debin Zhao. 2013. Arbitrary-sized motion detection in screen video coding. In Image Processing (ICIP), 2013 20th IEEE International Conference on. IEEE, 1943--1947.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. A High-Fidelity and Low-Interaction-Delay Screen Sharing System

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!