Abstract
The pervasive computing environment and wide network bandwidth provide users more opportunities to share screen content among multiple devices. In this article, we introduce a remote display system to enable screen sharing among multiple devices with high fidelity and responsive interaction. In the developed system, the frame-level screen content is compressed and transmitted to the client side for screen sharing, and the instant control inputs are simultaneously transmitted to the server side for interaction. Even if the screen responds immediately to the control messages and updates at a high frame rate on the server side, it is difficult to update the screen content with low delay and high frame rate in the client side due to non-negligible time consumption on the whole screen frame compression, transmission, and display buffer updating. To address this critical problem, we propose a layered structure for screen coding and rendering to deliver diverse screen content to the client side with an adaptive frame rate. More specifically, the interaction content with small region screen update is compressed by a blockwise screen codec and rendered at a high frame rate to achieve smooth interaction, while the natural video screen content is compressed by standard video codec and rendered at a regular frame rate for a smooth video display. Experimental results with real applications demonstrate that the proposed system can successfully reduce transmission bandwidth cost and interaction delay during screen sharing. Especially for user interaction in small regions, the proposed system can achieve a higher frame rate than most previous counterparts.
- Apple. 2016. Apple Airplay. Retrieved from http://www.apple.com/appletv/airplay/.Google Scholar
- AnyDesk. 2016a. Homepage. Retrieved from http://anydesk.com/remote-desktop.Google Scholar
- AnyDesk. 2016b. Remote Desktop Software Benchmark. Retrieved from http://anydesk.com/benchmark/anydesk-benchmark.pdf.Google Scholar
- Ricardo A. Baratto, Leonard N. Kim, and Jason Nieh. 2005. THINC: A virtual display architecture for thin-client computing. ACM SIGOPS Operat. Syst. Rev. 39, 5 (2005), 277--290. Google Scholar
Digital Library
- Surendar Chandra, Jacob T. Biehl, John Boreczky, Scott Carter, and Lawrence A. Rowe. 2012. Understanding screen contents for building a high performance, real time screen sharing system. In Proceedings of the 20th ACM International Conference on Multimedia. ACM, New York, NY, 389--398. Google Scholar
Digital Library
- Yu-Chun Chang, Po-Han Tseng, Kuan-Ta Chen, and Chin-Laung Lei. 2011. Understanding the performance of thin-client gaming. In Proceedings of the 2011 IEEE International Workshop Technical Committee on Communications Quality and Reliability (CQR). IEEE, 1--6.Google Scholar
Cross Ref
- Kuan-Ta Chen, Yu-Chun Chang, Hwai-Jung Hsu, De-Yu Chen, Chun-Ying Huang, and Cheng-Hsin Hsu. 2014. On the quality of service of cloud gaming systems. IEEE Trans. Multimed. 16, 2 (2014), 480--495. Google Scholar
Digital Library
- Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the latency of cloud gaming systems. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, New York, NY, 1269--1272. Google Scholar
Digital Library
- Tihao Chiang and Ya-Qin Zhang. 1997. A new rate control scheme using quadratic rate distortion model. IEEE Trans. Circuits Syst. Video Technol. 7, 1 (1997), 246--250. Google Scholar
Digital Library
- Sharon Choy, Bernard Wong, Gwendal Simon, and Catherine Rosenberg. 2012. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games. IEEE, 1--6. Google Scholar
Digital Library
- Bernd Oliver Christiansen and Klaus Erik Schauser. 2002. Fast motion detection for thin client compression. In Proceedings of the 2002 Data Compression Conference. IEEE, 332--341. Google Scholar
Digital Library
- Chromecast. 2016. Homepage. Retrieved from http://www.google.com/chrome/devices/chromecast/.Google Scholar
- Lien Deboosere, Jeroen De Wachter, Pieter Simoens, Filip De Turck, Bart Dhoedt, and Piet Demeester. 2007. Thin client computing solutions in low-and high-motion scenarios. In Proceedings of the 3rd International Conference on Networking and Services, 2007. IEEE, 38--38. Google Scholar
Digital Library
- Wenpeng Ding, Yan Lu, and Feng Wu. 2007. Enable efficient compound image compression in H. 264/AVC intra coding. In Proceedings of the IEEE International Conference on Image Processing (ICIP) 2007, Vol. 2. IEEE, 337--340.Google Scholar
Cross Ref
- DisplayCastSourcecode. 2016. Homepage. Retrieved from https://github.com/displayCast/.Google Scholar
- FFmpeg. 2016. Homepage. Retrieved from http://ffmpeg.org/.Google Scholar
- GamingAnywhere. 2016. GamingAnywhere: An Open Cloud Gaming System. Retrieved from https://github.com/chunying/gaminganywhere.Google Scholar
- Patrick Haffner, Paul G. Howard, Patrice Simard, Yoshua Bengio, Yann Lecun, and others. 1998. High quality document image compression with DjVu. J. Electron. Imag. 7, 3 (1998), 410--425.Google Scholar
Cross Ref
- Mark Handley, Sally Floyd, Jitendra Padhye, and Jörg Widmer. 2002. TCP Friendly Rate Control (TFRC): Protocol Specification. Technical Report. Google Scholar
Digital Library
- Chih-Fan Hsu, Tsung-Han Tsai, Chun-Ying Huang, Cheng-Hsin Hsu, and Kuan-Ta Chen. 2015. Screencast dissected: Performance measurements and design considerations. In Proceedings of the 6th ACM Multimedia Systems Conference. ACM, New York, NY, 177--188. Google Scholar
Digital Library
- Chun-Ying Huang, Cheng-Hsin Hsu, Yu-Chun Chang, and Kuan-Ta Chen. 2013. GamingAnywhere: An open cloud gaming system. In Proceedings of the 4th ACM Multimedia Systems Conference. ACM, New York, NY, 36--47. Google Scholar
Digital Library
- Michael Jarschel, Daniel Schlosser, Sven Scheuring, and Tobias Hofeld. 2011. An evaluation of QoE in cloud gaming based on subjective tests. In Proceedings of the 2011 5th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS). IEEE, 330--335. Google Scholar
Digital Library
- LogMeIn. 2016. Homepage. Retrieved from https://secure.logmein.com/.Google Scholar
- Yan Lu, Shipeng Li, and Huifeng Shen. 2011. Virtualized screen: A third element for cloud-mobile convergence. IEEE Trans. Multimed. 18, 2 (2011), 4--11. Google Scholar
Digital Library
- Dan Miao, Jingjing Fu, Yan Lu, Shipeng Li, and Chang Wen Chen. 2013. Layered screen video coding leveraging hardware video codec. In Proceedings of the 2013 IEEE International Conference on Multimedia and Expo. IEEE, 1--6.Google Scholar
Cross Ref
- Dan Miao, Jingjing Fu, Yan Lu, Shipeng Li, and Chang Wen Chen. 2014. High frame rate screen video coding for screen sharing applications. In Proceedings of the 2014 IEEE International Symposium on Circuits and Systems. IEEE, 2157--2160.Google Scholar
Cross Ref
- Microsoft. 2016a. Remote Desktop Protocol (RDP). Retrieved from http://msdn.microsoft.com/en-us/library/aa383015(v=vs.85).aspx.Google Scholar
- Microsoft. 2016b. Using Performance Monitor. Retrieved from https://technet.microsoft.com/en-us/library/cc749115.aspx.Google Scholar
- NCast. 2016. Homepage. Retrieved from http://www.ncast.com/.Google Scholar
- OnLive. 2016. Homepage. Retrieved from http://www.onlive.com/.Google Scholar
- Zhaotai Pan, Huifeng Shen, Yan Lu, Shipeng Li, and Nenghai Yu. 2013. A low-complexity screen compression scheme for interactive screen sharing. IEEE Trans. Circuits Syst. Video Technol. 23, 6 (2013), 949--960. Google Scholar
Digital Library
- Tristan Richardson, Quentin Stafford-Fraser, Kenneth R. Wood, and Andy Hopper. 1998. Virtual network computing. IEEE Trans. Internet Comput. 2, 1 (1998), 33--38. Google Scholar
Digital Library
- Robert W. Scheifler and Jim Gettys. 1986. The X window system. ACM Trans. Graphics 5, 2 (1986), 79--109. Google Scholar
Digital Library
- Huifeng Shen, Yan Lu, Feng Wu, and Shipeng Li. 2009. A high-performanance remote computing platform. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications, 2009. IEEE, 1--6. Google Scholar
Digital Library
- Pieter Simoens, Paul Praet, Bert Vankeirsbilck, Jeroen De Wachter, Lien Deboosere, Filip De Turck, Bart Dhoedt, and Piet Demeester. 2008. Design and implementation of a hybrid remote display protocol to optimize multimedia experience on thin client devices. In Proceedings of the Australian Telecommunication Networks and Applications Conference, 2008 (ATNAC’08). IEEE, 391--396.Google Scholar
Cross Ref
- Software Systems Laboratory. 2016. THINC. Retrieved from http://systems.cs.columbia.edu/projects/thinc/.Google Scholar
- StreamMyGame. 2016. Homepage. Retrieved from http://streammygame.com/smg/index.php.Google Scholar
- StreamMyGame. 2012. Homepage. Retrieved from http://streammygame.com/smg/index.php.Google Scholar
- Kheng-Joo Tan, Jia-Wei Gong, Bing-Tsung Wu, Dou-Cheng Chang, Hsin-Yi Li, Yi-Mao Hsiao, Yung-Chung Chen, Shi-Wu Lo, Yuan-Sun Chu, and Jiun-In Guo. 2010. A remote thin client system for real time multimedia streaming over VNC. In Proceedings of the 2010 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 992--997.Google Scholar
Cross Ref
- VideoLAN. 2016. x264. Retrieved from http://www.videolan.org/developers/x264.html.Google Scholar
- TeamViewer. 2016. Homepage. Retrieved from http://www.teamviewer.com/.Google Scholar
- TightVNC. 2016. Homepage. Retrieved from http://www.tightvnc.com/.Google Scholar
- Shiqi Wang, Jingjing Fu, Yan Lu, Shipeng Li, and Wen Gao. 2012. Content-aware layered compound video compression. In Proceedings of the 2012 IEEE International Symposium on Circuits and Systems. IEEE, 145--148.Google Scholar
Cross Ref
- Wireshark. 2016. Homepage. Retrieved from https://www.wireshark.org/.Google Scholar
- Wi-Fi Alliance. 2016. Wi-Fi CERTIFIED Miracast. Retrieved from http://www.wi-fi.org/discover-wi-fi/wi-fi-certified-miracast.Google Scholar
- Jiyan Wu, Bo Cheng, Chau Yuen, Yanlei Shang, and Junliang Chen. 2015a. Distortion-aware concurrent multipath transfer for mobile video streaming in heterogeneous wireless networks. IEEE Trans. Mobile Comput. 14, 4 (2015), 688--701.Google Scholar
Cross Ref
- Jiyan Wu, Chau Yuen, Ngai-Man Cheung, Junliang Chen, and Chang Wen Chen. 2015b. Enabling adaptive high-frame-rate video streaming in mobile cloud gaming applications. IEEE Trans. Circuits Syst. Video Technol. 25, 12 (2015), 1988--2001.Google Scholar
Cross Ref
- Tao Zhang, Xun Guo, Yan Lu, Shipeng Li, Siwei Ma, and Debin Zhao. 2013. Arbitrary-sized motion detection in screen video coding. In Image Processing (ICIP), 2013 20th IEEE International Conference on. IEEE, 1943--1947.Google Scholar
Cross Ref
Index Terms
A High-Fidelity and Low-Interaction-Delay Screen Sharing System
Recommendations
High performance many-to-many intranet screen sharing with DisplayCast
DisplayCast is a many to many Intranet screen sharing system. Its screen capture mechanism creates a sequence of pixmap images of the screen updates. Prior systems that used a similar approach were designed to operate over constrained wide-area networks ...
Understanding screen contents for building a high performance, real time screen sharing system
MM '12: Proceedings of the 20th ACM international conference on MultimediaFaithful sharing of screen contents is an important collaboration feature. Prior systems were designed to operate over constrained networks. They performed poorly even without such bottlenecks. To build a high performance screen sharing system, we ...
A Low-Complexity Screen Compression Scheme for Interactive Screen Sharing
Interactive screen sharing requires extremely low latency end-to-end transmission, which in turn requires highly efficient and low-complexity screen compression. In this paper, we present a block-based low-complexity screen compression scheme, in which ...






Comments