skip to main content
research-article

Subtitle Region Selection of S3D Images in Consideration of Visual Discomfort and Viewing Habit

Published:20 August 2019Publication History
Skip Abstract Section

Abstract

Subtitles, serving as a linguistic approximation of the visual content, are an essential element in stereoscopic advertisement and the film industry. Due to the vergence accommodation conflict, the stereoscopic 3D (S3D) subtitle inevitably causes visual discomfort. To meet the viewing experience, the subtitle region should be carefully arranged. Unfortunately, very few works have been dedicated to this area. In this article, we propose a method for S3D subtitle region selection in consideration of visual discomfort and viewing habit. First, we divide the disparity map into multiple depth layers according to the disparity value. The preferential processed depth layer is determined by considering the disparity value of the foremost object. Second, the optimal region and coarse disparity value for S3D subtitle insertion are chosen by convolving the selective depth layer with the mean filter. Specifically, the viewing habit is considered during the region selection. Finally, after region selection, the disparity value of the subtitle is further modified by using the just noticeable depth difference (JNDD) model. Given that there is no public database reported for the evaluation of S3D subtitle insertion, we collect 120 S3D images as the test platform. Both objective and subjective experiments are conducted to evaluate the comfort degree of the inserted subtitle. Experimental results demonstrate that the proposed method can obtain promising performance in improving the viewing experience of the inserted subtitle.

References

  1. Kurt Akeley, Simon J. Watt, Ahna Reza Girshick, and Martin S. Banks. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graphics 23, 3 (2004), 804--813. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Rec. ITU-R BT-500.11. 2002. Methodology for the subjective assessment of the quality of television pictures. (2002).Google ScholarGoogle Scholar
  3. Rec. ITU-R BT.1438. 2000. Methodology for the subjective assessment of the quality of television pictures. (2000).Google ScholarGoogle Scholar
  4. Zaiqing Chen, Junsheng Shi, Xiaoqiao Huang, Lijun Yun, and Yonghan Tai. 2014. Visual comfort modeling for disparity in 3D contents based on Weber--Fechner’s Law. J. Disp. Technol. 10, 12 (2014), 1001--1009.Google ScholarGoogle ScholarCross RefCross Ref
  5. Chung-Hua Chu. 2015. Visual comfort for stereoscopic 3D by using motion sensors on 3D mobile devices. ACM Trans. Multimedia Comput. Commun. Appl. 12, 1s (2015), 14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. D. Varuna SX De Silva, Erhan Ekmekcioglu, Warnakulasuriya Anil Chandana Fernando, and Stewart T. Worrall. 2011. Display dependent preprocessing of depth maps based on just noticeable depth difference modeling. IEEE J. Sel. Top. Signal Process. 5, 2 (2011), 335--351.Google ScholarGoogle ScholarCross RefCross Ref
  7. Song-Pei Du, Belen Masia, Shi-Min Hu, and Diego Gutierrez. 2013. A metric of visual comfort for stereoscopic motion. ACM Trans. Graph. 32, 6 (2013), 222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Yuming Fang, Junle Wang, Manish Narwaria, Patrick Le Callet, and Weisi Lin. 2014. Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23, 6 (2014), 2625--2636. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Yuming Fang, Chi Zhang, Jing Li, Jianjun Lei, Matthieu Perreira Da Silva, and Patrick Le Callet. 2017. Visual attention modeling for stereoscopic video: A benchmark and computational model. IEEE Trans. Image Process. 26, 10 (2017), 4684--4696.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Lutz Goldmann, Francesca De Simone, and Touradj Ebrahimi. 2010. Impact of acquisition distortion on the quality of stereoscopic images. In Proceedings of the International Workshop on Video Processing and Quality Metrics for Consumer Electronics.Google ScholarGoogle Scholar
  11. Nicolas S. Holliman, Neil A. Dodgson, Gregg E. Favalora, and Lachlan Pockett. 2011. Three-dimensional displays: A review and applications analysis. IEEE Trans. Broadcast. 57, 2 (2011), 362--371.Google ScholarGoogle ScholarCross RefCross Ref
  12. Qiuping Jiang, Feng Shao, Gangyi Jiang, Mei Yu, and Zongju Peng. 2017. Visual comfort assessment for stereoscopic images based on sparse coding with multi-scale dictionaries. Neurocomput. 252 (2017), 77--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Qiuping Jiang, Feng Shao, Weisi Lin, and Gangyi Jiang. 2016. On predicting visual comfort of stereoscopic images: A learning to rank based approach. IEEE Signal Process. Lett. 23, 2 (2016), 302--306.Google ScholarGoogle ScholarCross RefCross Ref
  14. Cheolkon Jung, Lihui Cao, Hongmin Liu, and Joongkyu Kim. 2015. Visual comfort enhancement in stereoscopic 3D images using saliency-adaptive nonlinear disparity mapping. Disp. 40 (2015), 17--23.Google ScholarGoogle ScholarCross RefCross Ref
  15. Seung-Won Jung and Sung-Jea Ko. 2012. Depth sensation enhancement using the just noticeable depth difference. IEEE Trans. Image Process. 21, 8 (2012), 3624--3637. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yong Ju Jung, Hosik Sohn, Seong-Il Lee, Hyun Wook Park, and Yong Man Ro. 2013. Predicting visual discomfort of stereoscopic images using human attention model. IEEE Trans. Circuits Syst. Video Technol. 23, 12 (2013), 2077--2082. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Bochao Kan, Yan Zhao, and Shigang Wang. 2018. Objective visual comfort evaluation method based on disparity information and motion for stereoscopic video. Opt. Express 26, 9 (2018), 11418--11437.Google ScholarGoogle ScholarCross RefCross Ref
  18. Taewan Kim, Sanghoon Lee, and Alan Conrad Bovik. 2015. Transfer function model of physiological mechanisms underlying temporal visual discomfort experienced when viewing stereoscopic 3D images. IEEE Trans. Image Process. 24, 11 (2015), 4335--4347.Google ScholarGoogle ScholarCross RefCross Ref
  19. M. Lambooij, M.J. Murdoch, Wijnand A. IJsselsteijn, and Ingrid Heynderickx. 2013. The impact of video characteristics and subtitles on visual comfort of 3D TV. Disp. 34, 1 (2013), 8--16.Google ScholarGoogle ScholarCross RefCross Ref
  20. Marc T.M. Lambooij, W.A. IJsselsteijn, and Ingrid Heynderickx. 2007. Visual discomfort in stereoscopic dsplays: A review. Proceedings of SPIE, 2007 vol. 6490.Google ScholarGoogle Scholar
  21. Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear disparity mapping for stereoscopic 3D. In ACM Trans. Graph. 29, 4 (2010), 75:1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jianjun Lei, Cuicui Zhang, Yuming Fang, Zhouye Gu, Nam Ling, and Chunping Hou. 2015. Depth sensation enhancement for multiple virtual view rendering. IEEE Trans. Multimedia 17, 4 (2015), 457--469.Google ScholarGoogle ScholarCross RefCross Ref
  23. Xingang Liu, Kai Kang, and Yinbo Liu. 2016. Stereoscopic image quality assessment based on depth and texture information. IEEE Syst. J. 2016.Google ScholarGoogle Scholar
  24. H. Oh and S. Lee. 2016. Visual presence: Viewing geometry visual information of UHD S3D entertainment. IEEE Trans. Image Process. 25, 7 (2016), 3358--3371.Google ScholarGoogle ScholarCross RefCross Ref
  25. Jincheol Park, Sanghoon Lee, and Alan Conrad Bovik. 2014. 3D visual discomfort prediction: Vergence, foveation, and the physiological optics of accommodation. IEEE J. Sel. Top. Signal Process. 3, 8 (2014), 415--427.Google ScholarGoogle ScholarCross RefCross Ref
  26. Jincheol Park, Heeseok Oh, Sanghoon Lee, and Alan Conrad Bovik. 2015. 3D visual discomfort predictor: Analysis of disparity and neural activity statistics. IEEE Trans. Image Process. 24, 3 (2015), 1101--1114.Google ScholarGoogle ScholarCross RefCross Ref
  27. Jérôme Perrin, Philippe Fuchs, Corinne Roumes, and Francois Perret. 1998. Improvement of stereoscopic comfort through control of the disparity and of the spatial frequency content. In Aerospace/Defense Sensing and Controls. International Society for Optics and Photonics, 124--134.Google ScholarGoogle Scholar
  28. Jenny Read. 2005. Early computational processing in binocular vision and depth perception. Prog. Biophys. Mol. Biol. 87, 1 (2005), 77--108.Google ScholarGoogle ScholarCross RefCross Ref
  29. Feng Shao, Qiuping Jiang, Randi Fu, Mei Yu, and Gangyi Jiang. 2016. Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals. Opt. Express 24, 11 (2016), 11640--11653.Google ScholarGoogle ScholarCross RefCross Ref
  30. Feng Shao, Zhutuan Li, Qiuping Jiang, Gangyi Jiang, Mei Yu, and Zongju Peng. 2015. Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection. Displ. 39 (2015), 125--132.Google ScholarGoogle ScholarCross RefCross Ref
  31. Feng Shao, Wenchong Lin, Randi Fu, Mei Yu, and Gangyi Jiang. 2017. Optimizing multiview video plus depth retargeting technique for stereoscopic 3D displays. Opt. Express 25, 11 (2017), 12478--12492.Google ScholarGoogle ScholarCross RefCross Ref
  32. Feng Shao, Weisi Lin, Zhutuan Li, Gangyi Jiang, and Qionghai Dai. 2017. Toward simultaneous visual comfort and depth sensation optimization for stereoscopic 3-D experience. IEEE Trans. Cybern. 47, 12 (2017), 4521--4533.Google ScholarGoogle ScholarCross RefCross Ref
  33. V. De Silva, A. Fernando, S. Worrall, H. K. Arachchi, and A. Kondoz. 2011. Sensitivity analysis of the human visual system for depth cues in stereoscopic 3-D displays. IEEE Trans. Multimedia 13, 3 (2011), 498--506. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Hosik Sohn, Yong Ju Jung, Seong-Il Lee, Filippo Speranza, and Yong Man Ro. 2014. Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes. IEEE Trans. Circuits Syst. Video Technol. 24, 5 (2014), 745--758.Google ScholarGoogle ScholarCross RefCross Ref
  35. Wa James Tam, Filippo Speranza, Sumio Yano, Koichi Shimono, and Hiroshi Ono. 2011. Stereoscopic 3D-TV: Visual comfort. IEEE Trans. Broadcast. 57, 2 (2011), 335--346.Google ScholarGoogle ScholarCross RefCross Ref
  36. Shuai Wan, Bo Chang, and Fuzheng Yang. 2013. Viewing experience of 3D movie with subtitles: Where to put subtitles in a 3D movie? In 5th International Workshop on Quality of Multimedia Experience (QoMEX’13). IEEE, 170--175.Google ScholarGoogle Scholar
  37. Yan Wu, Qi Li, Jingjing Yang, Xiujun Li, and Ning Gao. 2016. Study on the impact of the depth of 3D subtitles on visual comfort. In 3rd International Conference on Information Science and Control Engineering (ICISCE’16). IEEE, 659--663.Google ScholarGoogle ScholarCross RefCross Ref
  38. Guanghui Yue, Chunping Hou, Ke Gu, Tianwei Zhou, and Guangtao Zhai. 2019. Combining local and global measures for DIBR-synthesized image quality evaluation. IEEE Trans. Image Process. 28, 4 (2019), 2075--2088.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Guanghui Yue, Chunping Hou, Qiuping Jiang, and Yang Yang. 2018. Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry. Signal Process. 150 (2018), 204--214.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Guanghui Yue, Chunping Hou, Kaining Lu, Dandan Feng, and Yao Li. 2016. Subjective visual comfort assessment based on fusion time for depth information. In 2016 11th International Conference on Computer Science 8 Education (ICCSE). IEEE, 733--737.Google ScholarGoogle ScholarCross RefCross Ref
  41. Li Zhang, Ya-Qin Zhang, Jing-Shang Zhang, Liang Xu, and Jost B Jonas. 2013. Visual fatigue and discomfort after stereoscopic display viewing. Acta Ophthalmol. 91, 2 (2013).Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Subtitle Region Selection of S3D Images in Consideration of Visual Discomfort and Viewing Habit

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!