Abstract
Subtitles, serving as a linguistic approximation of the visual content, are an essential element in stereoscopic advertisement and the film industry. Due to the vergence accommodation conflict, the stereoscopic 3D (S3D) subtitle inevitably causes visual discomfort. To meet the viewing experience, the subtitle region should be carefully arranged. Unfortunately, very few works have been dedicated to this area. In this article, we propose a method for S3D subtitle region selection in consideration of visual discomfort and viewing habit. First, we divide the disparity map into multiple depth layers according to the disparity value. The preferential processed depth layer is determined by considering the disparity value of the foremost object. Second, the optimal region and coarse disparity value for S3D subtitle insertion are chosen by convolving the selective depth layer with the mean filter. Specifically, the viewing habit is considered during the region selection. Finally, after region selection, the disparity value of the subtitle is further modified by using the just noticeable depth difference (JNDD) model. Given that there is no public database reported for the evaluation of S3D subtitle insertion, we collect 120 S3D images as the test platform. Both objective and subjective experiments are conducted to evaluate the comfort degree of the inserted subtitle. Experimental results demonstrate that the proposed method can obtain promising performance in improving the viewing experience of the inserted subtitle.
- Kurt Akeley, Simon J. Watt, Ahna Reza Girshick, and Martin S. Banks. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graphics 23, 3 (2004), 804--813. Google Scholar
Digital Library
- Rec. ITU-R BT-500.11. 2002. Methodology for the subjective assessment of the quality of television pictures. (2002).Google Scholar
- Rec. ITU-R BT.1438. 2000. Methodology for the subjective assessment of the quality of television pictures. (2000).Google Scholar
- Zaiqing Chen, Junsheng Shi, Xiaoqiao Huang, Lijun Yun, and Yonghan Tai. 2014. Visual comfort modeling for disparity in 3D contents based on Weber--Fechner’s Law. J. Disp. Technol. 10, 12 (2014), 1001--1009.Google Scholar
Cross Ref
- Chung-Hua Chu. 2015. Visual comfort for stereoscopic 3D by using motion sensors on 3D mobile devices. ACM Trans. Multimedia Comput. Commun. Appl. 12, 1s (2015), 14. Google Scholar
Digital Library
- D. Varuna SX De Silva, Erhan Ekmekcioglu, Warnakulasuriya Anil Chandana Fernando, and Stewart T. Worrall. 2011. Display dependent preprocessing of depth maps based on just noticeable depth difference modeling. IEEE J. Sel. Top. Signal Process. 5, 2 (2011), 335--351.Google Scholar
Cross Ref
- Song-Pei Du, Belen Masia, Shi-Min Hu, and Diego Gutierrez. 2013. A metric of visual comfort for stereoscopic motion. ACM Trans. Graph. 32, 6 (2013), 222. Google Scholar
Digital Library
- Yuming Fang, Junle Wang, Manish Narwaria, Patrick Le Callet, and Weisi Lin. 2014. Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23, 6 (2014), 2625--2636. Google Scholar
Digital Library
- Yuming Fang, Chi Zhang, Jing Li, Jianjun Lei, Matthieu Perreira Da Silva, and Patrick Le Callet. 2017. Visual attention modeling for stereoscopic video: A benchmark and computational model. IEEE Trans. Image Process. 26, 10 (2017), 4684--4696.Google Scholar
Digital Library
- Lutz Goldmann, Francesca De Simone, and Touradj Ebrahimi. 2010. Impact of acquisition distortion on the quality of stereoscopic images. In Proceedings of the International Workshop on Video Processing and Quality Metrics for Consumer Electronics.Google Scholar
- Nicolas S. Holliman, Neil A. Dodgson, Gregg E. Favalora, and Lachlan Pockett. 2011. Three-dimensional displays: A review and applications analysis. IEEE Trans. Broadcast. 57, 2 (2011), 362--371.Google Scholar
Cross Ref
- Qiuping Jiang, Feng Shao, Gangyi Jiang, Mei Yu, and Zongju Peng. 2017. Visual comfort assessment for stereoscopic images based on sparse coding with multi-scale dictionaries. Neurocomput. 252 (2017), 77--86. Google Scholar
Digital Library
- Qiuping Jiang, Feng Shao, Weisi Lin, and Gangyi Jiang. 2016. On predicting visual comfort of stereoscopic images: A learning to rank based approach. IEEE Signal Process. Lett. 23, 2 (2016), 302--306.Google Scholar
Cross Ref
- Cheolkon Jung, Lihui Cao, Hongmin Liu, and Joongkyu Kim. 2015. Visual comfort enhancement in stereoscopic 3D images using saliency-adaptive nonlinear disparity mapping. Disp. 40 (2015), 17--23.Google Scholar
Cross Ref
- Seung-Won Jung and Sung-Jea Ko. 2012. Depth sensation enhancement using the just noticeable depth difference. IEEE Trans. Image Process. 21, 8 (2012), 3624--3637. Google Scholar
Digital Library
- Yong Ju Jung, Hosik Sohn, Seong-Il Lee, Hyun Wook Park, and Yong Man Ro. 2013. Predicting visual discomfort of stereoscopic images using human attention model. IEEE Trans. Circuits Syst. Video Technol. 23, 12 (2013), 2077--2082. Google Scholar
Digital Library
- Bochao Kan, Yan Zhao, and Shigang Wang. 2018. Objective visual comfort evaluation method based on disparity information and motion for stereoscopic video. Opt. Express 26, 9 (2018), 11418--11437.Google Scholar
Cross Ref
- Taewan Kim, Sanghoon Lee, and Alan Conrad Bovik. 2015. Transfer function model of physiological mechanisms underlying temporal visual discomfort experienced when viewing stereoscopic 3D images. IEEE Trans. Image Process. 24, 11 (2015), 4335--4347.Google Scholar
Cross Ref
- M. Lambooij, M.J. Murdoch, Wijnand A. IJsselsteijn, and Ingrid Heynderickx. 2013. The impact of video characteristics and subtitles on visual comfort of 3D TV. Disp. 34, 1 (2013), 8--16.Google Scholar
Cross Ref
- Marc T.M. Lambooij, W.A. IJsselsteijn, and Ingrid Heynderickx. 2007. Visual discomfort in stereoscopic dsplays: A review. Proceedings of SPIE, 2007 vol. 6490.Google Scholar
- Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear disparity mapping for stereoscopic 3D. In ACM Trans. Graph. 29, 4 (2010), 75:1--10. Google Scholar
Digital Library
- Jianjun Lei, Cuicui Zhang, Yuming Fang, Zhouye Gu, Nam Ling, and Chunping Hou. 2015. Depth sensation enhancement for multiple virtual view rendering. IEEE Trans. Multimedia 17, 4 (2015), 457--469.Google Scholar
Cross Ref
- Xingang Liu, Kai Kang, and Yinbo Liu. 2016. Stereoscopic image quality assessment based on depth and texture information. IEEE Syst. J. 2016.Google Scholar
- H. Oh and S. Lee. 2016. Visual presence: Viewing geometry visual information of UHD S3D entertainment. IEEE Trans. Image Process. 25, 7 (2016), 3358--3371.Google Scholar
Cross Ref
- Jincheol Park, Sanghoon Lee, and Alan Conrad Bovik. 2014. 3D visual discomfort prediction: Vergence, foveation, and the physiological optics of accommodation. IEEE J. Sel. Top. Signal Process. 3, 8 (2014), 415--427.Google Scholar
Cross Ref
- Jincheol Park, Heeseok Oh, Sanghoon Lee, and Alan Conrad Bovik. 2015. 3D visual discomfort predictor: Analysis of disparity and neural activity statistics. IEEE Trans. Image Process. 24, 3 (2015), 1101--1114.Google Scholar
Cross Ref
- Jérôme Perrin, Philippe Fuchs, Corinne Roumes, and Francois Perret. 1998. Improvement of stereoscopic comfort through control of the disparity and of the spatial frequency content. In Aerospace/Defense Sensing and Controls. International Society for Optics and Photonics, 124--134.Google Scholar
- Jenny Read. 2005. Early computational processing in binocular vision and depth perception. Prog. Biophys. Mol. Biol. 87, 1 (2005), 77--108.Google Scholar
Cross Ref
- Feng Shao, Qiuping Jiang, Randi Fu, Mei Yu, and Gangyi Jiang. 2016. Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals. Opt. Express 24, 11 (2016), 11640--11653.Google Scholar
Cross Ref
- Feng Shao, Zhutuan Li, Qiuping Jiang, Gangyi Jiang, Mei Yu, and Zongju Peng. 2015. Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection. Displ. 39 (2015), 125--132.Google Scholar
Cross Ref
- Feng Shao, Wenchong Lin, Randi Fu, Mei Yu, and Gangyi Jiang. 2017. Optimizing multiview video plus depth retargeting technique for stereoscopic 3D displays. Opt. Express 25, 11 (2017), 12478--12492.Google Scholar
Cross Ref
- Feng Shao, Weisi Lin, Zhutuan Li, Gangyi Jiang, and Qionghai Dai. 2017. Toward simultaneous visual comfort and depth sensation optimization for stereoscopic 3-D experience. IEEE Trans. Cybern. 47, 12 (2017), 4521--4533.Google Scholar
Cross Ref
- V. De Silva, A. Fernando, S. Worrall, H. K. Arachchi, and A. Kondoz. 2011. Sensitivity analysis of the human visual system for depth cues in stereoscopic 3-D displays. IEEE Trans. Multimedia 13, 3 (2011), 498--506. Google Scholar
Digital Library
- Hosik Sohn, Yong Ju Jung, Seong-Il Lee, Filippo Speranza, and Yong Man Ro. 2014. Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes. IEEE Trans. Circuits Syst. Video Technol. 24, 5 (2014), 745--758.Google Scholar
Cross Ref
- Wa James Tam, Filippo Speranza, Sumio Yano, Koichi Shimono, and Hiroshi Ono. 2011. Stereoscopic 3D-TV: Visual comfort. IEEE Trans. Broadcast. 57, 2 (2011), 335--346.Google Scholar
Cross Ref
- Shuai Wan, Bo Chang, and Fuzheng Yang. 2013. Viewing experience of 3D movie with subtitles: Where to put subtitles in a 3D movie? In 5th International Workshop on Quality of Multimedia Experience (QoMEX’13). IEEE, 170--175.Google Scholar
- Yan Wu, Qi Li, Jingjing Yang, Xiujun Li, and Ning Gao. 2016. Study on the impact of the depth of 3D subtitles on visual comfort. In 3rd International Conference on Information Science and Control Engineering (ICISCE’16). IEEE, 659--663.Google Scholar
Cross Ref
- Guanghui Yue, Chunping Hou, Ke Gu, Tianwei Zhou, and Guangtao Zhai. 2019. Combining local and global measures for DIBR-synthesized image quality evaluation. IEEE Trans. Image Process. 28, 4 (2019), 2075--2088.Google Scholar
Digital Library
- Guanghui Yue, Chunping Hou, Qiuping Jiang, and Yang Yang. 2018. Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry. Signal Process. 150 (2018), 204--214.Google Scholar
Digital Library
- Guanghui Yue, Chunping Hou, Kaining Lu, Dandan Feng, and Yao Li. 2016. Subjective visual comfort assessment based on fusion time for depth information. In 2016 11th International Conference on Computer Science 8 Education (ICCSE). IEEE, 733--737.Google Scholar
Cross Ref
- Li Zhang, Ya-Qin Zhang, Jing-Shang Zhang, Liang Xu, and Jost B Jonas. 2013. Visual fatigue and discomfort after stereoscopic display viewing. Acta Ophthalmol. 91, 2 (2013).Google Scholar
Cross Ref
Index Terms
Subtitle Region Selection of S3D Images in Consideration of Visual Discomfort and Viewing Habit
Recommendations
Eye movements and visual discomfort when viewing stereoscopic 3D content
AbstractThe visual brain fuses the left and right images projected onto the two eyes from a stereoscopic 3D (S3D) display, perceives parallax, and rebuilds a sense of depth. In this process, the eyes adjust vergence and accommodation to adapt ...
Optimal Region Selection for Stereoscopic Video Subtitle Insertion
Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public ...
Transfer Function Model of Physiological Mechanisms Underlying Temporal Visual Discomfort Experienced When Viewing Stereoscopic 3D Images
When viewing 3D images, a sense of visual comfort (or lack of) is developed in the brain over time as a function of binocular disparity and other 3D factors. We have developed a unique temporal visual discomfort model (TVDM) that we use to automatically ...






Comments