Abstract
Advanced 3D mobile devices attract a lot of attentions for 3D visualization nowadays. Stereoscopic images and video taken from the 3D mobile devices are uncomfortable for 3D viewing experiences due to the limited hardware for stereoscopic 3D stabilization. The existing stereoscopic 3D stabilization methods are computationally inefficient for the 3D mobile devices. In this article, we point out that this critical issue deteriorates the 3D viewing experiences on the 3D mobile devices. To improve visual comfort, we propose an efficient and effective algorithm to stabilize the stereoscopic images and video for the 3D mobile devices. To rectify the video jitter, we use the gyroscope and accelerometer embedded on the mobile devices to obtain the geometry information of the cameras. Using a different method than video-content-based motion estimation, our algorithm based on the gyroscope and acceleration data can achieve higher accuracy to effectively stabilize the video. Therefore, our approach is robust in video stabilization even under poor lighting and substantial foreground motion. Our algorithm outperforms previous approaches in not only smaller running time but also the better comfort of the stereoscopic 3D visualization for the 3D mobile devices.
- M. Abramowitz and I. A. Stegun. 1972. Handbook of Mathematical Functions. Dover Publications, New York, 72--89.Google Scholar
- R. S. Allison. 2007. Analysis of the influence of vertical disparities arising in toed-in stereoscopic cameras. J. Imag. Sci. Technol. 51, 4, 317--327.Google Scholar
Cross Ref
- Pravin Bhat, C. Lawrence Zitnick, Noah Snavely, Aseem Agrawala, Michael Cohen, Brian Curless, and Sing Bing Kang. 2007. Using photographs to enhance videos of a static scene. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (EGSR'07). Eurographics Association, Aire-la-Ville, Switzerland, 327--338. Google Scholar
Digital Library
- Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A perceptual model for disparity. ACM Trans. Graphics 30, 4 (2011), Article 96. DOI:10.1145/2010324.1964991 Google Scholar
Digital Library
- L. Falkenhagen. 1994. Depth estimation from stereoscopic image pairs assuming piecewise continuous surfaces. In Image Processing for Broadcast and Video Production, Springer, 115--127.Google Scholar
- M. A. Fischler, and R. C. Bolles. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 6, 381--395. Google Scholar
Digital Library
- Simon Heinzle, Pierre Greisen, David Gallup, Christine Chen, Daniel Saner, Aljoscha Smolic, Andreas Burg, Wojciech Matusik, and Markus Gross. 2011. Computational stereo camera system with programmable control loop. ACM Trans. Graphics 30, 4 (2011), Article 94. DOI:10.1145/2010324.1964989 Google Scholar
Digital Library
- T. S. Huang and A. N. Netravali. 1994. Motion and structure from feature correspondences: A review. Proc. IEEE 82, 2 (1994), 252--268.Google Scholar
Cross Ref
- S. Jain and U. Neumann. 2006. Real-time camera pose and focal length estimation. In Proceedings of the IEEE International Conference on Pattern Recognition. IEEE, 551--555. Google Scholar
Digital Library
- S. B. Kang. 1999. A survey of image-based rendering techniques. Ph.D dissertation, University of North Carolina at Chapel Hill. In VideoMetrics, SPIE, 2--16.Google Scholar
- Bahadir Karasulu and Sendar Korukoglu. 2013. Performance Evaluation Software: Moving Object Detection and Tracking in Videos. SpringerBriefs in Computer Science, Springer, 63--70. Google Scholar
Digital Library
- Frank L. Kooi and Alexander Toet. 2004. Visual comfort of binocular and 3D displays. Displays 25, 2--3 (2004), 99--108.Google Scholar
Cross Ref
- Marc Lambooij, Wijnand IJsselsteijn, Marten Fortuin, and Ingrid Heynderickx. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. J. Imag. Sci. Technol. 53, 3 (2009), 030201-1--030201-14.Google Scholar
Cross Ref
- Manuel Lang, Alexander, Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graphics 29, 4 (2010), Article 75. DOI:10.1145/1778765.1778812 Google Scholar
Digital Library
- Jeehong Lee, Kyu-yeol Chae, and S. Ji. 2012. The 3D video processing method in the stereoscopic camera for mobile devices. In Proceedings of the IEEE International Conference on Emerging Signal Processing Applications (ESPA). IEEE, 139--142.Google Scholar
- Ken-Yi Lee, Yung-Yu Chuang, Bing-Yu Chen, and Ming Ouhyoung. 2009. Video stabilization using robust feature trajectories. In Proceedings of the IEEE 12th International Conference on Computer Vision. IEEE, 1397--1404.Google Scholar
- K. Levenberg. 1944. A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math. 2. 164--168.Google Scholar
- Chun-Wei Liu, Tz-Huan Huang, Ming-Hsu Chang, Ken-Yi Lee, Chia-Kai Liang, and Yung-Yu Chuang. 2011. 3D cinematography principles and their applications to stereoscopic media processing. In Proceeings of the 19th ACM International Conference on Multimedia (MM'11). ACM, New York, 253--262. Google Scholar
Digital Library
- Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala. 2009. Content-preserving warps for 3D video stabilization. ACM Trans. Graphics 28, 3 (2009), Article 44. DOI:10.1145/1531326.1531350 Google Scholar
Digital Library
- Wan-Yen Lo., Jeroen van Baar, Claude Knaus, Matthias Zwicker, and Markus Gross. 2010. Stereoscopic 3D copy & paste. ACM Trans. Graphics 29, 6 (2010), Article 147. DOI:10.1145/1882261.1866173 Google Scholar
Digital Library
- S. Mangiat and J. Gibson. 2012. Disparity remapping for handheld 3D video communications. In Proceedings of the 2012 IEEE International Conference on Emerging Signal Processing Applications (ESPA). IEEE, 147--150.Google Scholar
- Wojciech Matusik and Hanspeter Pfister. 2004. 3D TV: A scalable for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graphics 23, 3 (August 2004), 814--824. DOI:10.1145/1015706.1015805 Google Scholar
Digital Library
- L. McMillan. 1997. An image-based approach on three-dimensional computer graphics. Ph.D dissertation. University of North Carolina at Chapel Hill, Chapel Hill, N.C. Google Scholar
Digital Library
- P. Mendapara. A. Baradarani, and Q. M. J. Wu. 2010. An efficient depth map estimation technique using complex wavelets. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1409--1414.Google Scholar
Cross Ref
- C. Morimoto and R. Chellapa. 1998. Evaluation of image stabilization algorithms. In Proceedings of the IEEE Internationsl Conference on Acoustics, Speech and Signal Processing. Vol. 5, IEEE, 2789--2792.Google Scholar
- Nguyen Ho Quoc Phuong, Hee-Jun Kang, Young-Soo Suh, and Young-Sik Ro. 2009. A DCM based orientation estimation algorithm with an inertial measurement unit and a magnetic compass. J. Univ. Comput. Sci. 15, 4, 859--876.Google Scholar
- V. Rabaud and S. Belongie. 2006. Counting crowded moving objects. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Google Scholar
Digital Library
- Rahul Raguram, Jan-Michael Frahm and Marc Pollefeys. 2008. A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus. In Proceedings of the 10th European Conference on Computer Vision, Part II (Computer Vision -- ECCV 2008). Lecture Notes in Computer Science, vol. 5303, Springer, Berlin, Heidelberg, 500--513. Google Scholar
Digital Library
- N. Ritter, R. Owens, J. Cooper, R. H. Eikelboom, and P. P. Van Saarloos. 1999. Registration of stereo and temporal images of the retina. IEEE Trans. Med. Imag. 18, 5, 404--418.Google Scholar
Cross Ref
- A. Sabatini. 2006. Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing. IEEE Trans. Biomed. Engin. 53, 7.Google Scholar
Cross Ref
- D. Scharstein and R. Szeliski. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 1, 7--42. Google Scholar
Digital Library
- Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered depth images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). ACM, New York, 231--242. DOI:10.1145/280814.280882 Google Scholar
Digital Library
- J. Shi and C. Tomasi. 1994. Good features to track. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Takashi Shibata, Joohwan Kim, David M. Hoffman, and Martin S. Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. J. Vis. 11, 8--11.Google Scholar
Cross Ref
- B. M. Smith, L. Zhang, H. Jin, and A. Agarwala. 2009. Light field video stabilization. In Proceedings of the IEEE 12th International Conference on Computer Vision. IEEE, 341--348.Google Scholar
- Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo tourism: Exploring photo collections in 3D. In Proceedings of ACM SIGGRAPH 2006 (SIGGRAPH'06). ACM, New York, 835--846. DOI:10.1145/1179352.1141964 Google Scholar
Digital Library
- Filippo Speranza, Wa J. Tam, Ron Renaud, and Namho Hur. 2006. Effect of disparity and motion on visual comfort of stereoscopic images. In Proceedings of the SPIE Stereoscopic Displays and Virtual Reality Systems XIII, Vol. 6055, 94--103.Google Scholar
Cross Ref
- Y. S. Suh. 2010. Orientation estimation using a quarternion-based Kalman filter with adative estimation acceleration. IEEE Trans. Instrum. Measure. 59, 12, 3296--3305.Google Scholar
Cross Ref
- Geng Sun and Nick Holliman. 2009. Evaluating methods for controlling depth perception in stereoscopic cinematography. Proc. SPIE, vol. 7237, Stereoscopic Displays and Applications XX, 72370I (2009). DOI:10.1117/12.807136Google Scholar
Cross Ref
- Wa James Tam, F. Speranza, S. Yano, K. Shimono, and H. Ono. 2011. Stereoscopic 3D-TV: Visual comfort. IEEE Trans. Broadcast. 57, 2, 335--346.Google Scholar
Cross Ref
- Wa James Tam and L. Zhang. 2006. 3D-TV content generation: 2D-to-3D conversion. In Proceedings of the IEEE International Conference on Multimedia and Expo. IEEE, 1869--1872.Google Scholar
- C. Tomasi, and R. Manduchi. 1998. Blateral filtering for gray and color images. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 839--846. Google Scholar
Digital Library
- N. Uchida, T. Shibahara, T. Aoki, H. Nakajima, and K. Kobayashi. 2005. 3D face recognition using passive stereo vision. In Proceedings of the IEEE International Conference on Image Processing (ICIP'05). IEEE, 950--953.Google Scholar
- Chiao Wang and Alexander A. Sawchuk. 2008. Disparity manipulation for stereo images and video. Proc. SPIE, vol. 6803, Stereoscopic Displays and Applications XIX, 68031E (February 29, 2008). DOI:10.1117/12.767702Google Scholar
- J. M. Wang, H. P. Chou, S. W. Chen, and C. S. Fuh. 2009. Video stabilization for a hand-held camera based on 3D motion model. In Proceedings of the 16th IEEE International Conference on Image Processing (ICIP). IEEE, 3477--3480. Google Scholar
Digital Library
- O. Wang, M. Lang, M. Frei, A. Hornung, A. Smolic, and M. Gross. 2011. Stereobrush: Interactive 2D to 3D conversion using discontinuous warps. In Proceedings of the EUROGRAPHICS Symposium on Sketch-Based Interfaces and Modeling. 47--54. Google Scholar
Digital Library
- L. Zhang and W. J. Tam. 2005. Stereoscopic image generation based on depth images for 3D TV. IEEE Trans. Broadcast. 51, 2, 191--199.Google Scholar
Cross Ref
Index Terms
Visual Comfort for Stereoscopic 3D by Using Motion Sensors on 3D Mobile Devices
Recommendations
Perceptual Tolerance to Stereoscopic 3D Image Distortion
An intriguing aspect of picture perception is the viewer’s tolerance to variation in viewing position, perspective, and display size. These factors are also present in stereoscopic media, where there are additional parameters associated with the camera ...
3D Elastic Control for Mobile Devices
To increase the input space of mobile devices, the authors developed a proof-of-concept 3D elastic controller that easily adapts to mobile devices. This embedded device improves the completion of high-level interaction tasks such as visualization of ...
Investigating mobile stereoscopic 3D touchscreen interaction
OzCHI '13: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration3D output is no longer limited to large screens in cinemas or living rooms. Nowadays more and more mobile devices are equipped with autostereoscopic 3D (S3D) touchscreens. As a consequence interaction with 3D content now also happens whilst users are on ...






Comments