Abstract
Video stabilization aims to eliminate camera jitter and improve the visual experience of shaky videos. Video stabilization methods often ignore the active movement of the foreground objects and the camera, and may result in distortion and over-smoothing problems. To resolve these issues, this paper proposes a novel video stabilization method based on motion decomposition. Since the inter-frame movement of foreground objects is different from that of the background, we separate foreground feature points from background feature points by modifying the classic density based spatial clustering method of applications with noise (DBSCAN). The movement of background feature points is consistent with the movement of the camera, which can be decomposed into the camera jitter and the active movement of the camera. And the movement of foreground feature points can be decomposed into the movement of the camera and the active movement of foreground objects. Based on motion decomposition, we design first-order and second-order trajectory smoothing constraints to eliminate the high-frequency and low-frequency components of the camera jitter. To reduce content distortion, shape-preserving constraints, and regularization constraints are taken to generate stabilized views of all feature points. Experimental results demonstrate the effectiveness and robustness of the proposed video stabilization method on a variety of challenging videos.
- [1] . 2001. Non-metric image-based rendering for video stabilization. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- [2] . 2004. A robust and efficient video stabilization algorithm. In Proceedings of the 2004 IEEE International Conference on Multimedia and Expo.29–32.Google Scholar
- [3] . 2006. A robust real-time video stabilization algorithm. In Proceedings of the Journal of Visual Communication and Image Representation. 659–673.Google Scholar
Cross Ref
- [4] . 2020. Deep iterative frame interpolation for full-frame video stabilization. In Proceedings of the ACM Transactions on Graphics.1–9.Google Scholar
Digital Library
- [5] . 2015. Visual comfort for stereoscopic 3D by using motion sensors on 3D mobile devices. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 1s (2015), 1–20.Google Scholar
Digital Library
- [6] . 2016. Video stabilization for strict real-time applications. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 716–724.Google Scholar
- [7] . 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the kdd. 226–231.Google Scholar
- [8] . 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. In Proceedings of the Association for Computing Machinery. 381–395.Google Scholar
Digital Library
- [9] . 2008. Re-cinematography: Improving the camerawork of casual video. ACM Transactions on Multimedia Computing, Communications, and Applications 5, 1 (2008), 1–28.Google Scholar
Digital Library
- [10] . 2012. Video stabilization using epipolar geometry. In Proceedings of the ACM Transactions on Graphics.1–10.Google Scholar
Digital Library
- [11] . 2012. Calibration-free rolling shutter removal. In Proceedings of the 2012 IEEE International Conference on Computational Photography. 1–8.Google Scholar
Cross Ref
- [12] . 2012. Calibration-free rolling shutter removal. In Proceedings of the 2012 IEEE International Conference on Computational Photography. 1–8.Google Scholar
Cross Ref
- [13] . 2011. Auto-directed video stabilization with robust L1 optimal camera paths. In Proceedings of the CVPR 2011. 225–232.Google Scholar
Digital Library
- [14] . 2005. As-rigid-as-possible shape manipulation. In Proceedings of the ACM Transactions on Graphics. 1134–1141.Google Scholar
Digital Library
- [15] . 2010. Forward-backward error: Automatic detection of tracking failures. In Proceedings of the 2010 20th International Conference on Pattern Recognition. 2756–2759.Google Scholar
Digital Library
- [16] . 2015. Video stabilization based on feature trajectory augmentation and selection and robust mesh grid warping. In Proceedings of the IEEE Transactions on Image Processing. 5260–5273.Google Scholar
Digital Library
- [17] . 1980. Two algorithms for constructing a delaunay triangulation. In Proceedings of the International Journal of Computer Information Sciences. 219–242.Google Scholar
Cross Ref
- [18] . 2009. Video stabilization using robust feature trajectories. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. 1397–1404.Google Scholar
Cross Ref
- [19] . 2014. Video saliency detection based on spatiotemporal feature learning. In Proceedings of the 2014 IEEE International Conference on Image Processing. 1120–1124.Google Scholar
Cross Ref
- [20] . 2016. A feedback-based robust video stabilization method for traffic videos. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 561–572.Google Scholar
- [21] . 2009. Content-preserving warps for 3D video stabilization. ACM Transactions on Graphics 28, 3 (2009), 1–9.Google Scholar
Digital Library
- [22] . 2011. Subspace video stabilization. In Proceedings of the ACM Transactions on Graphics. 1–10.Google Scholar
Digital Library
- [23] . 2016. Meshflow: Minimum latency online video stabilization. In Proceedings of the European Conference on Computer Vision. 800–815.Google Scholar
Cross Ref
- [24] . 2012. Video stabilization with a depth camera. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. 89–95.Google Scholar
- [25] . 2016. A hybrid approach for near-range video stabilization. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 1922–1933.Google Scholar
- [26] . 2013. Bundled camera paths for video stabilization. In Proceedings of the ACM Trans. Graph. 10 pages.Google Scholar
Digital Library
- [27] . 2014. Steadyflow: Spatially smooth optical flow for video stabilization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4209–4216.Google Scholar
Digital Library
- [28] . 2021. Hybrid neural fusion for full-frame video stabilization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2299–2308.Google Scholar
Cross Ref
- [29] . 2019. Effective video stabilization via joint trajectory smoothing and frame warping. IEEE Transactions on Visualization and Computer Graphics 26, 11 (2019), 3163–3176.Google Scholar
- [30] . 2006. Full-frame video stabilization with motion inpainting. In Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence. 1150–1163.Google Scholar
Digital Library
- [31] . 2020. Action recognition using form and motion modalities. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 1s (2020), 1–16.Google Scholar
Digital Library
- [32] . 1996. Fast electronic digital image stabilization. In Proceedings of the 13th International Conference on Pattern Recognition. 284–288.Google Scholar
Cross Ref
- [33] . 2017. Dynamic video stitching via shakiness removing. In Proceedings of the IEEE Transactions on Image Processing. 164–178.Google Scholar
- [34] . 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, 91–99.Google Scholar
- [35] . 2003. Multiple view geometry in computer vision. In Proceedings of the Cambridge University Press.Google Scholar
- [36] . 2009. Light field video stabilization. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. 341–348.Google Scholar
Cross Ref
- [37] . 1991. Detection and tracking of point features. In Proceedings of the International Journal of Computer Vision.Google Scholar
- [38] . 2018. Deep online video stabilization with multi-grid warping transformation learning. IEEE Transactions on Image Processing 28, 5 (2018), 2283–2292.Google Scholar
Cross Ref
- [39] . 2013. Spatially and temporally optimized video stabilization. In Proceedings of the IEEE Transactions on Visualization and Computer Graphics. 1354–1361.Google Scholar
- [40] . 2013. Multiplane video stabilization. In Proceedings of the Computer Graphics Forum. 265–273.Google Scholar
Cross Ref
- [41] . 2019. Locally low-rank regularized video stabilization with motion diversity constraints. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 2873–2887.Google Scholar
Cross Ref
- [42] . 2021. Simultaneous video stabilization and rolling shutter removal. IEEE Transactions on Image Processing 30 (2021), 4637–4652.Google Scholar
Digital Library
- [43] . 2022. Improving feature discrimination for object tracking by structural-similarity-based metric learning. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 4 (2022), 1–23.Google Scholar
Digital Library
- [44] . 2022. Dut: Learning video stabilization by simply watching unstable videos. IEEE Transactions on Image Processing 31 (2022), 4306–4320.Google Scholar
Digital Library
- [45] . 2009. Robust video stabilization based on particle filter tracking of projected camera motion. 19, 7 (2009), 945–954.Google Scholar
- [46] . 2020. Learning video stabilization using optical flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8159–8167.Google Scholar
Cross Ref
- [47] . 2015. Simultaneous camera path optimization and distraction removal for improving amateur video. In Proceedings of the IEEE Transactions on Image Processing. 5982–5994.Google Scholar
Digital Library
- [48] . 2009. Video stabilization based on a 3D perspective camera model. In Proceedings of the Visual Computer. 997–1008.Google Scholar
Digital Library
- [49] . 2015. A global approach to fast video stabilization. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 225–235.Google Scholar
- [50] . 2020. PWStableNet: Learning pixel-wise warping maps for video stabilization. In Proceedings of the IEEE Transactions on Image Processing. 3582–3595.Google Scholar
Digital Library
- [51] . 2021. Adaptively meshed video stabilization. In Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology. 3504–3517.Google Scholar
Cross Ref
- [52] . 2013. Plane-based content preserving warps for video stabilization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2299–2306.Google Scholar
Digital Library
Index Terms
Robust Video Stabilization based on Motion Decomposition
Recommendations
Dual-mode video stabilization based on adaptive motion clustering
ICIMCS '15: Proceedings of the 7th International Conference on Internet Multimedia Computing and ServiceMany target tracking videos suffer from significant shake due to the undesirable camera motion or target vibration. These degraded video records have a negative impact on viewing experience as well as the follow-up analysis and application. Most ...
Video stabilization using epipolar geometry
We present a new video stabilization technique that uses projective scene reconstruction to treat jittered video sequences. Unlike methods that recover the full three-dimensional geometry of the scene, this model accounts for simple geometric relations ...
Real-time video stabilization for fast-moving vehicle cameras
Most previous methods of real-time video stabilization are only effective for low-vibrating frames which are usually captured by in-vehicle camera at the low-speed moving. To overcome their ineffectiveness on high-vibrating frames, this paper presents a ...






Comments