Abstract
Video deblurring methods exploit the correlation between consecutive blurry inputs to generate sharp frames. However, designing an effective and efficient method is a challenging problem for video deblurring. To guarantee the effectiveness and further improve the deblurring performance, we adopt the recurrent-based method as the baseline and reconsider the recurrent mechanism as well as the temporal feature alignment in the state-of-the-art methods. For the recurrent mechanism, we add the local backward connection to the global forward recurrent backbone to effectively exploit accurate future information. For the temporal alignment, we adopt a fused temporal merge module that exploits the superiority of flow-based and kernel-based methods with progressive correlation volumes estimation. In addition, we evaluate our method with both synthetic datasets (GoPro, DVD) and a realistic dataset (BSD). The experimental results demonstrate that our method achieves significant performance improvement with a slight computational cost increase against the state-of-the-art video deblurring methods. The extended ablation studies verify the effectiveness of our model.
Supplemental Material
Available for Download
Supplementary material
- [1] . 2019. Depth-aware video frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3703–3712.Google Scholar
Cross Ref
- [2] . 2021. BasicVSR: The search for essential components in video super-resolution and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4947–4956.Google Scholar
Cross Ref
- [3] . 2022. BasicVSR++: Improving video super-resolution with enhanced propagation and alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5972–5981.Google Scholar
Cross Ref
- [4] . 2012. Video deblurring for hand-held cameras using patch-based synthesis. ACM Transactions on Graphics 31, 4 (2012), 1–9.Google Scholar
Digital Library
- [5] . 2015. Hand-held video deblurring via efficient fourier aggregation. IEEE Transactions on Computational Imaging 1, 4 (2015), 270–283.Google Scholar
Cross Ref
- [6] . 2020. Spatio-temporal saliency-based motion vector refinement for frame rate up-conversion. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 2 (2020), 1–18.Google Scholar
Digital Library
- [7] . 2022. You only align once: Bidirectional interaction for spatial-temporal video super-resolution. In Proceedings of the 30th ACM International Conference on Multimedia. 847–855.Google Scholar
Digital Library
- [8] . 2013. Dynamic scene deblurring. In Proceedings of the IEEE International Conference on Computer Vision. 3160–3167.Google Scholar
Digital Library
- [9] . 2014. Segmentation-free dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2766–2773.Google Scholar
Digital Library
- [10] . 2015. Generalized video deblurring for dynamic scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5426–5434.Google Scholar
Cross Ref
- [11] . 2017. Online video deblurring via dynamic temporal blending network. In Proceedings of the IEEE International Conference on Computer Vision. 4038–4047.Google Scholar
Cross Ref
- [12] . 2017. Dynamic video deblurring using a locally adaptive blur model. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 10 (2017), 2374–2387.Google Scholar
Digital Library
- [13] . 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- [14] . 2022. L0 structure-prior assisted blur-intensity aware efficient video deblurring. Neurocomputing 483 (2022), 195–209.Google Scholar
Digital Library
- [15] . 2021. Arvo: Learning all-range volumetric correspondence for video deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7721–7731.Google Scholar
Cross Ref
- [16] . 2019. Symmetrical residual connections for single image super-resolution. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 1 (2019), 1–10.Google Scholar
Digital Library
- [17] . 2010. Generating sharp panoramas from motion-blurred videos. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
IEEE ,Los Alamitos, CA , 2424–2431.Google ScholarCross Ref
- [18] . 2022. Flow-guided sparse transformer for video deblurring. In Proceedings of the International Conference on Machine Learning. 13334–13343.Google Scholar
- [19] . 2006. Full-frame video stabilization with motion inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence7 (2006), 1150–1163.Google Scholar
Digital Library
- [20] . 2019. NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Google Scholar
Cross Ref
- [21] . 2017. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3883–3891.Google Scholar
Cross Ref
- [22] . 2019. Recurrent neural networks with intra-frame iterations for video deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8102–8111.Google Scholar
Cross Ref
- [23] . 2020. Cascaded deep video deblurring using temporal sharpness prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3043–3051.Google Scholar
Cross Ref
- [24] . 2017. Video deblurring via semantic segmentation and pixel-wise non-linear kernel. In Proceedings of the IEEE International Conference on Computer Vision. 1077–1085.Google Scholar
Cross Ref
- [25] . 2021. Recurrent video deblurring with blur-invariant motion estimation and pixel volumes. ACM Transactions on Graphics 40, 5 (2021), 1–18.Google Scholar
Digital Library
- [26] . 2017. Deep video deblurring for hand-held cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1279–1288.Google Scholar
Cross Ref
- [27] . 2018. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8934–8943.Google Scholar
Cross Ref
- [28] . 2015. Learning a convolutional neural network for non-uniform motion blur removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 769–777.Google Scholar
Cross Ref
- [29] . 2019. EDVR: Video restoration with enhanced deformable convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Google Scholar
Cross Ref
- [30] . 2017. Learning blind motion deblurring. In Proceedings of the IEEE International Conference on Computer Vision. 231–240.Google Scholar
Cross Ref
- [31] . 2014. Modeling blurred video with layers. In Proceedings of the European Conference on Computer Vision. 236–252.Google Scholar
Cross Ref
- [32] . 2021. Omniscient video super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4429–4438.Google Scholar
Cross Ref
- [33] . 2014. Multi-observation blind deconvolution with an adaptive sparse prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 8 (2014), 1628–1643.Google Scholar
Digital Library
- [34] . 2022. Spatio-temporal deformable attention network for video deblurring. In Computer Vision—ECCV 2022. Lecture Notes in Computer Science, Vol. 13676. Springer, 581–596.Google Scholar
Digital Library
- [35] . 2020. Efficient spatio-temporal recurrent neural network for video deblurring. In Proceedings of the European Conference on Computer Vision. 191–207.Google Scholar
Digital Library
- [36] . 2019. Spatio-temporal filter adaptive network for video deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2482–2491.Google Scholar
Cross Ref
Index Terms
Local Bidirection Recurrent Network for Efficient Video Deblurring with the Fused Temporal Merge Module
Recommendations
Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes
For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help ...
Spatio-Temporal Deformable Attention Network for Video Deblurring
Computer Vision – ECCV 2022AbstractThe key success factor of the video deblurring methods is to compensate for the blurry pixels of the mid-frame with the sharp pixels of the adjacent video frames. Therefore, mainstream methods align the adjacent frames based on the estimated ...
Learn from the past – sequentially one-to-one video deblurring network
Highlights- A spatio-temporal recurrent network to utilize prior frames to benefit de-blurring.
AbstractWith the growing availability of hand-held cameras in recent years, more and more images and videos are taken at any time and any place. However, they usually suffer from undesirable blur due to camera shake or object motion in the ...






Comments