Abstract
The surround-view module is an indispensable component of a modern advanced driving assistance system. By calibrating the intrinsics and extrinsics of the surround-view cameras accurately, a top-down surround-view can be generated from raw fisheye images. However, poses of these cameras sometimes may change. At present, how to correct poses of cameras in a surround-view system online without re-calibration is still an open issue. To settle this problem, we introduce the sparse direct framework and propose a novel optimization scheme of a cascade structure. This scheme is actually composed of two levels of optimization and two corresponding photometric error based models are proposed. The model for the first-level optimization is called the ground model, as its photometric errors are measured on the ground plane. For the second level of the optimization, it’s based on the so-called ground-camera model, in which photometric errors are computed on the imaging planes. With these models, the pose correction task is formulated as a nonlinear least-squares problem to minimize photometric errors in overlapping regions of adjacent bird’s-eye-view images. With a cascade structure of these two levels of optimization, an appropriate balance between the speed and the accuracy can be achieved. Experiments show that our method can effectively eliminate the misalignment caused by cameras’ moderate pose changes in the surround-view system. Source code and test cases are available online at https://cslinzhang.github.io/CamPoseCorrection/.
- [1] . 1992. First- and second-order methods for learning: Between steepest descent and Newton’s method. Neural Computation 4, 2 (1992), 141–166.Google Scholar
Digital Library
- [2] . 2006. SURF: Speeded up robust features. In Proc. European Conf. Comput. Vis.404–417.Google Scholar
- [3] . 2018. Automatic calibration of an around view monitor system exploiting lane markings. Sensors 18, 9 (2018), 2956:1–26.Google Scholar
Cross Ref
- [4] . 2006. Self-calibration of an on-board stereo-vision system for driver assistance systems. In Proc. Int’l IEEE Conf. Intell. Vehicles Symposium. 156–162.Google Scholar
- [5] . 2006. Tracking camera parameters of an active stereo rig. In Joint DAGM Symposium. 627-636.Google Scholar
- [6] . 1983. Numerical methods for unconstrained optimization and nonlinear equations. Prentice Hall, Inc. 28, 3 (1983), 417–419.Google Scholar
- [7] . 1993. Self-calibration of the intrinsic parameters of cameras for active vision systems. In Proc. IEEE Int’l Conf. Comput. Vis. Pattern Recognit.477–482.Google Scholar
- [8] . 2018. Direct sparse odometry. IEEE Trans. Pattern Analysis and Machine Intell. 40, 3 (2018), 611–625.Google Scholar
Cross Ref
- [9] . 2014. LSD-SLAM: Large-scale direct monocular SLAM. In Proc. European Conf. Comput. Vis.834–849.Google Scholar
- [10] . 2014. SVO: Fast semi-direct monocular visual odometry. In Proc. IEEE Int’l Conf. Robotics and Automation. 15–22.Google Scholar
- [11] . 2011. Surround view pedestrian detection using heterogeneous classifier cascades. In Proc. Int’l IEEE Conf. Intell. Transportation Systems. 1317–1324.Google Scholar
- [12] . 2012. Online continuous stereo extrinsic parameter estimation. In Proc. IEEE Int’l Conf. Comput. Vis. Pattern Recognit.1059–1066.Google Scholar
- [13] . 2003. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press, USA.Google Scholar
Digital Library
- [14] . 2014. Infrastructure-based calibration of a multi-camera rig. In Proc. IEEE Int’l Conf. Robotics and Automation. 4912–4919.Google Scholar
- [15] . 2013. CamOdoCal: Automatic intrinsic and extrinsic calibration of a rig with multiple generic cameras and odometry. In Proc. IEEE/RSJ Int’l Conf. Intell. Robots and Systems. 1793–1800.Google Scholar
- [16] . 1966. The Lie algebra of visual perception. J. Mathematical Psychology 3, 1 (1966), 65–98.Google Scholar
Cross Ref
- [17] . 2009. A novel approach for the online initial calibration of extrinsic parameters for a car-mounted camera. In Proc. Int’l IEEE Conf. Intell. Transportation Systems. 420–425.Google Scholar
- [18] . 2007. Multiview pedestrian detection based on vector boosting. In Proc. Asian Conf. Comput. Vis.18–22.Google Scholar
- [19] . 1999. About direct methods. In Proc. Int’l Workshop on Vis. Algorithms. 267–277.Google Scholar
- [20] . 1998. Computer Vision: Three-dimensional Data from Images. Springer, Singapore.Google Scholar
Digital Library
- [21] . 2013. Online extrinsic multi-camera calibration using ground plane induced homographies. In Proc. IEEE Intell. Vehicles Symposium. 236–241.Google Scholar
- [22] . 2011. Fast calibration of embedded non-overlapping cameras. In Proc. IEEE Int’l Conf. Robotics and Automation. 221–227.Google Scholar
- [23] . 2017. Vision-based parking-slot detection: A benchmark and a learning-based approach. In Proc. IEEE Int’l Conf. Multimedia and Expo. 649–654.Google Scholar
- [24] . 2012. A vision based top-view transformation model for a vehicle parking assistant. Sensors 12, 4 (2012), 4431–4446.Google Scholar
- [25] . 2016. High-precision online markerless stereo extrinsic calibration. In Proc. IEEE/RSJ Int’l Conf. Intell. Robots and Systems. 1771–1778.Google Scholar
- [26] . 2019. Online camera pose optimization for the surround-view system. In Proc. ACM Int’l Conf. Multimedia. 383–391.Google Scholar
- [27] . 2019. Sparse non-linear least squares optimization for geometric vision. In Proc. European Conf. Comput. Vis.43–56.Google Scholar
- [28] . 2004. Distinctive image features from scale-invariant keypoints. Int’l J. Comput. Vis. 60, 2 (2004), 91–110.Google Scholar
Digital Library
- [29] . 1981. An iterative image registration technique with an application to stereo vision. In Proc. Int’l Joint Conf. Artificial Intell.674–679.Google Scholar
- [30] . 1978. The Levenberg-Marquardt algorithm: Implementation and theory. In Numerical Analysis.Google Scholar
- [31] . 2017. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robotics 33, 5 (2017), 1255–1262.Google Scholar
Digital Library
- [32] . 2007. Online extrinsic parameters calibration for stereo vision systems used in far-range detection vehicle applications. IEEE Trans. Intell. Transportation Systems 8, 4 (2007).Google Scholar
Digital Library
- [33] . 2011. DTAM: Dense tracking and mapping in real-time. In Proc. IEEE Int’l Conf. Comput. Vis.2320–2327.Google Scholar
- [34] . 2005. Surround video: A multihead camera approach. The Visual Computer 21, 1-2 (2005), 92–103.Google Scholar
Digital Library
- [35] . 1992. Theory of algorithms for unconstrained optimization. Acta Numerica 1, 8 (1992), 199–242.Google Scholar
Cross Ref
- [36] . 2015. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Trans. Robotics 31, 5 (2015), 1147–1163.Google Scholar
Digital Library
- [37] . 2011. ORB: An efficient alternative to SIFT or SURF. Proc. IEEE Int’l Conf. Comput. Vis. (2011), 2564–2571.Google Scholar
- [38] . 2013. Odometry-based online extrinsic sensor calibration. In Proc. IEEE/RSJ Int’l Conf. Intell. Robots and Systems. 1287–1292.Google Scholar
- [39] . 2019. Revisit surround-view camera system calibration. In Proc. IEEE Int’l Conf. Multimedia and Expo. 1486–1491.Google Scholar
- [40] . 1974. Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika 61, 3 (1974), 439–447.Google Scholar
- [41] . 2000. Vision-guided automatic parking for smart car. In Proc. IEEE Intell. Vehicles Symposium. 725–730.Google Scholar
- [42] . 2018. Vision-based parking-slot detection: A DCNN-based approach and a large-scale benchmark dataset. IEEE Trans. Image Processing 27, 11 (2018), 5350–5364.Google Scholar
Digital Library
- [43] . 1999. Flexible camera calibration by viewing a plane from unknown orientations. In Proc. IEEE Int’l Conf. Comput. Vis.666–673.Google Scholar
- [44] . 2014. An automatic online camera calibration system for vehicular applications. In Proc. Int’l IEEE Conf. Intell. Transportation Systems. 1490–1492.Google Scholar
- [45] . 2009. Fisheye camera calibration with two pairs of vanishing points. In Proc. Int’l Conf. Inf. Tech. Comput. Sci.321–324.Google Scholar
Index Terms
Online Correction of Camera Poses for the Surround-view System: A Sparse Direct Approach
Recommendations
ROECS: A Robust Semi-direct Pipeline Towards Online Extrinsics Correction of the Surround-view System
MM '21: Proceedings of the 29th ACM International Conference on MultimediaGenerally, a surround-view system (SVS), which is an indispensable component of advanced driving assistant systems (ADAS), consists of four to six wide-angle fisheye cameras. As long as both intrinsics and extrinsics of all cameras have been calibrated, ...
Online Camera Pose Optimization for the Surround-view System
MM '19: Proceedings of the 27th ACM International Conference on MultimediaSurround-view system is an important information medium for drivers to monitor the driving environment. A typical surround-view system consists of four to six fish-eye cameras arranged around the vehicle. From these camera inputs, a top-down image of ...
Handling pure camera rotation in semi-dense monocular SLAM
In this paper, we present a method for semi-dense monocular simultaneous localization and mapping (SLAM) that is capable of dealing with pure camera rotation motion which brings forward a severe challenge for current direct (featureless) monocular SLAM ...






Comments