skip to main content
research-article

Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

Authors Info & Claims
Published:28 September 2021Publication History
Skip Abstract Section

Abstract

The precise measuring of vehicle location has been a critical task in enhancing the autonomous driving in terms of intelligent decision making and safe transportation. Internet of Vehicles (IoV) is an important infrastructure in support of autonomous driving, allowing real-time road information exchanging and sharing for localizing vehicles. Global positioning System (GPS) is widely used in the traditional IoV system. GPS is unable to meet the key application requirements of autonomous driving due to meter level error and signal deterioration. In this article, we propose a novel solution, named Semi-Direct Monocular Visual-Inertial Odometry using Point and Line Features (SDMPL-VIO) for precise vehicle localization. Our SDMPL-VIO model takes advantage of a low-cost Inertial Measurement Unit (IMU) and monocular camera, using them as the sensor to acquire the surrounding environmental information. Visual-Inertial Odometry (VIO), taking into account both point and line features, is proposed, which is able to deal with both weak texture and dynamic environment. We use a semi-direct method to deal with keyframes and non-keyframes, respectively. Dual sliding window mechanisms can effectively fuse point-line and IMU information. To evaluate our SDMPL-VIO system model, we conduct extensive experiments on both an indoor dataset (i.e., EuRoC) and an outdoor dataset (i.e., KITTI) from the real-world applications, respectively. The experimental results show that the accuracy of SDMPL-VIO proposed by us is better than the mainstream VIO system at present. Especially in the weak texture of the datasets, fast-moving datasets, and other challenging datasets, SDMPL-VIO has a relatively high robustness.

References

  1. Mehiddin Albaali and Roger Fletcher. 1985. Variational methods for non-linear least-squares. Journal of the Operational Research Society 36, 5 (1985), 405–421.Google ScholarGoogle ScholarCross RefCross Ref
  2. Guillaume Bresson, Thomas Feraud, Romuald Aufrere, Paul Checchin, and Roland Chapuis. 2015. Real-time monocular slam with low memory requirements. IEEE Transactions on Intelligent Transportation Systems 16, 4 (2015), 1827–1839.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus Achtelik, and Roland Siegwart. 2016. The EuRoC micro aerial vehicle datasets. The International Journal of Robotics Research 35, 10 (2016), 1157–1163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Alejo Concha, Giuseppe Loianno, Vijay Kumar, and Javier Civera. 2016. Visual-inertial direct SLAM. In 2016 IEEE International Conference on Robotics and Automation. 1331–1338.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Christian Forster, Luca Carlone, Frank Dellaert, and Davide Scaramuzza. 2015. On-manifold preintegration for real-time visual-inertial odometry. IEEE Transactions on Robotics 33, 1 (2015), 1–21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Christian Forster, Zichao Zhang, Michael Gassner, Manuel Werlberger, and Davide Scaramuzza. 2017. SVO: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions on Robotics 33, 2 (2017), 249–265. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. 2013. Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research 32, 11 (2013), 1231–1237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ruben Gomezojeda and Javier Gonzalezjimenez. 2016. Robust stereo visual odometry through a probabilistic combination of points and line segments. In 2016 IEEE International Conference on Robotics and Automation. 2521–2526.Google ScholarGoogle Scholar
  9. Gottfried Graber, Thomas Pock, and Horst Bischof. 2011. Online 3D reconstruction using convex optimization. In 2011 IEEE International Conference on Computer Vision Workshops. 708–711.Google ScholarGoogle ScholarCross RefCross Ref
  10. Chunzhao Guo, Kiyosumi Kidono, Junichi Meguro, Yoshiko Kojima, Masaru Ogawa, and Takashi Naito. 2016. A low-cost solution for automatic lane-level map generation using conventional in-car sensors. IEEE Transactions on Intelligent Transportation Systems 17, 8 (2016), 2355–2366.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Yijia He, Ji Zhao, Yue Guo, Wenhao He, and Kui Yuan. 2018. PL-VIO: Tightly-coupled monocular visual-inertial odometry using point and line features. Sensors 18, 4 (2018), 1159.Google ScholarGoogle ScholarCross RefCross Ref
  12. Manuel Hofer, Michael Maurer, and Horst Bischof. 2015. Line3D: Efficient 3D scene abstraction for the built environment. In German Conference on Pattern Recognition. 237–248.Google ScholarGoogle ScholarCross RefCross Ref
  13. Georg Klein and D. W. Murray. 2007. Parallel tracking and mapping for small AR workspaces. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. 225–234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Xianglong Kong, Wenqi Wu, Lilian Zhang, and Yujie Wang. 2015. Tightly-coupled stereo visual-inertial navigation using point and line features. Sensors 15, 6 (2015), 12816–12833.Google ScholarGoogle ScholarCross RefCross Ref
  15. Stefan Leutenegger, Simon Lynen, Michael Bosse, Roland Siegwart, and Paul Furgale. 2014. Keyframe-based visual-inertial odometry using nonlinear optimization. International Journal of Robotics Research 34, 3 (2014), 314–334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xiaohu Lin, Fuhong Wang, Lei Guo, and Wanwei Zhang. 2019. An automatic key-frame selection method for monocular visual odometry of ground vehicle. IEEE Access 7 (2019), 70742–70754.Google ScholarGoogle ScholarCross RefCross Ref
  17. Yanqing Liu, Dongdong Yang, Jiamao Li, Yuzhang Gu, Jiatian Pi, and Xiaolin Zhang. 2018. Stereo visual-inertial SLAM with points and lines. IEEE Access 6 (2018), 69381–69392.Google ScholarGoogle ScholarCross RefCross Ref
  18. Lupton and Sukkarieh. 2012. Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions. IEEE Transactions on Robotics 28, 1 (2012), 61–76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Agostino Martinelli. 2012. Vision and IMU Data Fusion: Closed-form solutions for attitude, speed, absolute scale and bias determination. IEEE Transactions on Robotics 28, 1 (2012), 44–60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Chen Min, Yuanwen Tian, Giancarlo Fortino, Zhang Jing, and Iztok Humar. 2018. Cognitive internet of vehicles. Computer Communications (2018), 58–76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Anastasios I. Mourikis and Stergios I. Roumeliotis. 2007. A multi-state constraint Kalman filter for vision-aided inertial navigation. In 2007 IEEE International Conference on Robotics and Automation. 3565–3572.Google ScholarGoogle Scholar
  22. Raul Mur-Artal and Juan Domingo Tardos. 2016. ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Transactions on Robotics PP, 99 (2016), 1255–1262.Google ScholarGoogle Scholar
  23. Raul Mur-Artal and Juan Domingo Tardos. 2016. Visual-inertial monocular SLAM with map reuse. IEEE Robotics & Automation Letters 2, 2 (2016), 796–803.Google ScholarGoogle ScholarCross RefCross Ref
  24. Raul Murartal, J. M. M. Montiel, and Juan D. Tardos. 2017. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics 31, 5 (2017), 1147–1163.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Daiduong Nguyen, Abdelhafid Elouardi, Sergio Alberto Rodriguez Florez, and Samir Bouaziz. 2018. HOOFR SLAM System: An embedded vision SLAM algorithm and its hardware-software mapping-based intelligent vehicles applications. IEEE Transactions on Intelligent Transportation Systems (2018), 1–16.Google ScholarGoogle Scholar
  26. Taih Pire, Thomas Fischer, Gastn Castro, Pablo Decristforis, Javier Civera, and Julio Jacoboberlles. 2017. S-PTAM: Stereo parallel tracking and mapping. Robotics & Autonomous Systems 93 (2017), 27–42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Matia Pizzoli, Christian Forster, and Davide Scaramuzza. 2014. REMODE: Probabilistic, monocular dense reconstruction in real time. In 2014 IEEE International Conference on Robotics & Automation. 2609–2616.Google ScholarGoogle ScholarCross RefCross Ref
  28. Pedro F. Proena and Yang Gao. 2018. Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty. Robotics & Autonomous Systems 104 (2018), 25–39.Google ScholarGoogle ScholarCross RefCross Ref
  29. Albert Pumarola, Alexander Vakhitov, Antonio Agudo, Alberto Sanfeliu, and Francese Morenonoguer. 2017. PL-SLAM: Real-time monocular visual SLAM with points and lines. In 2017 IEEE International Conference on Robotics and Automation. 4503–4508.Google ScholarGoogle ScholarCross RefCross Ref
  30. Tong Qin, Peiliang Li, and Shaojie Shen. 2017. VINS-Mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics PP, 99 (2017), 1–17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Grompone Von Gioi Rafael, Jakubowicz Jĺȩrĺȩmie, Morel Jean-Michel, and Randall Gregory. 2010. LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis & Machine Intelligence 32, 4 (2010), 722–732. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Milad Ramezani and Kourosh Khoshelham. 2018. Vehicle positioning in GNSS-deprived urban areas by stereo visual-inertial odometry. IEEE Transactions on Intelligent Vehicles 3, 2 (2018), 208–217.Google ScholarGoogle ScholarCross RefCross Ref
  33. Jan Sthmer, Stefan Gumhold, and Daniel Cremers. 2010. Real-time dense geometry from a handheld camera. In Dagm Conference on Pattern Recognition. 11–20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. 2012. A benchmark for the evaluation of RGB-D SLAM systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. 573–580.Google ScholarGoogle ScholarCross RefCross Ref
  35. Fangchun Yang, Shangguang Wang, Jinglin Li, Zhihan Liu, and Qibo Sun. 2014. An overview of internet of vehicles. China Communications 11, 10 (2014), 1–15.Google ScholarGoogle ScholarCross RefCross Ref
  36. Shichao Yang and Sebastian Scherer. 2017. Direct monocular odometry using points and lines. In 2017 IEEE International Conference on Robotics and Automation. 3871–3877.Google ScholarGoogle ScholarCross RefCross Ref
  37. Hongsheng Yu and Anastasios I. Mourikis. 2017. Edge-based visual-inertial odometry. In 2017 IEEE/ RSJ International Conference on Intelligent Robots and Systems. 6670–6677.Google ScholarGoogle ScholarCross RefCross Ref
  38. Guoxuan Zhang, Han Lee Jin, Jongwoo Lim, and Il Hong Suh. 2015. Building a 3-D line-based map using stereo SLAM. IEEE Transactions on Robotics 31, 6 (2015), 1364–1377.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Xingxing Zuo, Xiaojia Xie, Yong Liu, and Guoquan Huang. 2017. Robust visual SLAM with point and line features. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. 1775–1782.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Internet Technology
          ACM Transactions on Internet Technology  Volume 22, Issue 1
          February 2022
          717 pages
          ISSN:1533-5399
          EISSN:1557-6051
          DOI:10.1145/3483347
          • Editor:
          • Ling Liu
          Issue’s Table of Contents

          Copyright © 2021 Association for Computing Machinery.

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 28 September 2021
          • Accepted: 1 October 2020
          • Revised: 1 September 2020
          • Received: 1 June 2020
          Published in toit Volume 22, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!