Abstract
Localization in urban environments is becoming increasingly important and used in tools such as ARCore [18], ARKit [34] and others. One popular mechanism to achieve accurate indoor localization and a map of the space is using Visual Simultaneous Localization and Mapping (Visual-SLAM). However, Visual-SLAM is known to be resource-intensive in memory and processing time. Furthermore, some of the operations grow in complexity over time, making it challenging to run on mobile devices continuously. Edge computing provides additional compute and memory resources to mobile devices to allow offloading tasks without the large latencies seen when offloading to the cloud.
In this article, we present Edge-SLAM, a system that uses edge computing resources to offload parts of Visual-SLAM. We use ORB-SLAM2 [50] as a prototypical Visual-SLAM system and modify it to a split architecture between the edge and the mobile device. We keep the tracking computation on the mobile device and move the rest of the computation, i.e., local mapping and loop closing, to the edge. We describe the design choices in this effort and implement them in our prototype. Our results show that our split architecture can allow the functioning of the Visual-SLAM system long-term with limited resources without affecting the accuracy of operation. It also keeps the computation and memory cost on the mobile device constant, which would allow for the deployment of other end applications that use Visual-SLAM. We perform a detailed performance and resources use (CPU, memory, network, and power) analysis to fully understand the effect of our proposed split architecture.
- [1] 2020. Computer Cpu Desktop—Free vector graphic on Pixabay. Retrieved April 1, 2020 from https://pixabay.com/images/id-156768/.Google Scholar
- [2] 2020. Interior Design Tv Multi-Screen—Free image on Pixabay. Retrieved April 1, 2020 from https://pixabay.com/images/id-828545/.Google Scholar
- [3] 2020. Smartphone Android Technology—Free vector graphic on Pixabay. Retrieved April 1, 2020 from https://pixabay.com/images/id-3358735/.Google Scholar
- [4] . 2019. WISDOM: WIreless sensing-assisted distributed online mapping. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA). 8026–8033.
DOI: Google ScholarDigital Library
- [5] . 2021. Serverless multi-query motion planning for fog robotics. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA). 7457–7463.
DOI: Google ScholarDigital Library
- [6] . 2006. Simultaneous localization and mapping (SLAM): Part II. IEEE Robotics Automation Magazine 13, 3 (
Sep. 2006), 108–117.DOI: Google ScholarCross Ref
- [7] . 2020. Edge-SLAM: Edge-assisted visual simultaneous localization and mapping. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services (MobiSys’20). Association for Computing Machinery, New York, NY, 325–337.
DOI: Google ScholarDigital Library
- [8] . 2021. Networking for cloud robotics: The DewROS platform and its application. Journal of Sensor and Actuator Networks 10, 2 (2021).
DOI: Google ScholarCross Ref
- [9] . 2019. SLAMBench 3.0: Systematic automated reproducible evaluation of SLAM systems for robot vision challenges and scene understanding. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA). 6351–6358.
DOI: Google ScholarDigital Library
- [10] . 2021. ORB-SLAM3: An accurate open-source library for visual, visual-inertial and multi-map SLAM. IEEE Transactions on Robotics 37, 6 (2021), 1874–1890.
DOI: Google ScholarCross Ref
- [11] . 2018. MARVEL: Enabling mobile augmented reality with low energy and low latency. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems (SenSys’18). ACM, New York, NY, 292–304.
DOI: Google ScholarDigital Library
- [12] . 2021. FogROS: An adaptive framework for automating fog robotics deployment. In Proceedings of the 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE). 2035–2042.
DOI: Google ScholarDigital Library
- [13] . 2021. Network offloading policies for cloud robotics: A learning-based approach. Autonomous Robots 45, 7 (2021), 997–1012.Google Scholar
Digital Library
- [14] . 2011. CloneCloud: Elastic execution between mobile device and cloud. In Proceedings of the 6th Conference on Computer Systems (EuroSys’11). ACM, New York, NY, 301–314.
DOI: Google ScholarDigital Library
- [15] . 2018. Data-efficient decentralized visual SLAM. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA’18). IEEE, 2466–2473.Google Scholar
Digital Library
- [16] . 2006. Using laser range data for 3D SLAM in outdoor environments. In Proceedings of the 2006 IEEE International Conference on Robotics and Automation, ICRA’06.1556–1563.
DOI: Google ScholarCross Ref
- [17] . 2010. MAUI: Making smartphones last longer with code offload. In Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services (MobiSys’10). ACM, New York, NY, 49–62.
DOI: Google ScholarDigital Library
- [18] . 2020. Build new augmented reality experiences that seamlessly blend the digital and physical worlds. Retrieved April 1, 2020 from https://developers.google.com/ar.Google Scholar
- [19] . 2001. A solution to the simultaneous localization and map building (SLAM) problem. IEEE Transactions on Robotics and Automation 17, 3 (
June 2001), 229–241.DOI: Google ScholarCross Ref
- [20] . 2006. Simultaneous localization and mapping: Part I. IEEE Robotics Automation Magazine 13, 2 (
June 2006), 99–110.DOI: Google ScholarCross Ref
- [21] . 2014. LSD-SLAM: Large-scale direct monocular SLAM. In Proceedings of the Computer Vision—ECCV 2014. , , , and (Eds.). Springer International Publishing, Cham, 834–849.Google Scholar
Cross Ref
- [22] . 2011. Real-time 3D visual SLAM with a hand-held RGB-D camera. In Proceedings of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, Vol. 180. 1–15.Google Scholar
- [23] . 2013. Relocating underwater features autonomously using sonar-based SLAM. IEEE Journal of Oceanic Engineering 38, 3 (
July 2013), 500–513.DOI: Google ScholarCross Ref
- [24] . 2007. WiFi-SLAM using gaussian process latent variable models. In Proceedings of the 20th International Joint Conference on Artifical Intelligence (IJCAI’07). Morgan Kaufmann Publishers Inc., San Francisco, CA, 2480–2485. Retrieved from http://dl.acm.org/citation.cfm?id=1625275.1625675.Google Scholar
Digital Library
- [25] . 2013. Collaborative monocular SLAM with multiple micro aerial vehicles. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. 3962–3970.
DOI: Google ScholarCross Ref
- [26] . 2020. Practical persistence reasoning in visual SLAM. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA). 7307–7313.
DOI: Google ScholarCross Ref
- [27] . 2018. Geometric mapping for sustained indoor autonomy. In Proceedings of the 1st International Workshop on Internet of People, Assistive Robots and Things (IoPARTS’18). Association for Computing Machinery, New York, NY, 19–24.
DOI: Google ScholarDigital Library
- [28] . 2019. Augmenting visual SLAM with Wi-Fi sensing for indoor applications. Autonomous Robots 43, 8 (2019), 2245–2260.Google Scholar
Digital Library
- [29] . 2017. Consistent cuboid detection for semantic mapping. In Proceedings of the 2017 IEEE 11th International Conference on Semantic Computing (ICSC’17). 526–531.
DOI: Google ScholarCross Ref
- [30] . 2016. Real-time loop closure in 2D LIDAR SLAM. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA’16). 1271–1278.
DOI: Google ScholarDigital Library
- [31] . 2011. Efficient, generalized indoor WiFi GraphSLAM. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation. 1038–1043.
DOI: Google ScholarCross Ref
- [32] . 2022. FogROS 2: An Adaptive and Extensible Platform for Cloud and Fog Robotics Using ROS 2. arXiv:2205.09778. Retrieved from https://arxiv.org/abs/2205.09778.Google Scholar
- [33] . 2020. Computer Vision Group—Dataset Download. Retrieved April 1, 2020 from https://vision.in.tum.de/data/datasets/rgbd-dataset/download.Google Scholar
- [34] 2020. Augmented Reality—Apple Developer. Retrieved April 1, 2020 from https://developer.apple.com/augmented-reality/.Google Scholar
- [35] . 2014. W-RGB-D: Floor-plan-based indoor global localization using a depth camera and WiFi. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA’14). 417–422.
DOI: Google ScholarCross Ref
- [36] . 2016. Low bandwidth offload for mobile AR. In Proceedings of the 12th International on Conference on Emerging Networking EXperiments and Technologies (CoNEXT’16). ACM, New York, NY, 237–251.
DOI: Google ScholarDigital Library
- [37] . 2007. Parallel tracking and mapping for small AR workspaces. In Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. 225–234.
DOI: Google ScholarDigital Library
- [38] . 2011. G2o: A general framework for graph optimization. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation. 3607–3613.
DOI: Google ScholarCross Ref
- [39] . 2013. Appearance-based loop closure detection for online large-scale and long-term operation. IEEE Transactions on Robotics 29, 3 (
June 2013), 734–745.DOI: Google ScholarDigital Library
- [40] . 2018. CORB-SLAM: A collaborative visual SLAM system for multiple robots. In Collaborative Computing: Networking, Applications and Worksharing. , , , , , and (Eds.). Springer International Publishing, Cham, 480–490.Google Scholar
- [41] . 2019. Edge assisted real-time object detection for mobile augmented reality. In Proceedings of the 25th Annual International Conference on Mobile Computing and Networking (MobiCom’19). ACM, New York, NY, Article
25 , 16 pages.DOI: Google ScholarDigital Library
- [42] 2021. Ubuntu Manpage: wondershaper—simple traffic shaping script. Retrieved from http://manpages.ubuntu.com/manpages/trusty/man8/wondershaper.8.html.Google Scholar
- [43] . 2017. Mobile edge computing: A survey on architecture and computation offloading. IEEE Communications Surveys Tutorials 19, 3 (
thirdquarter 2017), 1628–1656.DOI: Google ScholarDigital Library
- [44] . 2022. The poundcloud framework for ROS-based cloud robotics: Case studies on autonomous navigation and human-robot interaction. Robotics and Autonomous Systems 150 (2022), 103981.
DOI: Google ScholarDigital Library
- [45] . 2018. Visual slam for automated driving: Exploring the applications of deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 247–257.Google Scholar
Cross Ref
- [46] . 2013. SignalSLAM: Simultaneous localization and mapping with mixed WiFi, Bluetooth, LTE and magnetic signals. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation. 1–10.
DOI: Google ScholarCross Ref
- [47] . 2015. Rapyuta: A cloud robotics platform. IEEE Transactions on Automation Science and Engineering 12, 2 (2015), 481–493.
DOI: Google ScholarCross Ref
- [48] . 2015. Cloud-based collaborative 3d mapping in real-time with low-cost robots. IEEE Transactions on Automation Science and Engineering 12, 2 (2015), 423–431.
DOI: Google ScholarCross Ref
- [49] . 2015. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics 31, 5 (
Oct 2015), 1147–1163.DOI: Google ScholarDigital Library
- [50] . 2017. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Transactions on Robotics 33, 5 (
Oct 2017), 1255–1262.DOI: Google ScholarDigital Library
- [51] . 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality. 127–136.
DOI: Google ScholarDigital Library
- [52] . 2011. DTAM: Dense tracking and mapping in real-time. In Proceedings of the 2011 International Conference on Computer Vision. 2320–2327.
DOI: Google ScholarDigital Library
- [53] . 2017. PL-SLAM: Real-time monocular visual SLAM with points and lines. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA’17). IEEE, 4503–4508.Google Scholar
Digital Library
- [54] . 2018. VINS-Mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics 34, 4 (2018), 1004–1020.Google Scholar
Digital Library
- [55] . 2010. Sub-meter indoor localization in unmodified environments with inexpensive sensors. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2039–2046.
DOI: Google ScholarCross Ref
- [56] . 2014. C2TAM: A cloud framework for cooperative tracking and mapping. Robotics and Autonomous Systems 62, 4 (2014), 401–413.
DOI: Google ScholarDigital Library
- [57] . 2017. The emergence of edge computing. Computer 50, 1 (
Jan 2017), 30–39.DOI: Google ScholarDigital Library
- [58] . 2009. The case for VM-based cloudlets in mobile computing. IEEE Pervasive Computing 8, 4 (
Oct 2009), 14–23.DOI: Google ScholarDigital Library
- [59] . 2017. Multi-UAV collaborative monocular SLAM. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA’17). 3863–3870.
DOI: Google ScholarDigital Library
- [60] . 2016. Edge computing: Vision and challenges. IEEE Internet of Things Journal 3, 5 (
Oct 2016), 637–646.DOI: Google ScholarCross Ref
- [61] Sebastian Thrun. 2003. Robotic mapping: A survey. In Exploring Artificial Intelligence in the New Millennium. Morgan Kaufmann Publishers Inc., 1–35.Google Scholar
- [62] . 2020. Pseudo rgb-d for self-improving monocular slam and depth prediction. In Proceedings of the European Conference on Computer Vision. Springer, 437–455.Google Scholar
Digital Library
- [63] . 2018. Ultimate SLAM? Combining events, images, and IMU for robust visual SLAM in HDR and high-speed scenarios. IEEE Robotics and Automation Letters 3, 2 (2018), 994–1001.Google Scholar
Cross Ref
- [64] . 2013. Robust real-time visual odometry for dense RGB-D mapping. In Proceedings of the 2013 IEEE International Conference on Robotics and Automation. 5724–5731.
DOI: Google ScholarCross Ref
- [65] . 2020. CloudSLAM: Edge offloading of stateful vehicular applications. In Proceedings of the 5th ACM/IEEE Symposium on Edge Computing (SEC’20).Google Scholar
Cross Ref
- [66] . 2020. Edge assisted mobile semantic visual SLAM. In Proceedings of the IEEE INFOCOM 2020—IEEE Conference on Computer Communications. 1828–1837.
DOI: Google ScholarDigital Library
- [67] . 2018. DS-SLAM: A semantic visual SLAM towards dynamic environments. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’18). IEEE, 1168–1174.Google Scholar
Digital Library
- [68] . 2014. LOAM: Lidar odometry and mapping in real-time. In Proceedings of the Robotics: Science and Systems, Vol. 2. Berkeley, 1–9.Google Scholar
Cross Ref
- [69] . 2017. Robust visual SLAM with point and line features. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’17). IEEE, 1775–1782.Google Scholar
Digital Library
Index Terms
(auto-classified)Edge-SLAM: Edge-Assisted Visual Simultaneous Localization and Mapping
Recommendations
Edge-SLAM: edge-assisted visual simultaneous localization and mapping
MobiSys '20: Proceedings of the 18th International Conference on Mobile Systems, Applications, and ServicesLocalization in urban environments is becoming increasingly important and used in tools such as ARCore [11], ARKit [27] and others. One popular mechanism to achieve accurate indoor localization as well as a map of the space is using Visual Simultaneous ...
Edge-SLAM: edge-assisted visual simultaneous localization and mapping
MobiCom '20: Proceedings of the 26th Annual International Conference on Mobile Computing and NetworkingThe recent advances in mobile devices have allowed them to run spatial sensing algorithms such as Visual Simultaneous Localization and Mapping (Visual-SLAM). However, the resource requirements of Visual-SLAM prevents long-operation of such algorithm on ...






Comments