skip to main content
research-article

DyCo: Dynamic, Contextualized AI Models

Published:12 December 2022Publication History
Skip Abstract Section

Abstract

Devices with limited computing resources use smaller AI models to achieve low-latency inferencing. However, model accuracy is typically much lower than the accuracy of a bigger model that is trained and deployed in places where the computing resources are relatively abundant. We describe DyCo, a novel system that ensures privacy of stream data and dynamically improves the accuracy of small models used in devices. Unlike knowledge distillation or federated learning, DyCo treats AI models as black boxes. DyCo uses a semi-supervised approach to leverage existing training frameworks and network model architectures to periodically train contextualized, smaller models for resource-constrained devices. DyCo uses a bigger, highly accurate model in the edge-cloud to auto-label data received from each sensor stream. Training in the edge-cloud (as opposed to the public cloud) ensures data privacy, and bespoke models for thousands of live data streams can be designed in parallel by using multiple edge-clouds. DyCo uses the auto-labeled data to periodically re-train, stream-specific, bespoke small models. To reduce the periodic training costs, DyCo uses different policies that are based on stride, accuracy, and confidence information.

We evaluate our system, and the contextualized models, by using two object detection models for vehicles and people, and two datasets (a public benchmark and another real-world proprietary dataset). Our results show that DyCo increases the mAP accuracy measure of small models by an average of 16.3% (and up to 20%) for the public benchmark and an average of 19.0% (and up to 64.9%) for the real-world dataset. DyCo also decreases the training costs for contextualized models by more than an order of magnitude.

REFERENCES

  1. [1] Arasteh Hamidreza, Hosseinnezhad Vahid, Loia Vincenzo, Tommasetti Aurelio, Troisi Orlando, Shafie-khah Miadreza, and Siano Pierluigi. 2016. IoT-based smart cities: A survey. In IEEE 16th International Conference on Environment and Electrical Engineering (EEEIC). IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Barakabitze Alcardo Alex, Ahmad Arslan, Mijumbi Rashid, and Hines Andrew. 2020. 5G network slicing using SDN and NFV: A survey of taxonomy, architectures and future challenges. Comput. Netw. 167 (Feb.2020). DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Bonawitz Keith, Eichner Hubert, Grieskamp Wolfgang, Huba Dzmitry, Ingerman Alex, Ivanov Vladimir, Kiddon Chloe, Konečnỳ Jakub, Mazzocchi Stefano, McMahan H. Brendan, et al. 2019. Towards federated learning at scale: System design. arXiv preprint arXiv:1902.01046 (2019).Google ScholarGoogle Scholar
  4. [4] Bouyakoub Samia, Belkhir Abdelkader, Bouyakoub Fayçal M’hamed, and Guebli Wassila. 2017. Smart airport: An IoT-based airport management system. In International Conference on Future Networks and Distributed Systems. 17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Chapelle Olivier, Scholkopf Bernhard, and Zien Alexander. 2009. Semi-supervised learning. IEEE Trans. Neural Netw. 20, 3 (2009), 542542.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Chen Guobin, Choi Wongun, Yu Xiang, Han Tony, and Chandraker Manmohan. 2017. Learning efficient object detection models with knowledge distillation. In Conference on Advances in Neural Information Processing Systems. 742751.Google ScholarGoogle Scholar
  7. [7] Chuanqi. 2017. A caffe implementation of MobileNet-SSD detection network. Retrieved from https://github.com/chuanqi305/MobileNet-SSD.Google ScholarGoogle Scholar
  8. [8] Daga Harshit, Nicholson Patrick K., Gavrilovska Ada, and Lugones Diego. 2019. Cartel: A system for collaborative transfer learning at the edge. In ACM Symposium on Cloud Computing. 2537.Google ScholarGoogle Scholar
  9. [9] Dao Nhu-Ngoc, Na Woongsoo, and Cho Sungrae. 2020. Mobile cloudization storytelling: Current issues from optimization perspective. IEEE Internet Comput. PP (12020), 11. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Dendorfer Patrick, Rezatofighi Hamid, Milan Anton, Shi Javen, Cremers Daniel, Reid Ian, Roth Stefan, Schindler Konrad, and Leal-Taixé Laura. 2020. Mot20: A benchmark for multi object tracking in crowded scenes. arXiv preprint arXiv:2003.09003 (2020).Google ScholarGoogle Scholar
  11. [11] Fang Biyi, Zeng Xiao, Zhang Faen, Xu Hui, and Zhang Mi. 2020. FlexDNN: Input-adaptive on-device deep learning for efficient mobile vision. In 5th ACM/IEEE Symposium on Edge Computing (SEC).Google ScholarGoogle Scholar
  12. [12] Girshick Ross, Radosavovic Ilija, Gkioxari Georgia, Dollár Piotr, and He Kaiming. 2018. Detectron. Retrieved from https://github.com/facebookresearch/detectron.Google ScholarGoogle Scholar
  13. [13] Han Bing and Roy Kaushik. 2018. DeltaFrame-BP: An algorithm using frame difference for deep convolutional neural networks training and inference on video data. IEEE Trans. Multi-scale Comput. Syst. 4, 4 (2018), 624634.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Haowei Chen, Liekang Zeng, Shuai Yu, and Xu Chen. 2020. Knowledge distillation for mobile edge computation offloading. ZTE Commun. 18, 2 (2020), 4048.Google ScholarGoogle Scholar
  15. [15] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770778.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Hinton Geoffrey, Vinyals Oriol, and Dean Jeff. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).Google ScholarGoogle Scholar
  17. [17] Howard Andrew G., Zhu Menglong, Chen Bo, Kalenichenko Dmitry, Wang Weijun, Weyand Tobias, Andreetto Marco, and Adam Hartwig. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).Google ScholarGoogle Scholar
  18. [18] Hsieh Kevin, Harlap Aaron, Vijaykumar Nandita, Konomis Dimitris, Ganger Gregory R., Gibbons Phillip B., and Mutlu Onur. 2017. Gaia: Geo-distributed machine learning approaching \(\lbrace\)LAN\(\rbrace\) speeds. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI’17). 629647.Google ScholarGoogle Scholar
  19. [19] Kang Yiping, Hauswald Johann, Gao Cao, Rovinski Austin, Mudge Trevor, Mars Jason, and Tang Lingjia. 2017. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Comput. Archit. News 45, 1 (2017), 615629.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Li En, Zeng Liekang, Zhou Zhi, and Chen Xu. 2019. Edge AI: On-demand accelerating deep neural network inference via edge computing. IEEE Trans. Wirel. Commun. 19, 1 (2019), 447457.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Lin Tsung-Yi, Maire Michael, Belongie Serge, Bourdev Lubomir, Girshick Ross, Hays James, Perona Pietro, Ramanan Deva, Zitnick C. Lawrence, and Dollár Piotr. 2015. Microsoft COCO: Common Objects in Context. arXiv:cs.CV/1405.0312.Google ScholarGoogle Scholar
  22. [22] Liu Wei, Anguelov Dragomir, Erhan Dumitru, Szegedy Christian, Reed Scott, Fu Cheng-Yang, and Berg Alexander C.. 2016. SSD: Single shot multibox detector. In European Conference on Computer Vision. Springer, 2137.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Masaki Ichiro. 1998. Machine-vision systems for intelligent transportation systems. IEEE Intell. Syst. Applic. 13, 6 (1998), 2431.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Massa Francisco and Girshick Ross. 2018. maskrcnn-benchmark: Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch. Retrieved from https://github.com/facebookresearch/maskrcnn-benchmark. (2018).Google ScholarGoogle Scholar
  25. [25] Matsubara Yoshitomo, Baidya Sabur, Callegaro Davide, Levorato Marco, and Singh Sameer. 2019. Distilled split deep neural networks for edge-assisted real-time systems. In Workshop on Hot Topics in Video Analytics and Intelligent Edges. 2126.Google ScholarGoogle Scholar
  26. [26] Mullapudi Ravi Teja, Chen Steven, Zhang Keyi, Ramanan Deva, and Fatahalian Kayvon. 2019. Online model distillation for efficient video inference. In IEEE International Conference on Computer Vision. 35733582.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Noghabi Shadi A., Kolb John, Bodik Peter, and Cuervo Eduardo. 2018. Steel: Simplified development and deployment of edge-cloud applications. In 10th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud’18).Google ScholarGoogle Scholar
  28. [28] Pang Yanwei, Yuan Yuan, Li Xuelong, and Pan Jing. 2011. Efficient HOG human detection. Sig. Process. 91, 4 (2011), 773781.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Paszke Adam, Gross Sam, Massa Francisco, Lerer Adam, Bradbury James, Chanan Gregory, Killeen Trevor, Lin Zeming, Gimelshein Natalia, Antiga Luca, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Conference on Advances in Neural Information Processing Systems. 80248035.Google ScholarGoogle Scholar
  30. [30] Rafie Manouchehr. 2020. AI-Powered Camera Sensors. Retrieved from https://www.gyrfalcontech.ai/ai-powered-camera-sensors-whitepaper/. (2020).Google ScholarGoogle Scholar
  31. [31] Ren Shaoqing, He Kaiming, Girshick Ross, and Sun Jian. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Conference on Advances in Neural Information Processing Systems. 9199.Google ScholarGoogle Scholar
  32. [32] Sawhney Shreyak, Kacker Karan, Jain Samyak, Singh Shailendra Narayan, and Garg Rakesh. 2019. Real-time smart attendance system using face recognition techniques. In 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE, 522525.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Schroff Florian, Kalenichenko Dmitry, and Philbin James. 2015. FaceNet: A unified embedding for face recognition and clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 815823.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Suleiman Amr, Chen Yu-Hsin, Emer Joel, and Sze Vivienne. 2017. Towards closing the energy gap between HOG and CNN features for embedded vision. In IEEE International Symposium on Circuits and Systems (ISCAS). 14. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Sun Yunchuan, Zhang Junsheng, Xiong Yongping, and Zhu Guangyu. 2014. Data security and privacy in cloud computing. Int. J. Distrib. Sensor Netw. 10, 7 (2014), 190903.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Suo Hui, Liu Zhuohua, Wan Jiafu, and Zhou Keliang. 2013. Security and privacy in mobile cloud computing. In 9th International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 655659.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Tang Feng, Brennan Shane, Zhao Qi, and Tao Hai. 2007. Co-tracking using semi-supervised support vector machines. In IEEE 11th International Conference on Computer Vision. IEEE, 18.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Teerapittayanon Surat, McDanel Bradley, and Kung Hsiang-Tsung. 2017. Distributed deep neural networks over the cloud, the edge and end devices. In IEEE 37th International Conference on Distributed Computing Systems (ICDCS). IEEE, 328339.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Zhou Zhi-Hua. 2018. A brief introduction to weakly supervised learning. Nat. Sci. Rev. 5, 1 (2018), 4453.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Zhou Zhi-Hua and Li Ming. 2005. Semi-supervised regression with co-training. In International Joint Conferences on Artificial Intelligence, Vol. 5. 908913.Google ScholarGoogle Scholar
  41. [41] Zhu Xiaojin Jerry. 2005. Semi-supervised Learning Literature Survey. Technical Report. University of Wisconsin-Madison Department of Computer Sciences.Google ScholarGoogle Scholar

Index Terms

  1. DyCo: Dynamic, Contextualized AI Models

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Embedded Computing Systems
      ACM Transactions on Embedded Computing Systems  Volume 21, Issue 6
      November 2022
      498 pages
      ISSN:1539-9087
      EISSN:1558-3465
      DOI:10.1145/3561948
      • Editor:
      • Tulika Mitra
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 December 2022
      • Online AM: 26 March 2022
      • Accepted: 19 February 2022
      • Revised: 23 January 2022
      • Received: 15 July 2021
      Published in tecs Volume 21, Issue 6

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)107
      • Downloads (Last 6 weeks)6

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!