skip to main content
research-article

Towards Semantic Management of On-Device Applications in Industrial IoT

Published:14 November 2022Publication History
Skip Abstract Section

Abstract

The Internet of Things (IoT) is revolutionizing the industry. Powered by pervasive embedded devices, the Industrial IoT (IIoT) provides a unique solution for retrieving and analyzing data near the source in real-time. Many emerging techniques, such as Tiny Machine Learning (TinyML) and Complex Event Processing (CEP), are actively being developed to support decision making at the edge, shifting the paradigm from centralized processing to distributed computing. However, distributed computing presents management challenges, as IoT devices are diverse and constrained, and their number is growing exponentially. The situation is even more challenging when various on-device applications (so-called artifacts) are deployed across decentralized IoT networks. Questions to be addressed include how to discover an appropriate function, whether that function can be executed on a certain device, and how to orchestrate a cross-platform service. To tackle these challenges, we propose an approach for the scalable management of on-device applications among distributed IoT devices. By leveraging the W3C Web of Things (WoT), the capabilities of each IoT device, or more precisely, its interaction patterns, can be semantically expressed in a Thing Description (TD). In addition, we introduce semantic modeling of on-device applications to supplement an TD with additional information regarding applications on the device. Specifically, we demonstrate two examples of semantic modeling: neural networks (NN) and CEP rules. The ontologies are evaluated by answering a set of competency questions. By hosting the enriched semantic knowledge of the entire IoT system in a Knowledge Graph (KG), we can discover and interoperate edge devices and artifacts across the decentralized network. This can reduce fragmentation and increase the reusability of IoT components. We demonstrate the feasibility of our concept on an industrial workstation consisting of a conveyor belt and several IoT devices. Finally, the requirements for constructing an IoT semantic management system are discussed.

REFERENCES

  1. [1] Alaa Mussab, Zaidan Aws Alaa, Zaidan Bilal Bahaa, Talal Mohammed, and Kiah Miss Laiha Mat. 2017. A review of smart home applications based on Internet of Things. Journal of Network and Computer Applications 97 (2017), 4865. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Arduino. 2021. Arduino Nano 33 BLE Sense. https://store.arduino.cc/arduino-nano-33-ble-senseRetrieved May 04, 2021Google ScholarGoogle Scholar
  3. [3] Barbierato Luca, Estebsari Abouzar, Pons Enrico, Pau Marco, Salassa Fabio, Ghirardi Marco, and Patti Edoardo. 2019. A distributed IoT infrastructure to test and deploy real-time demand response in smart grids. IEEE Internet of Things Journal 6, 1 (2019), 11361146. Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Bauer Martin. 2021. IoT Virtualization with ML-based Information Extraction. arxiv:2106.06022. [cs.DC]Google ScholarGoogle Scholar
  5. [5] Bermudez-Edo Maria, Elsaleh Tarek, Barnaghi Payam, and Taylor Kerry. 2016. IoT-Lite: A lightweight semantic model for the Internet of Things. In 2016 INTL IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress. IEEE, IEEE, Toulouse, France, 9097. Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] IO GmbH Bosch. 2021. Bosch IoT Suite. https://developer.bosch-iot-suite.com/. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  7. [7] Charpenay Victor, Käbisch Sebastian, and Kosch Harald. 2016. Introducing thing descriptions and interactions: An ontology for the Web of Things. In 2016 International Semantic Web Conference. Springer, Kobe, Hyogo, Japan, 5566.Google ScholarGoogle Scholar
  8. [8] Choi Chang, Esposito Christian, Wang Haoxiang, Liu Zhe, and Choi Junho. 2018. Intelligent power equipment management based on distributed context-aware inference in smart cities. IEEE Communications Magazine 56, 7 (2018), 212217. Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Christensen Erik, Curbera Francisco, Meredith Greg, Weerawarana Sanjiva, et al. 2001. Web Services Description Language (WSDL) 1.1.Google ScholarGoogle Scholar
  10. [10] Compton Michael, Barnaghi Payam, Bermudez Luis, Garcia-Castro Raul, Corcho Oscar, Cox Simon, Graybeal John, Hauswirth Manfred, Henson Cory, Herzog Arthur, et al. 2012. The SSN ontology of the W3C semantic sensor network incubator group. Journal of Web Semantics 17 (2012), 2532. Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Daniele Laura, Hartog Frank den, and Roes Jasper. 2015. Created in close interaction with the industry: The smart appliances reference (SAREF) ontology. In International Workshop Formal Ontologies Meet Industries. Springer, Springer, Berlin, Germany, 100112. Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Almeida Vitor Pinheiro de, Bhowmik Sukanya, Lima Guilherme, Endler Markus, and Rothermel Kurt. 2020. DSCEP: An infrastructure for decentralized semantic complex event processing. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, Atlanta, USA, 391398. Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Dorofeev Kirill and Zoitl Alois. 2018. Skill-based engineering approach using OPC UA programs. In 2018 IEEE 16th International Conference on Industrial Informatics (INDIN). IEEE, Porto, Portugal, 10981103. Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Encinas Cesar, Ruiz Erica, Cortez Joaquin, and Espinoza Adolfo. 2017. Design and implementation of a distributed IoT system for the monitoring of water quality in aquaculture. In 2017 Wireless Telecommunications Symposium (WTS). IEEE, Chicago, IL, USA, 17. Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] al. Bert Moons et2018. BinarEye: An always-on energy-accuracy-scalable binary CNN processor with all memory on chip in 28nm CMOS. In 2018 IEEE Custom Integrated Circuits Conference (CICC). IEEE, IEEE, San Diego, CA, USA, 14. Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] al. Jinook Song et2019. 7.1 an 11.5 tops/w 1024-mac butterfly structure dual-core sparsity-aware neural processing unit in 8nm flagship mobile soc. In 2019 IEEE International Solid-State Circuits Conference (ISSCC). IEEE, IEEE, San Francisco, CA, USA, 130132. Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] al. Robert David et2020. Tensorflow Lite Micro: Embedded Machine Learning on tinyML Systems. arxiv:1801.06601. [cs.LG]Google ScholarGoogle Scholar
  18. [18] al. Tianqi Chen et2018. TVM: An automated end-to-end optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, USA, 578594.Google ScholarGoogle Scholar
  19. [19] Facebook Microsoft. 2021. ONNX: Open standard for machine learning interoperability. https://github.com/onnx/onnx. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  20. [20] Flores-Martin Daniel, Berrocal Javier, García-Alonso José, and Murillo Juan M.. 2020. Extending W3C thing description to provide support for interactions of things in real-time. In International Conference on Web Engineering. Springer, Springer, Bangkok, Thailand, 3041. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Foundation Eclipse. 2021. Eclipse IoT. https://iot.eclipse.org/. Retrieved August 09, 2021.Google ScholarGoogle Scholar
  22. [22] Foundation OpenJS. 2021. Node-RED. https://nodered.org/. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  23. [23] Foundation TinyML. 2021. tinyML.org. https://www.tinyml.org. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  24. [24] Fürst Jonathan, Argerich Mauricio Fadel, Cheng Bin, and Kovacs Ernö. 2020. Towards knowledge infusion for robust and transferable machine learning in IoT. Open Journal of Internet Of Things (OJIOT) 6, 1 (2020), 2434.Google ScholarGoogle Scholar
  25. [25] Georgios Lampropoulos, Kerstin Siakas, and Theofylaktos Anastasiadis. 2019. Internet of Things in the context of industry 4.0: An overview. International Journal of Entrepreneurial Knowledge 7, 1 (2019), 419. Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Gomes Porfírio, Cavalcante Everton, Batista Thais, Taconet Chantal, Conan Denis, Chabridon Sophie, Delicato Flavia C., and Pires Paulo F.. 2019. A semantic-based discovery service for the Internet of Things. Journal of Internet Services and Applications 10, 1 (2019), 114. Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Google. 2021. Google Cloud IoT. https://cloud.google.com/solutions/iot. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  28. [28] Google. 2021. TensorFlow Lite Micro Converter. https://www.tensorflow.org/lite/microcontrollers/build_convert. Retrieved August 04, 2021Google ScholarGoogle Scholar
  29. [29] al Pablo Graubne et. 2018. Multimodal complex event processing on mobile devices. In Proceedings of the 12th ACM International Conference on Distributed and Event-Based Systems (Hamilton, New Zealand) (DEBS ’18). Association for Computing Machinery, New York, NY, USA, 112123. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Guinard Dominique, Trifa Vlad, and Wilde Erik. 2010. A resource oriented architecture for the Web of Things. In 2010 Internet of Things (IOT). IEEE, IEEE, Tokyo, Japan, 18. Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Gyrard Amelie, Patel Pankesh, Datta S., and Ali M.. 2016. Semantic web meets Internet of Things (IoT) and Web of Things (WoT). In The 15th International Conference on Semantic Web (ISWC). sn, ACM, Perth, Australia, 0.Google ScholarGoogle Scholar
  32. [32] Haller Armin et al. 2019. The modular SSN ontology: A joint W3C and OGC standard specifying the semantics of sensors, observations, sampling, and actuation. Semantic Web 10, 1 (2019), 932.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Hassine Hassib Belhaj, Korkan Ege, and Steinhorst Sebastian. 2019. Virtual-Thing: Thing Description based Virtualization. arxiv:1909.03297. [cs.SE]Google ScholarGoogle Scholar
  34. [34] Himmelhuber Anna, Grimm Stephan, Runkler Thomas, and Zillner Sonja. 2020. Ontology-based skill description learning for flexible production systems. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. IEEE, IEEE, Vienna, Austria, 975981.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Hur Kangho, Jin Xiongnan, and Lee Kyong-Ho. 2015. Automated deployment of IoT services based on semantic description. In 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT). IEEE, IEEE, Milan, Italy, 4045. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Jabbar Sohail, Ullah Farhan, Khalid Shehzad, Khan Murad, and Han Kijun. 2017. Semantic interoperability in heterogeneous IoT infrastructure for healthcare. Wireless Communications and Mobile Computing 2017 (2017), 110.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Janowicz Krzysztof, Haller Armin, Cox Simon J. D., Phuoc Danh Le, and Lefrançois Maxime. 2019. SOSA: A lightweight ontology for sensors, observations, samples, and actuators. Journal of Web Semantics 56 (2019), 110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] al Krzysztof Janowicz et. 2019. SOSA: A lightweight ontology for sensors, observations, samples, and actuators. Journal of Web Semantics 56 (2019), 110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Kotha Harika Devi and Gupta V. Mnssvkr. 2018. IoT application: A survey. International Journal of Engineering Knowledge 7, 2.7 (2018), 891896. Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Kursuncu Ugur, Gaur Manas, and Sheth Amit. 2020. Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning. arxiv:1912.00512. [cs.AI]Google ScholarGoogle Scholar
  41. [41] Köcher Aljosha, Hildebrandt Constantin, Caesar Birte, Bakakeu Jupiter, Peschke Jörn, Scholz André, and Fay Alexander. 2020. Automating the development of machine skills and their semantic description. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. IEEE, Vienna, Austria, 10131018. Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] al Lina Lan et. 2019. A universal complex event processing mechanism based on edge computing for Internet of Things real-time monitoring. IEEE Access 7 (2019), 101865101878. Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Ma Meng, Wang Ping, and Chu Chao-Hsien. 2014. Ontology-based semantic modeling and evaluation for Internet of Things applications. In 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom). IEEE, Taipei, Taiwan, 2430. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Microsoft. 2021. Azure IoT. https://azure.microsoft.com/en-us/overview/iot/#overview. Retrieved May 04, 2021Google ScholarGoogle Scholar
  45. [45] Nguyen Anna, Weller Tobias, Färber Michael, and Sure-Vetter York. 2020. Making neural networks FAIR. In Knowledge Graphs and Semantic Web, Villazón-Terrazas Boris, Ortiz-Rodríguez Fernando, Tiwari Sanju M., and Shandilya Shishir K. (Eds.). Springer International Publishing, Cham, 2944. Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Noy Natalya F. et al. 2001. Ontology development 101: A guide to creating your first ontology.Google ScholarGoogle Scholar
  47. [47] Paganelli Federica, Turchi Stefano, and Giuli Dino. 2014. A Web of Things framework for restful applications and its experimentation in a smart city. IEEE Systems Journal 10, 4 (2014), 14121423. Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Petrović Nenad. 2019. Model-based approach for semantic-driven deployment of containerized applications to support future internet services and architectures. SERBIAN JOURNAL OF ELECTRICAL ENGINEERING 16, 1 (2019), 2144. Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] al Miguel Prado et. 2021. Robustifying the Deployment of tinyML Models for Autonomous Mini-Vehicles. arxiv:2007.00302. [cs.CV]Google ScholarGoogle Scholar
  50. [50] al Issam Rebai et. 2021. Semantic Smart Sensor Network (S3N) Ontology. https://www.w3.org/2019/wot/td. Retrieved May 04, 2021Google ScholarGoogle Scholar
  51. [51] Ren Haoyu, Anicic Darko, and Runkler Thomas. 2021. The Synergy of Complex Event Processing and Tiny Machine Learning in Industrial IoT. Accepted by The 15th ACM International Conference on Distributed and Event-based Systems (DEBS) 2021. arxiv:2105.03371 [cs.DC]Google ScholarGoogle Scholar
  52. [52] Ren Haoyu, Anicic Darko, and Runkler Thomas. 2021. TinyOL: TinyML with Online-Learning on Microcontrollers. Accepted by The International Joint Conference on Neural Network (IJCNN) 2021. arxiv:2103.08295. [cs.LG]Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Sagar Samya, Lefrançois Maxime, Rebaï Issam, Khemaja Maha, Garlatti Serge, Feki Jamel, and Médini Lionel. 2018. Modeling smart sensors on top of SOSA/SSN and WoT TD with the semantic smart sensor network (S3N) modular ontology. In ISWC 2018: 17th Internal Semantic Web Conference. SWSA, Monterey, USA, 163177.Google ScholarGoogle Scholar
  54. [54] Sanchez-Iborra Ramon and Skarmeta Antonio F.. 2020. TinyML-enabled frugal smart objects: Challenges and opportunities. IEEE Circuits and Systems Magazine 20, 3 (2020), 418. Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Services Amazon Web. 2021. AWS IoT Greengrass. https://aws.amazon.com/greengrass/. Retrieved May 04, 2021Google ScholarGoogle Scholar
  56. [56] Sinche Soraya, Raposo Duarte, Armando Ngombo, Rodrigues André, Boavida Fernando, Pereira Vasco, and Silva Jorge Sá. 2020. A survey of IoT management protocols and frameworks. IEEE Communications Surveys Tutorials 22, 2 (2020), 11681190. Google ScholarGoogle ScholarCross RefCross Ref
  57. [57] Singh Ravi Pratap, Javaid Mohd, Haleem Abid, and Suman Rajiv. 2020. Internet of Things (IoT) applications to fight against COVID-19 pandemic. Diabetes and Metabolic Syndrome: Clinical Research and Reviews 14, 4 (2020), 521524. Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] STMicroelectronics. 2021. AI expansion pack for STM32CubeMX. https://www.st.com/en/embedded-software/x-cube-ai.html. Retrieved Feb 27, 2021.Google ScholarGoogle Scholar
  59. [59] ThingsBoard. 2021. ThingsBoard. https://thingsboard.io/. Retrieved August 09, 2021.Google ScholarGoogle Scholar
  60. [60] Thuluva Aparna Saisree, Anicic Darko, Rudolph Sebastian, and Adikari Malintha. 2020. Semantic Node-RED for rapid development of interoperable industrial IoT applications. Semantic Web 11, 6 (2020), 949975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. [61] Topp Elin A., Stenmark Maj, Ganslandt Alexander, Svensson Andreas, Haage Mathias, and Malec Jacek. 2018. Ontology-based knowledge representation for increased skill reusability in industrial robots. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Madrid, Spain, 56725678. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] W3C. 2021. Thing Description Playground. http://plugfest.thingweb.io/playground/. Retrieved August 09, 2021.Google ScholarGoogle Scholar
  63. [63] W3C. 2021. Thing Description (TD) Ontology. https://www.w3.org/2019/wot/td. Retrieved May 04, 2021.Google ScholarGoogle Scholar
  64. [64] Wang Xiaofei, Han Yiwen, Leung Victor C. M., Niyato Dusit, Yan Xueqiang, and Chen Xu. 2020. Convergence of edge computing and deep learning: A comprehensive survey. IEEE Communications Surveys Tutorials 22, 2 (2020), 869904. Google ScholarGoogle ScholarCross RefCross Ref
  65. [65] Xiao Guangyi, Guo Jingzhi, Xu Li Da, and Gong Zhiguo. 2014. User interoperability with heterogeneous IoT devices through transformation. IEEE Transactions on Industrial Informatics 10, 2 (2014), 14861496. Google ScholarGoogle ScholarCross RefCross Ref
  66. [66] Yermalovich Pavel. 2020. Ontology-based model for security assessment: Predicting cyberattacks through threat activity analysis. International Journal of Network Security & Its Applications (IJNSA) Vol 12 (2020), 6178. Google ScholarGoogle ScholarCross RefCross Ref
  67. [67] Zantalis Fotios, Koulouras Grigorios, Karabetsos Sotiris, and Kandris Dionisis. 2019. A review of machine learning and IoT in smart transportation. Future Internet 11, 4 (2019), 94. Google ScholarGoogle ScholarCross RefCross Ref
  68. [68] Zeng Deze, Guo Song, and Cheng Zixue. 2011. The Web of Things: A survey. Journal of Communication 6, 6 (2011), 424438.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Towards Semantic Management of On-Device Applications in Industrial IoT

                Recommendations

                Comments

                Login options

                Check if you have access through your login credentials or your institution to get full access on this article.

                Sign in

                Full Access

                • Published in

                  cover image ACM Transactions on Internet Technology
                  ACM Transactions on Internet Technology  Volume 22, Issue 4
                  November 2022
                  642 pages
                  ISSN:1533-5399
                  EISSN:1557-6051
                  DOI:10.1145/3561988
                  Issue’s Table of Contents

                  Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

                  Publisher

                  Association for Computing Machinery

                  New York, NY, United States

                  Publication History

                  • Published: 14 November 2022
                  • Online AM: 3 February 2022
                  • Accepted: 6 January 2022
                  • Revised: 25 December 2021
                  • Received: 31 May 2021
                  Published in toit Volume 22, Issue 4

                  Permissions

                  Request permissions about this article.

                  Request Permissions

                  Check for updates

                  Qualifiers

                  • research-article
                  • Refereed

                PDF Format

                View or Download as a PDF file.

                PDF

                eReader

                View online with eReader.

                eReader

                Full Text

                View this article in Full Text.

                View Full Text

                HTML Format

                View this article in HTML Format .

                View HTML Format
                About Cookies On This Site

                We use cookies to ensure that we give you the best experience on our website.

                Learn more

                Got it!