Abstract
The Internet of Things (IoT) is revolutionizing the industry. Powered by pervasive embedded devices, the Industrial IoT (IIoT) provides a unique solution for retrieving and analyzing data near the source in real-time. Many emerging techniques, such as Tiny Machine Learning (TinyML) and Complex Event Processing (CEP), are actively being developed to support decision making at the edge, shifting the paradigm from centralized processing to distributed computing. However, distributed computing presents management challenges, as IoT devices are diverse and constrained, and their number is growing exponentially. The situation is even more challenging when various on-device applications (so-called artifacts) are deployed across decentralized IoT networks. Questions to be addressed include how to discover an appropriate function, whether that function can be executed on a certain device, and how to orchestrate a cross-platform service. To tackle these challenges, we propose an approach for the scalable management of on-device applications among distributed IoT devices. By leveraging the W3C Web of Things (WoT), the capabilities of each IoT device, or more precisely, its interaction patterns, can be semantically expressed in a Thing Description (TD). In addition, we introduce semantic modeling of on-device applications to supplement an TD with additional information regarding applications on the device. Specifically, we demonstrate two examples of semantic modeling: neural networks (NN) and CEP rules. The ontologies are evaluated by answering a set of competency questions. By hosting the enriched semantic knowledge of the entire IoT system in a Knowledge Graph (KG), we can discover and interoperate edge devices and artifacts across the decentralized network. This can reduce fragmentation and increase the reusability of IoT components. We demonstrate the feasibility of our concept on an industrial workstation consisting of a conveyor belt and several IoT devices. Finally, the requirements for constructing an IoT semantic management system are discussed.
- [1] . 2017. A review of smart home applications based on Internet of Things. Journal of Network and Computer Applications 97 (2017), 48–65. Google Scholar
Digital Library
- [2] . 2021. Arduino Nano 33 BLE Sense. https://store.arduino.cc/arduino-nano-33-ble-senseRetrieved May 04, 2021Google Scholar
- [3] . 2019. A distributed IoT infrastructure to test and deploy real-time demand response in smart grids. IEEE Internet of Things Journal 6, 1 (2019), 1136–1146. Google Scholar
Cross Ref
- [4] . 2021. IoT Virtualization with ML-based Information Extraction.
arxiv:2106.06022 . [cs.DC]Google Scholar - [5] . 2016. IoT-Lite: A lightweight semantic model for the Internet of Things. In 2016 INTL IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress. IEEE, IEEE, Toulouse, France, 90–97. Google Scholar
Cross Ref
- [6] . 2021. Bosch IoT Suite. https://developer.bosch-iot-suite.com/. Retrieved May 04, 2021.Google Scholar
- [7] . 2016. Introducing thing descriptions and interactions: An ontology for the Web of Things. In 2016 International Semantic Web Conference. Springer, Kobe, Hyogo, Japan, 55–66.Google Scholar
- [8] . 2018. Intelligent power equipment management based on distributed context-aware inference in smart cities. IEEE Communications Magazine 56, 7 (2018), 212–217. Google Scholar
Cross Ref
- [9] . 2001. Web Services Description Language (WSDL) 1.1.Google Scholar
- [10] . 2012. The SSN ontology of the W3C semantic sensor network incubator group. Journal of Web Semantics 17 (2012), 25–32. Google Scholar
Cross Ref
- [11] . 2015. Created in close interaction with the industry: The smart appliances reference (SAREF) ontology. In International Workshop Formal Ontologies Meet Industries. Springer, Springer, Berlin, Germany, 100–112. Google Scholar
Cross Ref
- [12] . 2020. DSCEP: An infrastructure for decentralized semantic complex event processing. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, Atlanta, USA, 391–398. Google Scholar
Cross Ref
- [13] . 2018. Skill-based engineering approach using OPC UA programs. In 2018 IEEE 16th International Conference on Industrial Informatics (INDIN). IEEE, Porto, Portugal, 1098–1103. Google Scholar
Cross Ref
- [14] . 2017. Design and implementation of a distributed IoT system for the monitoring of water quality in aquaculture. In 2017 Wireless Telecommunications Symposium (WTS). IEEE, Chicago, IL, USA, 1–7. Google Scholar
Cross Ref
- [15] 2018. BinarEye: An always-on energy-accuracy-scalable binary CNN processor with all memory on chip in 28nm CMOS. In 2018 IEEE Custom Integrated Circuits Conference (CICC). IEEE, IEEE, San Diego, CA, USA, 1–4. Google Scholar
Cross Ref
- [16] 2019. 7.1 an 11.5 tops/w 1024-mac butterfly structure dual-core sparsity-aware neural processing unit in 8nm flagship mobile soc. In 2019 IEEE International Solid-State Circuits Conference (ISSCC). IEEE, IEEE, San Francisco, CA, USA, 130–132. Google Scholar
Cross Ref
- [17] 2020. Tensorflow Lite Micro: Embedded Machine Learning on tinyML Systems.
arxiv:1801.06601 . [cs.LG]Google Scholar - [18] 2018. TVM: An automated end-to-end optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, USA, 578–594.Google Scholar
- [19] . 2021. ONNX: Open standard for machine learning interoperability. https://github.com/onnx/onnx. Retrieved May 04, 2021.Google Scholar
- [20] . 2020. Extending W3C thing description to provide support for interactions of things in real-time. In International Conference on Web Engineering. Springer, Springer, Bangkok, Thailand, 30–41. Google Scholar
Digital Library
- [21] . 2021. Eclipse IoT. https://iot.eclipse.org/. Retrieved August 09, 2021.Google Scholar
- [22] . 2021. Node-RED. https://nodered.org/. Retrieved May 04, 2021.Google Scholar
- [23] . 2021. tinyML.org. https://www.tinyml.org. Retrieved May 04, 2021.Google Scholar
- [24] . 2020. Towards knowledge infusion for robust and transferable machine learning in IoT. Open Journal of Internet Of Things (OJIOT) 6, 1 (2020), 24–34.Google Scholar
- [25] . 2019. Internet of Things in the context of industry 4.0: An overview. International Journal of Entrepreneurial Knowledge 7, 1 (2019), 4–19. Google Scholar
Cross Ref
- [26] . 2019. A semantic-based discovery service for the Internet of Things. Journal of Internet Services and Applications 10, 1 (2019), 1–14. Google Scholar
Cross Ref
- [27] . 2021. Google Cloud IoT. https://cloud.google.com/solutions/iot. Retrieved May 04, 2021.Google Scholar
- [28] . 2021. TensorFlow Lite Micro Converter. https://www.tensorflow.org/lite/microcontrollers/build_convert. Retrieved August 04, 2021Google Scholar
- [29] . 2018. Multimodal complex event processing on mobile devices. In Proceedings of the 12th ACM International Conference on Distributed and Event-Based Systems (Hamilton, New Zealand) (
DEBS ’18 ). Association for Computing Machinery, New York, NY, USA, 112–123. Google ScholarDigital Library
- [30] . 2010. A resource oriented architecture for the Web of Things. In 2010 Internet of Things (IOT). IEEE, IEEE, Tokyo, Japan, 1–8. Google Scholar
Cross Ref
- [31] . 2016. Semantic web meets Internet of Things (IoT) and Web of Things (WoT). In The 15th International Conference on Semantic Web (ISWC). sn, ACM, Perth, Australia, 0.Google Scholar
- [32] . 2019. The modular SSN ontology: A joint W3C and OGC standard specifying the semantics of sensors, observations, sampling, and actuation. Semantic Web 10, 1 (2019), 9–32.Google Scholar
Digital Library
- [33] . 2019. Virtual-Thing: Thing Description based Virtualization.
arxiv:1909.03297 . [cs.SE]Google Scholar - [34] . 2020. Ontology-based skill description learning for flexible production systems. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. IEEE, IEEE, Vienna, Austria, 975–981.Google Scholar
Cross Ref
- [35] . 2015. Automated deployment of IoT services based on semantic description. In 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT). IEEE, IEEE, Milan, Italy, 40–45. Google Scholar
Digital Library
- [36] . 2017. Semantic interoperability in heterogeneous IoT infrastructure for healthcare. Wireless Communications and Mobile Computing 2017 (2017), 1–10.Google Scholar
Digital Library
- [37] . 2019. SOSA: A lightweight ontology for sensors, observations, samples, and actuators. Journal of Web Semantics 56 (2019), 1–10. Google Scholar
Digital Library
- [38] . 2019. SOSA: A lightweight ontology for sensors, observations, samples, and actuators. Journal of Web Semantics 56 (2019), 1–10. Google Scholar
Digital Library
- [39] . 2018. IoT application: A survey. International Journal of Engineering Knowledge 7, 2.7 (2018), 891–896. Google Scholar
Cross Ref
- [40] . 2020. Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning.
arxiv:1912.00512 . [cs.AI]Google Scholar - [41] . 2020. Automating the development of machine skills and their semantic description. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. IEEE, Vienna, Austria, 1013–1018. Google Scholar
Cross Ref
- [42] . 2019. A universal complex event processing mechanism based on edge computing for Internet of Things real-time monitoring. IEEE Access 7 (2019), 101865–101878. Google Scholar
Cross Ref
- [43] . 2014. Ontology-based semantic modeling and evaluation for Internet of Things applications. In 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom). IEEE, Taipei, Taiwan, 24–30. Google Scholar
Digital Library
- [44] . 2021. Azure IoT. https://azure.microsoft.com/en-us/overview/iot/#overview. Retrieved May 04, 2021Google Scholar
- [45] . 2020. Making neural networks FAIR. In Knowledge Graphs and Semantic Web, , , , and (Eds.). Springer International Publishing, Cham, 29–44. Google Scholar
Cross Ref
- [46] . 2001. Ontology development 101: A guide to creating your first ontology.Google Scholar
- [47] . 2014. A Web of Things framework for restful applications and its experimentation in a smart city. IEEE Systems Journal 10, 4 (2014), 1412–1423. Google Scholar
Cross Ref
- [48] . 2019. Model-based approach for semantic-driven deployment of containerized applications to support future internet services and architectures. SERBIAN JOURNAL OF ELECTRICAL ENGINEERING 16, 1 (2019), 21–44. Google Scholar
Cross Ref
- [49] . 2021. Robustifying the Deployment of tinyML Models for Autonomous Mini-Vehicles.
arxiv:2007.00302 . [cs.CV]Google Scholar - [50] . 2021. Semantic Smart Sensor Network (S3N) Ontology. https://www.w3.org/2019/wot/td. Retrieved May 04, 2021Google Scholar
- [51] . 2021. The Synergy of Complex Event Processing and Tiny Machine Learning in Industrial IoT. Accepted by The 15th ACM International Conference on Distributed and Event-based Systems (DEBS) 2021.
arxiv:2105.03371 [cs.DC]Google Scholar - [52] . 2021. TinyOL: TinyML with Online-Learning on Microcontrollers. Accepted by The International Joint Conference on Neural Network (IJCNN) 2021.
arxiv:2103.08295 . [cs.LG]Google ScholarCross Ref
- [53] . 2018. Modeling smart sensors on top of SOSA/SSN and WoT TD with the semantic smart sensor network (S3N) modular ontology. In ISWC 2018: 17th Internal Semantic Web Conference. SWSA, Monterey, USA, 163–177.Google Scholar
- [54] . 2020. TinyML-enabled frugal smart objects: Challenges and opportunities. IEEE Circuits and Systems Magazine 20, 3 (2020), 4–18. Google Scholar
Cross Ref
- [55] . 2021. AWS IoT Greengrass. https://aws.amazon.com/greengrass/. Retrieved May 04, 2021Google Scholar
- [56] . 2020. A survey of IoT management protocols and frameworks. IEEE Communications Surveys Tutorials 22, 2 (2020), 1168–1190. Google Scholar
Cross Ref
- [57] . 2020. Internet of Things (IoT) applications to fight against COVID-19 pandemic. Diabetes and Metabolic Syndrome: Clinical Research and Reviews 14, 4 (2020), 521–524. Google Scholar
Cross Ref
- [58] . 2021. AI expansion pack for STM32CubeMX. https://www.st.com/en/embedded-software/x-cube-ai.html. Retrieved Feb 27, 2021.Google Scholar
- [59] . 2021. ThingsBoard. https://thingsboard.io/. Retrieved August 09, 2021.Google Scholar
- [60] . 2020. Semantic Node-RED for rapid development of interoperable industrial IoT applications. Semantic Web 11, 6 (2020), 949–975. Google Scholar
Digital Library
- [61] . 2018. Ontology-based knowledge representation for increased skill reusability in industrial robots. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Madrid, Spain, 5672–5678. Google Scholar
Digital Library
- [62] . 2021. Thing Description Playground. http://plugfest.thingweb.io/playground/. Retrieved August 09, 2021.Google Scholar
- [63] . 2021. Thing Description (TD) Ontology. https://www.w3.org/2019/wot/td. Retrieved May 04, 2021.Google Scholar
- [64] . 2020. Convergence of edge computing and deep learning: A comprehensive survey. IEEE Communications Surveys Tutorials 22, 2 (2020), 869–904. Google Scholar
Cross Ref
- [65] . 2014. User interoperability with heterogeneous IoT devices through transformation. IEEE Transactions on Industrial Informatics 10, 2 (2014), 1486–1496. Google Scholar
Cross Ref
- [66] . 2020. Ontology-based model for security assessment: Predicting cyberattacks through threat activity analysis. International Journal of Network Security & Its Applications (IJNSA) Vol 12 (2020), 61–78. Google Scholar
Cross Ref
- [67] . 2019. A review of machine learning and IoT in smart transportation. Future Internet 11, 4 (2019), 94. Google Scholar
Cross Ref
- [68] . 2011. The Web of Things: A survey. Journal of Communication 6, 6 (2011), 424–438.Google Scholar
Cross Ref
Index Terms
Towards Semantic Management of On-Device Applications in Industrial IoT
Recommendations
The synergy of complex event processing and tiny machine learning in industrial IoT
DEBS '21: Proceedings of the 15th ACM International Conference on Distributed and Event-based SystemsFocusing on comprehensive networking, the Industrial Internet-of-Things (IIoT) facilitates efficiency and robustness in factory operations. Various intelligent sensors play a central role, as they generate a vast amount of real-time data that can ...
Towards a Semantic Model for Automated Deployment of IoT Services across Platforms
SERVICES '15: Proceedings of the 2015 IEEE World Congress on ServicesIn the landscape of Web of Things (WoT), we encounter various WoT platforms. The WoT platforms can collect, manage and mash up a huge mass of things and their data by using Web technologies. Considering heterogeneous things deployed across different ...
Semantic modeling and design patterns for IoT ecosystems
AbstractIn recent years, there has been an increasing diffusion of IoT-based devices that collect and share sensor data for a wide variety of applications. These data are highly heterogeneous as they are obtained from various data sources ...






Comments