Abstract
Mobile crowdsensing becomes a promising technology for the emerging Internet of Things (IoT) applications in smart environments. Fog computing is enabling a new breed of IoT services, which is also a new opportunity for mobile crowdsensing. Thus, in this article, we introduce a framework enabling mobile crowdsensing in fog environments with a hierarchical scheduling strategy. We first introduce the crowdsensing framework that has a hierarchical structure to organize different resources. Since different positions and performance of fog nodes influence the quality of service (QoS) of IoT applications, we formulate a scheduling problem in the hierarchical fog structure and solve it by using a deep reinforcement learning–based strategy. From extensive simulation results, our solution outperforms other scheduling solutions for mobile crowdsensing in the given fog computing environment.
- Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, and Amin Vahdat. 2010. Hedera: Dynamic flow scheduling for data center networks. In Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation (NSDI’10). USENIX Association, 19--19. http://dl.acm.org/citation.cfm?id=1855711.1855730. Google Scholar
Digital Library
- M. A. Alsheikh, D. Niyato, S. Lin, H. P. Tan, and Z. Han. 2016. Mobile big data analytics using deep learning and apache spark. IEEE Network 30, 3 (May 2016), 22--29.Google Scholar
Cross Ref
- S. Arai, K. Sycara, and T. R. Payne. 2000. Multi-agent reinforcement learning for planning and scheduling multiple goals. In Proceedings of the 4th International Conference on MultiAgent Systems. 359--360. Google Scholar
Digital Library
- R. Atallah, C. Assi, and M. Khabbaz. 2017. Deep reinforcement learning-based scheduling for roadside communication networks. In Proceedings of the 2017 15th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt’17). 1--8.Google Scholar
- S. Basudan, X. Lin, and K. Sankaranarayanan. 2017. A privacy-preserving vehicular crowdsensing-based road surface condition monitoring system using fog computing. IEEE Internet of Things Journal 4, 3 (June 2017), 772--782.Google Scholar
Cross Ref
- Richard Bellman. 2013. Dynamic Programming. Courier Corporation.Google Scholar
- K. Bessai, S. Youcef, A. Oulamara, C. Godart, and S. Nurcan. 2012. Bi-criteria workflow tasks allocation and scheduling in cloud computing environments. In Proceedings of the 2012 IEEE 5th International Conference on Cloud Computing. 638--645. Google Scholar
Digital Library
- L. F. Bittencourt, J. Diaz-Montes, R. Buyya, O. F. Rana, and M. Parashar. 2017. Mobility-aware application scheduling in fog computing. IEEE Cloud Computing 4, 2 (March 2017), 26--35.Google Scholar
Cross Ref
- Flavio Bonomi, Rodolfo Milito, Preethi Natarajan, and Jiang Zhu. 2014. Fog Computing: A Platform for Internet of Things and Analytics. Springer International Publishing, Cham, 169--186.Google Scholar
- Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. 2012. Fog computing and its role in the internet of things. In Proceedings of the 1st Edition of the MCC Workshop on Mobile Cloud Computing (MCC’12). ACM, New York, 13--16. Google Scholar
Digital Library
- D. Bruneo, S. Distefano, F. Longo, G. Merlino, A. Puliafito, V. D’Amico, M. Sapienza, and G. Torrisi. 2016. Stack4Things as a fog computing platform for smart city applications. In Proceedings of the 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). 848--853.Google Scholar
- Shin-Ming Cheng, Phone Lin, Di-Wei Huang, and Shun-Ren Yang. 2006. A study on distributed/centralized scheduling for wireless mesh network. In Proceedings of the 2006 International Conference on Wireless Communications and Mobile Computing (IWCMC’06). ACM, New York, 599--604. Google Scholar
Digital Library
- Yohan Chon, Nicholas D. Lane, Fan Li, Hojung Cha, and Feng Zhao. 2012. Automatically characterizing places with opportunistic crowdsensing using smartphones. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp’12). ACM, New York, 481--490. Google Scholar
Digital Library
- Salvatore Distefano, Francesco Longo, and Marco Scarpa. 2015. QoS assessment of mobile crowdsensing services. Journal of Grid Computing 13, 4 (Dec. 2015), 629--650. Google Scholar
Digital Library
- Nathan Eagle and Alex (Sandy) Pentland. 2005. CRAWDAD dataset mit/reality (v. 2005-07-01). Downloaded from https://crawdad.org/mit/reality/20050701.Google Scholar
- Eyal Even-Dar and Yishay Mansour. 2001. Learning rates for Q-Learning. In Computational Learning Theory, David Helmbold and Bob Williamson (Eds.). Springer, Berlin, 589--604. Google Scholar
Digital Library
- R. K. Ganti, F. Ye, and H. Lei. 2011. Mobile crowdsensing: Current state and future challenges. IEEE Communications Magazine 49, 11 (Nov. 2011), 32--39.Google Scholar
Cross Ref
- Robert Glaubius, Terry Tidwell, Christopher Gill, and William D. Smart. 2010. Real-time scheduling via reinforcement learning. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI’ 10). AUAI Press, Arlington, Virginia, 201--209. http://dl.acm.org/citation.cfm?id=3023549.3023573 Google Scholar
Digital Library
- D. Hoang and T. D. Dang. 2017. FBRC: Optimization of task scheduling in fog-based region and cloud. In 2017 IEEE Trustcom/BigDataSE/ICESS. 1109--1114.Google Scholar
- Kirak Hong, David Lillethun, Umakishore Ramachandran, Beate Ottenwälder, and Boris Koldehofe. 2013. Mobile fog: A programming model for large-scale applications on the internet of things. In Proceedings of the S2nd ACM SIGCOMM Workshop on Mobile Cloud Computing (MCC’13). ACM, New York, 15--20. Google Scholar
Digital Library
- H. Hu, Y. Wen, T. S. Chua, J. Huang, W. Zhu, and X. Li. 2016. Joint content replication and request routing for social video distribution over cloud CDN: A community clustering method. IEEE Transactions on Circuits and Systems for Video Technology 26, 7 (July 2016), 1320--1333.Google Scholar
Cross Ref
- H. Hu, Y. Wen, and D. Niyato. 2017. Public cloud storage-assisted mobile social video sharing: A supermodular game approach. IEEE Journal on Selected Areas in Communications 35, 3 (March 2017), 545--556. Google Scholar
Digital Library
- H. Hu, Y. Wen, and D. Niyato. 2017. Spectrum allocation and bitrate adjustment for mobile social video sharing: Potential game with online QoS learning approach. IEEE Journal on Selected Areas in Communications 35, 4 (April 2017), 935--948.Google Scholar
Cross Ref
- S. Kosta, A. Aucinas, Pan Hui, R. Mortier, and Xinwen Zhang. 2012. ThinkAir: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading. In 2012 Proceedings IEEE INFOCOM. 945--953.Google Scholar
Cross Ref
- H. Li, K. Ota, and M. Dong. 2018. Learning IoT in edge: Deep learning for the internet of things with edge computing. IEEE Network 32, 1 (Jan. 2018), 96--101.Google Scholar
Cross Ref
- H. Li, K. Ota, M. Dong, and M. Guo. 2017. Mobile crowdsensing in software defined opportunistic networks. IEEE Communications Magazine 55, 6 (2017), 140--145.Google Scholar
Cross Ref
- H. Li, K. Ota, M. Dong, A. Vasilakos, and K. Nagano. 2017. Multimedia processing pricing strategy in GPU-accelerated cloud computing. IEEE Transactions on Cloud Computing PP, 99 (2017), 1--1.Google Scholar
- L. Li, K. Ota, M. Dong, and W. Borjigin. 2017. Eyes in the dark: Distributed scene understanding for disaster management. IEEE Transactions on Parallel and Distributed Systems 28, 12 (Dec. 2017), 3458--3471.Google Scholar
Cross Ref
- T. Li, Y. Liu, L. Gao, and A. Liu. 2017. A cooperative-based model for smart-sensing tasks in fog computing. IEEE Access 5 (2017), 21296--21311.Google Scholar
Cross Ref
- Yuanyuan Li, Zhiyang Li, Mianxiong Dong, Wenyu Qu, Changqing Ji, and Junfeng Wu. 2015. Efficient subspace skyline query based on user preference using MapReduce. Ad Hoc Networks 35, Supplement C (2015), 105--115. Special Issue on Big Data Inspired Data Sensing, Processing and Networking Technologies. Google Scholar
Digital Library
- Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. 2016. Resource management with deep reinforcement learning. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (HotNets’16). ACM, New York, 50--56. Google Scholar
Digital Library
- Giovanni Merlino, Stamatis Arkoulis, Salvatore Distefano, Chrysa Papagianni, Antonio Puliafito, and Symeon Papavassiliou. 2016. Mobile crowdsensing as a service: A platform for applications on top of sensing clouds. Future Generation Computer Systems 56, Supplement C (2016), 623--639. Google Scholar
Digital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. 2013. Playing atari with deep reinforcement learning. CoRR abs/1312.5602 (2013). arxiv:1312.5602 http://arxiv.org/abs/1312.5602Google Scholar
- M. Mohammadi, A. Al-Fuqaha, M. Guizani, and J. S. Oh. 2018. Semisupervised deep reinforcement learning in support of IoT and smart city services. IEEE Internet of Things Journal 5, 2 (April 2018), 624--635.Google Scholar
Cross Ref
- Xuan-Qui Pham and Eui-Nam Huh. 2016. Towards task scheduling in a cloud-fog computing system. In Proceedings of the 2016 18th Asia-Pacific Network Operations and Management Symposium (APNOMS’16). 1--4.Google Scholar
Cross Ref
- Mahadev Satyanarayanan. 2010. Mobile computing: The next decade. In Proceedings of the 1st ACM Workshop on Mobile Cloud Computing 8 Services: Social Networks and Beyond (MCS’10). ACM, New York, Article 5, 6 pages. Google Scholar
Digital Library
- M. Satyanarayanan, P. Simoens, Y. Xiao, P. Pillai, Z. Chen, K. Ha, W. Hu, and B. Amos. 2015. Edge analytics in the internet of things. IEEE Pervasive Computing 14, 2 (Apr. 2015), 24--31.Google Scholar
Digital Library
- S. K. Sharma and X. Wang. 2017. Live data analytics with collaborative edge and cloud processing in wireless IoT networks. IEEE Access 5 (2017), 4621--4635.Google Scholar
Cross Ref
- W. Sherchan, P. P. Jayaraman, S. Krishnaswamy, A. Zaslavsky, S. Loke, and A. Sinha. 2012. Using on-the-move mining for mobile crowdsensing. In Proceedings of the 2012 IEEE 13th International Conference on Mobile Data Management. 115--124. Google Scholar
Digital Library
- David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of Go without human knowledge. Nature 550 (Oct. 2017), 354.Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). arxiv:1409.1556 http://arxiv.org/abs/1409.1556Google Scholar
- Karolj Skala, Davor Davidovic, Enis Afgan, Ivan Sovic, and Zorislav Sojat. 2015. Scalable distributed computing hierarchy: Cloud, fog and dew computing. Open Journal of Cloud Computing (OJCC) 2, 1 (2015), 16--24.Google Scholar
- M. Wang, Y. Cui, X. Wang, S. Xiao, and J. Jiang. 2018. Machine learning for networking: Workflow, advances and opportunities. IEEE Network 32, 2 (March 2018), 92--99.Google Scholar
- Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. Machine Learning 8, 3 (May 1992), 279--292. Google Scholar
Digital Library
- L. Xiao, Y. Li, G. Han, H. Dai, and H. V. Poor. 2018. A secure mobile crowdsensing game with deep reinforcement learning. IEEE Transactions on Information Forensics and Security 13, 1 (Jan. 2018), 35--47.Google Scholar
Cross Ref
- L. Xiao, X. Wan, C. Dai, X. Du, X. Chen, and M. Guizani. 2018. Security in mobile edge caching with reinforcement learning. ArXiv e-prints (Jan. 2018). arxiv:cs.CR/1801.05915Google Scholar
- Yu Xiao, Pieter Simoens, Padmanabhan Pillai, Kiryong Ha, and Mahadev Satyanarayanan. 2013. Lowering the barriers to large-scale mobile crowdsensing. In Proceedings of the 14th Workshop on Mobile Computing Systems and Applications (HotMobile’13). ACM, New York, Article 9, 6 pages. Google Scholar
Digital Library
- Shanhe Yi, Cheng Li, and Qun Li. 2015. A survey of fog computing: Concepts, applications and issues. In Proceedings of the 2015 Workshop on Mobile Big Data (Mobidata’15). ACM, New York, 37--42. Google Scholar
Digital Library
- Wei Zhang and Thomas G. Dietterich. 1995. A reinforcement learning approach to job-shop scheduling. In Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 2 (IJCAI’95). Morgan Kaufmann Publishers Inc., San Francisco, CA, 1114--1120. http://dl.acm.org/citation.cfm?id=1643031.1643044 Google Scholar
Digital Library
- D. Zhao, X. Y. Li, and H. Ma. 2014. How to crowdsource tasks truthfully without sacrificing utility: Online incentive mechanisms with budget constraint. In Proceedings of the IEEE Conference on Computer Communications (IEEE INFOCOM’14). 1213--1221.Google Scholar
- Z. Zheng, L. X. Cai, M. Dong, X. Shen, and H. V. Poor. 2011. Constrained energy-aware AP placement with rate adaptation in WLAN mesh networks. In Proceedings of the 2011 IEEE Global Telecommunications Conference (GLOBECOM’11). 1--5.Google Scholar
Index Terms
Deep Reinforcement Scheduling for Mobile Crowdsensing in Fog Computing
Recommendations
Fog Computing: Issues and Challenges in Security and Forensics
COMPSAC '15: Proceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 03Although Fog Computing is defined as the extension of the Cloud Computing paradigm, its distinctive characteristics in the location sensitivity, wireless connectivity, and geographical accessibility create new security and forensics issues and ...
From Cloud Computing to Fog Computing: Platforms for the Internet of Things (IoT)
This article describes how in recent years, Cloud Computing has emerged as a fundamental computing paradigm that has significantly changed the approach of enterprises as well as end users towards implementation of Internet technology. The key ...
Resource Allocation for Blockchain-Enabled Fog Computing with Deep Reinforcement Learning
ICCNS '22: Proceedings of the 2022 12th International Conference on Communication and Network SecurityWith the developments of the Internet of Things, the demands of low latency, high bandwidth and high-performance computing has increased higher. Therefore, the distributed computing named Fog Computing has proposed to solve the problem above. Fog ...






Comments