Abstract
Service migration is an often-used approach in cloud computing to minimize the access cost by moving the service close to most users. Although it is effective in a certain sense, the service migration in existing research still suffers from some deficiencies in its evolutionary abilities in scalability, sensitivity, and adaptability to effectively react to the dynamically changing environments. This article proposes an evolutionary framework based on deep reinforcement learning for virtual service migration in large-scale mobile cloud centers. To enhance the spatio-temporal sensitivity of the algorithm, we design a scalable reward function for virtual service migration, redefine the input state, and add a Recurrent Neural Network (RNN) to the learning framework. Additionally, in order to enhance the adaptability of the algorithm, we also decompose the action space and exploit the network cost to adjust the number of virtual machine (VMs). The experimental results show that, compared with the existing results, the migration strategy generated by the algorithm can not only significantly reduce the total service cost and achieve the load balancing at the same time, but also address the burst situations with low cost in dynamic environments.
- Shan Cao, Yang Wang, and Chengzhong Xu. 2017. Service migrations in the cloud for mobile accesses: A reinforcement learning approach. In 2017 International Conference on Networking, Architecture, and Storage (NAS). 1--10.Google Scholar
Cross Ref
- Min Chen, Wei Li, Giancarlo Fortino, Yixue Hao, Long Hu, and Iztok Humar. 2018. A dynamic service-migration mechanism in edge cognitive computing.arXiv preprint arXiv:1808.07198 (2018).Google Scholar
- Zong-Gan Chen, Ke-Jing Du, Zhi-Hui Zhan, and Jun Zhang. 2015. Deadline constrained cloud computing resources scheduling for cost optimization based on dynamic objective genetic algorithm. In 2015 IEEE Congress on Evolutionary Computation (CEC). 708--714.Google Scholar
Cross Ref
- Martin Duggan, Jim Duggan, Enda Howley, and Enda Barrett. 2017. A network aware approach for the scheduling of virtual machine migration during peak loads. Cluster Computing 20, 3 (2017), 2083--2094.Google Scholar
Digital Library
- Song Fu and Cheng Zhong Xu. 2005. Service migration in distributed virtual machines for adaptive grid computing. In 2005 International Conference on Parallel Processing (ICPP’05). 358--365.Google Scholar
- Chaima Ghribi, Makhlouf Hadji, and Djamal Zeghlache. 2013. Energy efficient VM scheduling for cloud data centers: Exact allocation and migration algorithms. (2013), 671--678.Google Scholar
- Hadi Goudarzi and Massoud Pedram. 2012. Energy-efficient virtual machine replication and placement in a cloud computing system. In 2012 IEEE 5th International Conference on Cloud Computing. 750--757.Google Scholar
Digital Library
- Alexander Graves. 2012. Supervised sequence labelling with recurrent neural networks. 385 (2012), 1--131.Google Scholar
Cross Ref
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780.Google Scholar
Digital Library
- Jeffrey D. Johnson, Jinghong Li, and Zengshi Chen. 2000. Reinforcement learning: An introduction: R.S. Sutton, A.G. Barto, MIT Press, Cambridge, MA 1998, 322 pp. ISBN 0-262-19398-1. Neurocomputing 35, 1 (2000), 205--206.Google Scholar
- Kenji Kanai, Takeshi Muto, Jiro Katto, Shinya Yamamura, and Takura Sato. 2016. Proactive content caching for mobile video utilizing transportation systems and evaluation through field experiments. IEEE Journal on Selected Areas in Communications 34, 8 (2016), 1--1.Google Scholar
Digital Library
- Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations (2015).Google Scholar
- Timothy Paul Lillicrap, Jonathan James Hunt, Alexander Pritzel, Nicolas Manfred Otto Heess, Tom Erez, Yuval Tassa, David Silver, and Daniel Pieter Wierstra. 2015. Continuous control with deep reinforcement learning. Computer Science 8, 6 (2015), A187.Google Scholar
- Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Tim Harley, Timothy P. Lillicrap, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. International Conference on Machine Learning (2016), 1928--1937.Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529--533.Google Scholar
- Konstantinos Oikonomou and Ioannis Stavrakakis. 2010. Scalable service migration in autonomic network environments. IEEE Journal on Selected Areas in Communications 28, 1 (2010), 84--94.Google Scholar
Digital Library
- Jia Rao, Xiangping Bu, Cheng-Zhong Xu, Le Yi Wang, and Gang George Yin. 2009. VCONF: A reinforcement learning approach to virtual machines auto-configuration. In 6th International Conference on Autonomic Computing. 137--146.Google Scholar
Digital Library
- Ajay Thomas S. and Santhiya C. 2017. Dynamic resource scheduling using Delay time algorithm in Cloud environment. In 2017 2nd International Conference on Computing and Communications Technologies (ICCCT). 55--58.Google Scholar
- John Schulman, Sergey Levine, Pieter Abbeel, Michael I. Jordan, and Philipp Moritz. 2015. Trust region policy optimization. International Conference on Machine Learning (2015), 1889--1897.Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin A. Riedmiller. 2014. Deterministic policy gradient algorithms. In Proceedings of the 31st International Conference on Machine Learning. 387--395.Google Scholar
- Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12. 1057--1063.Google Scholar
- Rahul Urgaonkar, Shiqiang Wang, Ting He, Murtaza Zafer, Kevin S. Chan, and Kin K. Leung. 2015. Dynamic service migration and workload scheduling in edge-clouds. Performance Evaluation 91 (2015), 205--228.Google Scholar
Digital Library
- Paul J. Werbos. 1990. Backpropagation through time: What it does and how to do it. Proc. IEEE 78, 10 (1990), 1550--1560.Google Scholar
Cross Ref
- Cheng Zhong Xu, Jia Rao, and Xiangping Bu. 2012. URL: A unified reinforcement learning approach for autonomic cloud management. J. Parallel and Distrib. Comput. 72, 2 (2012), 95--105.Google Scholar
Digital Library
- Tianqi Zhao, Wei Zhang, Haiyan Zhao, and Zhi Jin. 2017. A reinforcement learning-based framework for the generation and evolution of adaptation rules. In 2017 IEEE International Conference on Autonomic Computing (ICAC). 103--112.Google Scholar
Cross Ref
Index Terms
SMig-RL: An Evolutionary Migration Framework for Cloud Services Based on Deep Reinforcement Learning
Recommendations
On Service Migrations in the Cloud for Mobile Accesses: A Distributed Approach
We study the problem of dynamically migrating a service in the cloud to satisfy an online sequence of mobile batch-request demands in a cost-effective way. The service may have single or multiple replicas, each running on a virtual machine. As the ...
Proposal and evaluation of deep exploitation-oriented learning under multiple reward environment
AbstractRecently, deep reinforcement learning (DRL) has attracted considerable attention. The well-known deep Q-network (DQN) architecture successfully combines deep learning and Q-learning which is a representative reinforcement learning (RL) ...
Reinforcement Learning Based Service Provisioning for a Greener Cloud
ICECCS '14: Proceedings of the 2014 3rd International Conference on Eco-friendly Computing and Communication SystemsCloud computing is an emerging distributed computing model consisting of massive data enters for making different services available to the users. In the current scenario where energy consumption and wastage in the IT field is looked upon with growing ...






Comments