skip to main content
research-article

Sequential Learning-based IaaS Composition

Published:14 July 2021Publication History
Skip Abstract Section

Abstract

We propose a novel Infrastructure-as-a-Service composition framework that selects an optimal set of consumer requests according to the provider’s qualitative preferences on long-term service provisions. Decision variables are included in the temporal conditional preference networks to represent qualitative preferences for both short-term and long-term consumers. The global preference ranking of a set of requests is computed using a k-d tree indexing-based temporal similarity measure approach. We propose an extended three-dimensional Q-learning approach to maximize the global preference ranking. We design the on-policy-based sequential selection learning approach that applies the length of request to accept or reject requests in a composition. The proposed on-policy-based learning method reuses historical experiences or policies of sequential optimization using an agglomerative clustering approach. Experimental results prove the feasibility of the proposed framework.

References

  1. Pegah Alizadeh, Aomar Osmani, Mohamed Essaid Khanouche, Abdelghani Chibani, and Yacine Amirat. 2021. Reinforcement learning for interactive QoS-aware services composition. IEEE Syst. J. 15, 1 (2021), 1098–1108. DOI:10.1109/JSYST.2020.2997069Google ScholarGoogle ScholarCross RefCross Ref
  2. Mohammad Alrifai and Thomas Risse. 2009. Combining global optimization with local selection for efficient QoS-aware service composition. In Proceedings of WWW. ACM, 881–890. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Alexandr Andoni and Piotr Indyk. 2006. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In Proceedings of FOCS. IEEE, 459–468. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Pavel Berkhin. 2006. A survey of clustering data mining techniques. In Grouping Multidimensional Data. Springer, 25–71.Google ScholarGoogle Scholar
  5. Marko Bohanec, Antoine Messean, Sara Scatasta, Frederique Angevin, Bryan Griffiths, Paul Henning Krogh, Martin Žnidaršič, and Sašo Džeroski. 2008. A qualitative multi-attribute model for economic and ecological assessment of genetically modified crops. Ecol. Model. 215, 1 (2008), 247–261.Google ScholarGoogle ScholarCross RefCross Ref
  6. Athman Bouguettaya. 1996. On-line clustering. IEEE Trans. Knowl. Data Eng. 8, 2 (1996), 333–339. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Athman Bouguettaya, Qi Yu, Xumin Liu, Xiangmin Zhou, and Andy Song. 2015. Efficient agglomerative hierarchical clustering. Expert Syst. Appl. 42, 5 (2015), 2785–2797. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Craig Boutilier, Ronen I. Brafman, Carmel Domshlak, Holger H. Hoos, and David Poole. 2004. CP-nets: A tool for representing and reasoning with conditional ceteris paribus preference statements. J. Artific. Intell. Res. 21 (2004), 135–191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Sivadon Chaisiri, Bu-Sung Lee, and Dusit Niyato. 2012. Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5, 2 (2012), 164–177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Chaisiri, D. Niyato, and B. Lee. 2014. Capacity planning for data center to support green computing. In Proceedings of JCSSE. 152–157.Google ScholarGoogle Scholar
  11. Jaihak Chung and Vithala R. Rao. 2012. A general consumer preference model for experience products: Application to internet recommendation services. J. Market. Res. 49, 3 (2012), 289–305.Google ScholarGoogle ScholarCross RefCross Ref
  12. Peijin Cong, Guo Xu, Tongquan Wei, and Keqin Li. 2020. A survey of profit optimization techniques for cloud providers. ACM Comput. Surveys 53, 2 (2020), 1–35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cristina Cornelio, Judy Goldsmith, Nicholas Mattei, Francesca Rossi, and K. Brent Venable. 2013. Updates and uncertainty in CP-nets. In Proceedings of AI. Springer, 301–312. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Debabrata Dash, Verena Kantere, and Anastasia Ailamaki. 2009. An economic model for self-tuned cloud caching. In Proceedings of ICDE. IEEE, 1687–1693. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Marco Dorigo and L. M. Gambardella. 2016. Ant-Q: A reinforcement learning approach to the traveling salesman problem. In Proceedings of ICML. 252–260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Schahram Dustdar, Yike Guo, Benjamin Satzger, and Hong-Linh Truong. 2011. Principles of elastic processes. IEEE Internet Comput. 15, 5 (2011), 66–71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Sheik Mohammad Mostakim Fattah and Athman Bouguettaya. 2020. Event-based detection of changes in IaaS performance signatures. In Proceedings of SCC. IEEE, 210–217.Google ScholarGoogle Scholar
  18. Sheik Mohammad Mostakim Fattah, Athman Bouguettaya, and Sajib Mistry. 2018. A CP-net based qualitative composition approach for an IaaS provider. In Proceedings of WISE. Springer, 151–166.Google ScholarGoogle Scholar
  19. Sheik Mohammad Mostakim Fattah, Athman Bouguettaya, and Sajib Mistry. 2019. Long-term IaaS provider selection using short-term trial experience. In Proceedings of ICWS. IEEE, 304–311.Google ScholarGoogle Scholar
  20. Sheik Mohammad Mostakim Fattah, Athman Bouguettaya, and Sajib Mistry. 2020. Long-term IaaS selection using performance discovery. IEEE Trans. Services Comput. (2020). DOI:10.1109/TSC.2020.3036677Google ScholarGoogle Scholar
  21. Sheik Mohammad Mostakim Fattah, Athman Bouguettaya, and Sajib Mistry. 2020. Signature-based selection of IaaS cloud services. In Proceedings of ICWS. IEEE, 50–57.Google ScholarGoogle Scholar
  22. Alberto Fernández and Sergio Gómez. 2008. Solving non-uniqueness in agglomerative hierarchical clustering using multidendrograms. J. Classif. 25, 1 (2008), 43–65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Fernando Fernández and Manuela Veloso. 2006. Probabilistic policy reuse in a reinforcement learning agent. In Proceedings of AAMAS. ACM, 720–727. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Siva Kumar Gavvala, Chandrashekar Jatoth, G. R. Gangadharan, and Rajkumar Buyya. 2019. QoS-aware cloud service composition using eagle strategy. Future Gen. Comput. Syst. 90 (2019), 273–290.Google ScholarGoogle ScholarCross RefCross Ref
  25. Adelina Gnanlet and Wendell G. Gilland. 2009. Sequential and simultaneous decision making for optimizing health care resource flexibilities. Decis. Sci. 40, 2 (2009), 295–326.Google ScholarGoogle ScholarCross RefCross Ref
  26. Íñigo Goiri, Jordi Guitart, and Jordi Torres. 2012. Economic model of a Cloud provider operating in a federated Cloud. Info. Syst. Front. 14 (2012), 827–843.Google ScholarGoogle ScholarCross RefCross Ref
  27. Anna Großwendt and Heiko Röglin. 2017. Improved analysis of complete-linkage clustering. Algorithmica 78, 4 (1 Aug 2017), 1131–1150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kai Hwang, Xiaoying Bai, Yue Shi, Muyang Li, Wen-Guang Chen, and Yongwei Wu. 2015. Cloud performance modeling with benchmark evaluation of elastic scaling strategies. IEEE Trans. Parallel Distrib. Syst. 27, 1 (2015), 130–143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sadeka Islam, Jacky Keung, Kevin Lee, and Anna Liu. 2012. Empirical prediction models for adaptive resource provisioning in the cloud. Future Gen. Comput. Syst. 28, 1 (2012), 155–162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. You Jia, Jingdong Wang, Gang Zeng, Hongbin Zha, and Xian-Sheng Hua. 2010. Optimizing kd-trees for scalable visual descriptor indexing. In Proceedings of CVPR. IEEE, 3392–3399.Google ScholarGoogle ScholarCross RefCross Ref
  31. Wei Jiang, Dongwon Lee, and Songlin Hu. 2012. Large-scale longitudinal analysis of SOAP-based and RESTful web services. In Proc. of ICWS. 218–225. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Yexi Jiang, Chang-shing Perng, Tao Li, and Rong Chang. 2011. Asap: A self-adaptive prediction system for instant cloud resource demand provisioning. In Proceedings of ICDM. IEEE, 1104–1109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. 1996. Reinforcement learning: A survey. J. Artific. Intell. Res. 4 (1996), 237–285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Hisham A. Kholidy, Hala Hassan, Amany M. Sarhan, Abdelkarim Erradi, and Sherif Abdelwahed. 2014. Qos optimization for cloud service composition based on economic model. In Proceedings of GIoTS. Springer, 355–366.Google ScholarGoogle Scholar
  35. Sheryl E. Kimes and Gary M. Thompson. 2004. Restaurant revenue management: Determining the best table mix. Decis. Sci. 35, 3 (2004), 371–392.Google ScholarGoogle ScholarCross RefCross Ref
  36. Sven Koenig and Reid G. Simmons. 1993. Complexity analysis of real-time reinforcement learning. In Proceedings of AAAI. 99–107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Frank L. Lewis, Draguna Vrabie, and Kyriakos G. Vamvoudakis. 2012. Using natural decision methods to design optimal adaptive controllers. IEEE Control Syst. 32, 6 (2012), 76–105.Google ScholarGoogle ScholarCross RefCross Ref
  38. Sajib Mistry, Athman Bouguettaya, and Hai Dong. 2018. Economic Models for Managing Cloud Services. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Sajib Mistry, Athman Bouguettaya, and Hai Dong. 2018. Long-term qualitative IaaS composition. In Economic Models for Managing Cloud Services. Springer, 77–110.Google ScholarGoogle Scholar
  40. Sajib Mistry, Athman Bouguettaya, Hai Dong, and Abdelkarim Erradi. 2016. Qualitative Economic Model for Long-Term IaaS Composition. Springer International Publishing, 317–332.Google ScholarGoogle Scholar
  41. Sajib Mistry, Athman Bouguettaya, Hai Dong, and Abdelkarim Erradi. 2017. Probabilistic qualitative preference matching in long-term IaaS composition. In Proceedings of ICSOC. Springer, 256–271.Google ScholarGoogle Scholar
  42. S. Mistry, A. Bouguettaya, H. Dong, and A. K. Qin. 2016. Metaheuristic optimization for long-term IaaS service composition. IEEE Trans. Serv. Comput.99 (2016), 1–1.Google ScholarGoogle Scholar
  43. Ahmed Moustafa and Minjie Zhang. 2013. Multi-objective service composition using reinforcement learning. In Proceedings of ICSOC. Springer, 298–312. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Rémi Munos, Tom Stepleton, Anna Harutyunyan, and Marc Bellemare. 2016. Safe and efficient off-policy reinforcement learning. In Advances in Neural Information Processing Systems. MIT Press, 1054–1062. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Fionn Murtagh and Pedro Contreras. 2012. Algorithms for hierarchical clustering: An overview. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 2, 1 (2012), 86–97.Google ScholarGoogle ScholarCross RefCross Ref
  46. Beomseok Nam and Alan Sussman. 2004. A comparative study of spatial indexing techniques for multidimensional scientific datasets. In Proceedings of SSDBM. IEEE, 171–180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Ranjan Pal and Pan Hui. 2013. Economic models for cloud service markets: Pricing and Capacity planning. Theoret. Comput. Sci. 496 (2013), 113–124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Edwin Pednault, Naoki Abe, and Bianca Zadrozny. 2002. Sequential cost-sensitive decision making with reinforcement learning. In Proceedings of SIGKDD. ACM, 259–268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Charles Reiss, John Wilkes, and Joseph L. Hellerstein. 2011. Google Cluster-usage Traces: Format + Schema. Technical Report. Google Inc., Mountain View, CA.Google ScholarGoogle Scholar
  50. John T. Robinson. 1981. The KDB-tree: A search structure for large multidimensional dynamic indexes. In Proceedings of ICMD. ACM, 10–18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. G. R. Santhanam, S. Basu, and V. Honavar. 2009. Web service substitution based on preferences over non-functional attributes. In Proceedings of SCC. 210–217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Andrew J. Schaefer, Matthew D. Bailey, Steven M. Shechter, and Mark S. Roberts. 2005. Modeling medical treatment using Markov decision processes. In Operations Research and Health Care. Springer, 593–612.Google ScholarGoogle Scholar
  53. Timos K. Sellis, Nick Roussopoulos, and Christos Faloutsos. 1997. Multidimensional access methods: Trees have grown everywhere. In Proceedings of VLDB, Vol. 97. 13–14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Guy Shani, David Heckerman, and Ronen I. Brafman. 2005. An MDP-based recommender system. J. Mach. Learn. Res. 6(Sep.2005), 1265–1295. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Bhanu Sharma, Ruppa K. Thulasiram, Parimala Thulasiraman, Saurabh K. Garg, and Rajkumar Buyya. 2012. Pricing cloud compute commodities: A novel financial economic model. In Proceedings of CCGRID. IEEE Computer Society, 451–457. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Robert R. Sokal and F. James Rohlf. 1962. The comparison of dendrograms by objective methods. Taxon (1962), 33–40.Google ScholarGoogle Scholar
  57. Alireza Souri, Amir Masoud Rahmani, Nima Jafari Navimipour, and Reza Rezaei. 2020. A hybrid formal verification approach for QoS-aware multi-cloud service composition. Cluster Comput. 23, 4 (2020), 2453–2470.Google ScholarGoogle ScholarCross RefCross Ref
  58. T. Thanakornworakij, R. Nassar, C. B. Leangsuksun, and M. Paun. 2012. An economic model for maximizing profit of a cloud service provider. In Proceedings of ARES. 274–279. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Hado Van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Blesson Varghese and Rajkumar Buyya. 2018. Next generation cloud computing: New trends and research directions. Future Gen. Comput. Syst. 79 (2018), 849–861. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Hongbign Wang, Xin Chen, Qin Wu, Qi Yu, Xingguo Hu, Zibin Zheng, and Athman Bouguettaya. 2017. Integrating reinforcement learning with multi-agent techniques for adaptive service composition. ACM Trans. Auton. Adapt. Syst. 12, 2 (2017), 1–42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Hongbing Wang, Xingguo Hu, Qi Yu, Mingzhu Gu, Wei Zhao, Jia Yan, and Tianjing Hong. 2020. Integrating reinforcement learning and skyline computing for adaptive service composition. Info. Sci. 519 (2020), 141–160.Google ScholarGoogle Scholar
  63. Hongbing Wang, Shizhi Shao, Xuan Zhou, Cheng Wan, and Athman Bouguettaya. 2009. Web service selection with incomplete or inconsistent user preferences. In Proceedings of ICSOC. Springer, 83–98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Hongbing Wang, Jie Zhang, Wenlong Sun, Hongye Song, Guibing Guo, and Xiang Zhou. 2012. WCP-Nets: A weighted extension to CP-Nets for web service selection. In Proceedings of ICSOC. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Hongbing Wang, Xuan Zhou, Xiang Zhou, Weihong Liu, Wenya Li, and Athman Bouguettaya. 2010. Adaptive service composition based on reinforcement learning. In Proceedings of ICSOC. Springer, 92–107.Google ScholarGoogle Scholar
  66. Xinyu Wang, Jianke Zhu, Zibin Zheng, Wenjie Song, Yuanhong Shen, and Michael R. Lyu. 2016. A spatial-temporal QoS prediction approach for time-aware web service recommendation. ACM Trans. Web 10, 1 (2016), 1–25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. Mach. Learn. 8, 3-4 (1992), 279–292.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Hong Xu and Baochun Li. 2013. Dynamic cloud pricing for revenue maximization. IEEE Trans. Cloud Comput. 1, 2 (July 2013), 158–171. DOI:https://doi.org/10.1109/TCC.2013.15 Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Zhen Ye, Athman Bouguettaya, and Xiaofang Zhou. 2013. QoS-aware cloud service composition using time series. In Proceedings of ICSOC. Vol. 8274. 9–22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Zhen Ye, Athman Bouguettaya, and Xiaofang Zhou. 2014. Economic model-driven cloud service composition. ACM Trans. Internet Technol. 14, 2--3 (2014), 20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Z. Ye, S. K. Mistry, A. Bouguettaya, and H. Dong. 2014. Long-term QoS-aware cloud service composition using multivariate time series analysis. IEEE Trans. Serv. Comput. 99 (2014), 1–1.Google ScholarGoogle Scholar
  72. Qi Yu and Athman Bouguettaya. 2008. Framework for web service query algebra and optimization. ACM Trans. Web 2, 1 (2008), 1–35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Tao Yu, Yue Zhang, and Kwei-Jay Lin. 2007. Efficient algorithms for Web services selection with end-to-end QoS constraints. ACM Trans. Web 1, 1 (2007), 6–es. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Sharrukh Zaman and Daniel Grosu. 2013. Combinatorial auction-based allocation of virtual machine instances in clouds. J. Parallel Distrib. Comput. 73, 4 (2013), 495–508. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Huiyuan Zheng, Jian Yang, and Weiliang Zhao. 2016. Probabilistic QoS aggregations for service composition. ACM Trans. Web 10, 2 (2016), 1–36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Qian Zhu and Gagan Agrawal. 2010. Resource provisioning with budget constraints for adaptive applications in cloud environments. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing. ACM, 304–307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Guobing Zou, Qiang Lu, Yixin Chen, Ruoyun Huang, You Xu, and Yang Xiang. 2012. QoS-aware dynamic composition of web services using numerical temporal planning. IEEE Trans. Services Comput. 7, 1 (2012), 18–31. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Sequential Learning-based IaaS Composition

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Article Metrics

        • Downloads (Last 12 months)28
        • Downloads (Last 6 weeks)1

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!