Abstract
With the advent of cloud computing and the availability of data collected from increasingly powerful scientific instruments, workflows have become a prevailing mean to achieve significant scientific advances at an increased pace. Scheduling algorithms are crucial in enabling the efficient automation of these large-scale workflows, and considerable effort has been made to develop novel heuristics tailored for the cloud resource model. The majority of these algorithms focus on coarse-grained billing periods that are much larger than the average execution time of individual tasks. Instead, our work focuses on emerging finer-grained pricing schemes (e.g., per-minute billing) that provide users with more flexibility and the ability to reduce the inherent wastage that results from coarser-grained ones. We propose a scheduling algorithm whose objective is to optimize a workflow’s execution time under a budget constraint; quality of service requirement that has been overlooked in favor of optimizing cost under a deadline constraint. Our proposal addresses fundamental challenges of clouds such as resource elasticity, abundance, and heterogeneity, as well as resource performance variation and virtual machine provisioning delays. The simulation results demonstrate our algorithm’s responsiveness to environmental uncertainties and its ability to generate high-quality schedules that comply with the budget constraint while achieving faster execution times when compared to state-of-the-art algorithms.
- Eun-Kyu Byun, Yang-Suk Kee, Jin-Soo Kim, and Seungryoul Maeng. 2011. Cost optimized provisioning of elastic resources for application workflows. Future Gener. Comput. Syst. 27, 8 (2011), 1011--1026. Google Scholar
Digital Library
- Rodrigo N. Calheiros, Rajiv Ranjan, Anton Beloglazov, César A. F. De Rose, and Rajkumar Buyya. 2011. CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exper. 41, 1 (2011), 23--50. Google Scholar
Digital Library
- Thiago A. L. Genez, Luiz F. Bittencourt, and Edmundo R. M. Madeira. 2012. Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels. In Proceedings of the Network Operations Management Symposium (NOMS’12).Google Scholar
- Yolanda Gil, Ewa Deelman, Mark Ellisman, Thomas Fahringer, Geoffrey Fox, Dennis Gannon, Carole Goble, Miron Livny, Luc Moreau, and Jim Myers. 2007. Examining the challenges of scientific workflows. IEEE Comput. 40, 12 (2007), 26--34. Google Scholar
Digital Library
- Google. 2015a. Amazon Simple Storage Service. (Nov 2015). Retrieved October 2016 from http://aws.amazon.com/s3/.Google Scholar
- Google. 2015b. Google Cloud Storage. (Nov 2015). Retrieved October 2016 from https://cloud.google.com/storage/.Google Scholar
- Google. 2015c. Google Compute Engine. (Nov 2015). Retrieved October 2016 from https://cloud.google.com/compute/.Google Scholar
- A. Gupta and D. Milojicic. 2011. Evaluation of HPC applications on cloud. In Proceedings of the 2011 6th Open Cirrus Summit (OCS’11). 22--26. Google Scholar
Digital Library
- A. Iosup, S. Ostermann, M. N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. 2011. Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Parallel Distrib. Syst. 22, 6 (June 2011), 931--945. Google Scholar
Digital Library
- Keith R. Jackson, Lavanya Ramakrishnan, Krishna Muriki, Shane Canon, Shreyas Cholia, John Shalf, Harvey J. Wasserman, and Nicholas J. Wright. 2010. Performance analysis of high performance computing applications on the amazon web services cloud. In Proceedings of the International Conference on Cloud Computing Technology and Science (CloudCom). Google Scholar
Digital Library
- Gideon Juve, Ann Chervenak, Ewa Deelman, Shishir Bharathi, Gaurang Mehta, and Karan Vahi. 2013. Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29, 3 (2013), 682--692. Google Scholar
Digital Library
- Maciej Malawski, Kamil Figiela, Marian Bubak, Ewa Deelman, and Jarek Nabrzyski. 2015. Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Sci. Program. 2015, 5 (2015). Google Scholar
Digital Library
- Maciej Malawski, Gideon Juve, Ewa Deelman, and Jarek Nabrzyski. 2012. Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In Proceedings of the International Conference on High Performance Computing, Networking, and Storage Analysis (SC’12). Google Scholar
Digital Library
- Ming Mao and Marty Humphrey. 2011. Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In Proceedings of the International Conference on High Performance Computing, Networking, and Storage Analysis (SC’11). Google Scholar
Digital Library
- Ming Mao and Marty Humphrey. 2013. Scaling and scheduling to maximize application performance within budget constraints in cloud workflows. In Proceedings of the International Parallel 8 Distributed Processing Symposium (IPDPS'13). IEEE, 67--78. Google Scholar
Digital Library
- Microsoft. 2015. Microsoft Azure. (Nov 2015). Retrieved October 2016 from https://azure.microsoft.com.Google Scholar
- Simon Ostermann, Alexandria Losup, Nezih Yigitbasi, Radu Prodan, Thomas Fahringer, and Dick Epema. 2010. A performance analysis of EC2 cloud computing services for scientific computing. In Cloud Computing. Springer, 115--131.Google Scholar
- Ilia Pietri, Maciej Malawski, Gideon Juve, Ewa Deelman, Jarek Nabrzyski, and Rizos Sakellariou. 2013. Energy-constrained provisioning for scientific workflow ensembles. In Proceedings of the International Conference on Cloud Green Computing (CGC’13). Google Scholar
Digital Library
- Deepak Poola, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2014. Fault-tolerant workflow scheduling using spot instances on clouds. Proc. Comput. Sci. 29 (2014), 523--533.Google Scholar
Cross Ref
- Rackspace. 2015. Rackspace Block Storage. (Nov 2015). Retrieved October 2016 from http://www.rackspace.com.au/cloud/block-storage.Google Scholar
- Jörg Schad, Jens Dittrich, and Jorge-Arnulfo Quiané-Ruiz. 2010. Runtime measurements in the cloud: Observing, analyzing, and reducing variance. Proc. VLDB Endow. 3, 1--2 (2010), 460--471. Google Scholar
Digital Library
- Jianwu Wang, Prakashan Korambath, Ilkay Altintas, Jim Davis, and Daniel Crawl. 2014. Workflow as a service in the cloud: Architecture and scheduling algorithms. Proc. Comput. Sci. 29 (2014), 546--556.Google Scholar
Cross Ref
- Chunlin Wu, Xingqin Lin, Daren Yu, Wei Xu, and Luoqing Li. 2015. End-to-end delay minimization for scientific workflows in clouds under budget constraint. IEEE Trans. Cloud Comput. 3, 2 (2015), 169--181.Google Scholar
Cross Ref
- Zhangjun Wu, Zhiwei Ni, Lichuan Gu, and Xiao Liu. 2010. A revised discrete particle swarm optimization for cloud workflow scheduling. In Proceedings of the International Conference on Computational Intelligence Security (CIS’10). Google Scholar
Digital Library
- Jia Yu, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2009. Deadline/budget-based scheduling of workflows on utility grids. Market-Oriented Grid and Utility Computing (2009), John Wiley 8 Sons, Inc., 427--450.Google Scholar
Cross Ref
- Lingfang Zeng, Bharadwaj Veeravalli, and Xiaorong Li. 2012. Scalestar: Budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In Proceedings of the International Conference on Advanced Information Network Applications (AINA’12). Google Scholar
Digital Library
- Amelie Chi Zhou, Bingsheng He, and Cheng Liu. 2016. Monetary cost optimizations for hosting workflow-as-a-service in IaaS clouds. IEEE Trans. Cloud Comput. 4, 1 (2016), 34--48. Google Scholar
Digital Library
Index Terms
Budget-Driven Scheduling of Scientific Workflows in IaaS Clouds with Fine-Grained Billing Periods
Recommendations
Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints
With the popularization and development of cloud computing, lots of scientific computing applications are conducted in cloud environments. However, current application scenario of scientific computing is also becoming increasingly dynamic and ...
Budget-Constrained Resource Provisioning for Scientific Applications in Clouds
CLOUDCOM '13: Proceedings of the 2013 IEEE International Conference on Cloud Computing Technology and Science - Volume 01Public commercial clouds emerged as new and attractive resource provisioning option for scientific computing. This new alternative raises new challenges for users of such clouds, since optimizing the completion time of scientific applications might ...
A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds
In the last years, scientific workflows have emerged as a fundamental abstraction for structuring and executing scientific experiments in computational environments. Scientific workflows are becoming increasingly complex and more demanding in terms of ...






Comments