Abstract
Scientific workflows enable scientists to undertake analysis on large datasets and perform complex scientific simulations. These workflows are often mapped onto distributed and parallel computational infrastructures to speed up their executions. Prior to its execution, a workflow structure may suffer transformations to accommodate the computing infrastructures, normally involving task clustering and partitioning. However, these transformations may cause workflow imbalance because of the difference between execution task times (runtime imbalance) or because of unconsidered data dependencies that lead to data locality issues (data imbalance). In this article, to mitigate these imbalances, we enhance the workflow lifecycle process in use by introducing a workflow imbalance phase that quantifies workflow imbalance after the transformations. Our technique is based on structural analysis of Petri nets, obtained by model transformation of a data-intensive workflow, and Linear Programming techniques. Our analysis can be used to assist workflow practitioners in finding more efficient ways of transforming and scheduling their workflows. Moreover, based on our analysis, we also propose a technique to mitigate workflow imbalance by data throttling. Our approach is based on autonomic computing principles that determine how data transmission must be throttled throughout workflow jobs. Our autonomic data-throttling approach mainly monitors the execution of the workflow and recompute data-throttling values when certain watchpoints are reached and time derivation is observed. We validate our approach by a formal proof and by simulations along with the Montage workflow. Our findings show that a dynamic data-throttling approach is feasible, does not introduce a significant overhead, and minimizes the usage of input buffers and network bandwidth.
- Wil M. P. van der Aalst, Alexander Hirnschall, and H. M. W. (Eric) Verbeek. 2002. An alternative way to analyze workflow graphs. In Proceedings of the 14th International Conference on Advanced Information Systems Engineering (CAiSE’02). Springer-Verlag, London, 535--552. Google Scholar
Digital Library
- M. Ajmone Marsan, G. Balbo, G. Conte, S. Donatelli, and G. Franceschinis. 1995a. Modelling with Generalized Stochastic Petri Nets. John Wiley 8 Sons. Google Scholar
Digital Library
- M. Ajmone Marsan, G. Balbo, G. Conte, S. Donatelli, and G. Franceschinis. 1995b. Modelling with Generalized Stochastic Petri Nets. John Wiley 8 Sons. Google Scholar
Digital Library
- L. Aversano, A. Cimitile, P. Gallucci, and M. L. Villani. 2002. FlowManager: A workflow management system based on Petri nets. In Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC’02). 1054--1059. Google Scholar
Digital Library
- G. Bruce Berriman, Ewa Deelman, John Good, Joseph C. Jacob, Daniel S. Katz, Anastasia C. Laity, Thomas A. Prince, Gurmeet Singh, and Mei-Hui Su. 2007. Generating complex astronomy workflows. In Workflows for e-Science, Ian J. Taylor, Ewa Deelman, Dennis B. Gannon, and Matthew Shields (Eds.). Springer, London, 19--38.Google Scholar
- J. Campos and M. Silva. 1992. Structural techniques and performance bounds of stochastic Petri net models. In Proceedings of the Advances in Petri Nets 1992, G. Rozenberg (Ed.). Lecture Notes in Computer Science, vol 609. Springer, Berlin, Heidelberg. 352--391. Google Scholar
Digital Library
- J. Carmona, J. Júlvez, J. Cortadella, and M. Kishinevsky. 2009. Scheduling synchronous elastic designs. In Proceedings of the 2009 Application of Concurrency to System Design Conference (ACSD’09). Google Scholar
Digital Library
- Henri Casanova, Arnaud Legrand, and Martin Quinson. 2008. SimGrid: A generic framework for large-scale distributed experiments. In Proceedings of the 10th IEEE International Conference on Computer Modeling and Simulation. Google Scholar
Digital Library
- Weiwei Chen, Rafael Ferreira da Silva, Ewa Deelman, and Rizos Sakellariou. 2015. Using imbalance metrics to optimize task clustering in scientific workflow executions. Fut. Gener. Comput. Syst. 46, C (May 2015), 69--84. Google Scholar
Digital Library
- G. Chiola, C. Anglano, J. Campos, J. M. Colom, and M. Silva. 1993. Operational analysis of timed Petri nets and application to the computation of performance bounds. In Proceedings of the 5th International Workshop on Petri Nets and Performance Models (PNPM’93). IEEE Computer Society Press, Los Alamitos, CA, 128--137.Google Scholar
- E. Deelman, G. Mehta, G. Singh, M. Su, and K. Vahi. 2007. Pegasus: Mapping large-scale workflows to distributed resources. In Workflows for eScience. Springer, 376--394.Google Scholar
- Rubing Duan, Thomas Fahringer, Radu Prodan, Jun Qin, Alex Villazón, and Marek Wieczorek. 2005. Real world workflow applications in the Askalon grid environment. In Proceedings of the Advances in Grid Computing European Grid Conference (EGC’05). Springer, Berlin, 454--463. Google Scholar
Digital Library
- Rubing Duan, Farrukh Nadeem, Jie Wang, Yun Zhang, Radu Prodan, and Thomas Fahringer. 2009. A hybrid intelligent method for performance modeling and prediction of workflow activities in Grids. In Proceedings of the 9th IEEE International Symposium on Cluster Computing and the Grid (CCGrid’09). IEEE Computer Society, Los Alamitos, CA, 339--347. Google Scholar
Digital Library
- Rubing Duan, Radu Prodan, and Thomas Fahringer. 2006. Run-time optimisation of grid workflow applications. In Proceedings of the 7th IEEE/ACM International Conference on Grid Computing (GRID’06). IEEE Computer Society, Los Alamitos, CA, 33--40. Google Scholar
Digital Library
- G. Florin and S. Natkin. 1985. Les réseaux de Petri stochastiques. Techn. Sci. Inf. 4 (1985), 143--160.Google Scholar
- Zhijie Guan, Francisco Hernandez, Purushotham Bangalore, Jeff Gray, Anthony Skjellum, Vijay Velusamy, and Yin Liu. 2006. Grid-flow: A Grid-enabled scientific workflow system with a Petri-net-based interface: Research articles. Concurr. Comput. Pract. Exper. 18 (Aug. 2006), 1115--1140. Issue 10. Google Scholar
Digital Library
- Andreas Hoheisel. 2006. User tools and languages for graph-based Grid workflows: Research articles. Concurr. Comput. Pract. Exper. 18, 10 (2006), 1101--1113. Google Scholar
Digital Library
- Jeffrey O. Kephart and David M. Chess. 2003. The vision of autonomic computing. Computer 36, 1 (2003), 41--50. Google Scholar
Digital Library
- John D. C. Little. 1961. A proof for the queuing formula: L= λ W. Oper. Res. 9, 3 (1961), 383--387. Google Scholar
Digital Library
- Tadao Murata. 1989. Petri nets: Properties, analysis and applications. In Proceedings of the IEEE, Vol. 77. 541--580.Google Scholar
- F. Nerieri, R. Prodan, T. Fahringer, and Hong-Linh Truong. 2006. Overhead analysis of Grid workflow applications. In Proceedings of the 7th IEEE/ACM International Conference on Grid Computing (GRID’06). IEEE Computer Society, Los Alamitos, CA, 17--24. Google Scholar
Digital Library
- Tom Oinn, Mark Greenwood, Matthew Addis, M. Nedim Alpdemir, Justin Ferris, Kevin Glover, Carole Goble, Antoon Goderis, Duncan Hull, Darren Marvin, Peter Li, Phillip Lord, Matthew R. Pocock, Martin Senger, Robert Stevens, Anil Wipat, and Chris Wroe. 2006. Taverna: Lessons in creating a workflow environment for the life sciences: Research articles. Concurr. Comput. Pract. Exper. 18, 10 (2006), 1067--1100. Google Scholar
Digital Library
- Sang-Min Park and M. Humphrey. 2008. Data throttling for data-intensive workflows. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing. 1--11.Google Scholar
- Simone Pellegrini, Andreas Hoheisel, Francesco Giacomini, and Antonia Ghiselli. 2008. Using GWorkflowDL for middleware-independent modeling and enactment of workflows. In Proceedings of the CoreGRID Integration Workshop 2008.Google Scholar
- C. Ramchandani. 1974. Analysis of Asynchronous Concurrent Systems by Petri Nets. Ph.D. Dissertation. Department of Electrical Engineering, Massachusetts Institute of Technology, Cambridge, MA.Google Scholar
- Ricardo J. Rodríguez and Jorge Júlvez. 2010. Accurate performance estimation for stochastic marked graphs by bottleneck regrowing. In Proceedings of the 7th European Performance Engineering Workshop (EPEW’10), Lecture Notes in Computer Science, Vol. 6342. Springer, 175--190. Google Scholar
Digital Library
- Ricardo J. Rodríguez, Rafael Tolosana-Calasanz, and Omer F. Rana. 2012. Automating data-throttling analysis for data-intensive workflows. In Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid’12). IEEE, 310--317. Google Scholar
Digital Library
- Ricardo J. Rodríguez, Rafael Tolosana-Calasanz, and Omer F. Rana. 2012. Measuring the effectiveness of throttled data transfers on data-intensive workflows. In Proceedings of the 6th KES International Conference on Agent and Multi-Agent Systems, Technologies, and Applications (KES-AMSTA’12). 144--153. Google Scholar
Digital Library
- Jacek Sroka and Jan Hidders. 2009. Towards a formal semantics for the process model of the Taverna workbench. Part ii. Fundam. Inf. 92, 4 (2009), 373--396. Google Scholar
Digital Library
- I. J. Taylor, E. Deelman, D. B. Gannon, and M. Shields (Eds.). 2007. Workflows for eScience: Scientific Workflows for Grids. Springer. Google Scholar
Digital Library
- Rafael Tolosana-Calasanz, José A. Bañares, Pedro Álvarez, Joaquín Ezpeleta, and Omer F. Rana. 2010a. An uncoordinated asynchronous checkpointing model for hierarchical scientific workflows. J. Comput. Syst. Sci. 76, 6 (Sep. 2010), 403--415. Google Scholar
Digital Library
- Rafael Tolosana-Calasanz, José A. Bañares, Omer F. Rana, Pedro Álvarez, Joaquín Ezpeleta, and Andreas Hoheisel. 2010b. Adaptive exception handling for scientific workflows. Concurr. Comput. Pract. Exper. 22, 5 (Apr. 2010), 617--642. Google Scholar
Digital Library
- Rafael Tolosana-Calasanz, Omer Rana, and José Bañares. 2008. Automating performance analysis from Taverna workflows. In Component-Based Software Engineering, Michel Chaudron, Clemens Szyperski, and Ralf Reussner (Eds.). Lecture Notes in Computer Science, Vol. 5282. Springer, Berlin, 1--15. Google Scholar
Digital Library
- Wil van der Aalst and Kees van Hee. 2004. Workflow Management: Models, Methods, and Systems. The MIT Press. Google Scholar
Digital Library
- Michal Vossberg, Andreas Hoheisel, Thomas Tolxdorff, and Dagmar Krefting. 2008. A reliable DICOM transfer grid service based on Petri net workflows. In Proceedings of the 2008 8th IEEE International Symposium on Cluster Computing and the Grid. IEEE Computer Society, Los Alamitos, CA, 441--448. Google Scholar
Digital Library
- Jia Yu and Rajkumar Buyya. 2005. A taxonomy of workflow management systems for Grid computing. CoRR 34, 3 (Sep. 2005), 44--49.Google Scholar
Index Terms
A Dynamic Data-throttling Approach to Minimize Workflow Imbalance
Recommendations
Pegasus, a workflow management system for science automation
Modern science often requires the execution of large-scale, multi-stage simulation and data analysis pipelines to enable the study of complex systems. The amount of computation and data involved in these pipelines requires scalable workflow management ...
Building the Trident Scientific Workflow Workbench for Data Management in the Cloud
ADVCOMP '09: Proceedings of the 2009 Third International Conference on Advanced Engineering Computing and Applications in SciencesScientific workflows have gained popularity for modeling and executing in silico experiments by scientists for problem-solving. These workflows primarily engage in computation and data transformation tasks to perform scientific analysis in the Science ...
A taxonomy of scientific workflow systems for grid computing
With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios ...






Comments