skip to main content
research-article

VizSciFlow: A Visually Guided Scripting Framework for Supporting Complex Scientific Data Analysis

Published:18 June 2020Publication History
Skip Abstract Section

Abstract

Scientific workflow management systems such as Galaxy, Taverna and Workspace, have been developed to automate scientific workflow management and are increasingly being used to accelerate the specification, execution, visualization, and monitoring of data-intensive tasks. For example, the popular bioinformatics platform Galaxy is installed on over 168 servers around the world and the social networking space myExperiment shares almost 4,000 Galaxy scientific workflows among its 10,665 members. Most of these systems offer graphical interfaces for composing workflows. However, while graphical languages are considered easier to use, graphical workflow models are more difficult to comprehend and maintain as they become larger and more complex. Text-based languages are considered harder to use but have the potential to provide a clean and concise expression of workflow even for large and complex workflows. A recent study showed that some scientists prefer script/text-based environments to perform complex scientific analysis with workflows. Unfortunately, such environments are unable to meet the needs of scientists who prefer graphical workflows. In order to address the needs of both types of scientists and at the same time to have script-based workflow models because of their underlying benefits, we propose a visually guided workflow modeling framework that combines interactive graphical user interface elements in an integrated development environment with the power of a domain-specific language to compose independently developed and loosely coupled services into workflows. Our domain-specific language provides scientists with a clean, concise, and abstract view of workflow to better support workflow modeling. As a proof of concept, we developed VizSciFlow, a generalized scientific workflow management system that can be customized for use in a variety of scientific domains. As a first use case, we configured and customized VizSciFlow for the bioinformatics domain. We conducted three user studies to assess its usability, expressiveness, efficiency, and flexibility. Results are promising, and in particular, our user studies show that VizSciFlow is more desirable for users to use than either Python or Galaxy for solving complex scientific problems.

References

  1. Enis Afgan, Dannon Baker, Marius Van den Beek, Daniel Blankenberg, Dave Bouvier, Martin vC ech, John Chilton, Dave Clements, Nate Coraor, Carl Eberhard, et al. 2016. The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic acids research, Vol. 44, W1 (2016), W3--W10.Google ScholarGoogle Scholar
  2. Ilkay Altintas. 2011. Distributed Workflow-driven Analysis of Large-scale Biological Data Using Biokepler. In Proceedings of the 2Nd International Workshop on Petascal Data Analytics: Challenges and Opportunities (Seattle, Washington, USA) (PDAC '11). ACM, New York, NY, USA, 41--42. https://doi.org/10.1145/2110205.2110215Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Ilkay Altintas, Chad Berkley, Efrat Jaeger, Matthew Jones, Bertram Ludascher, and Steve Mock. 2004. Kepler: An Extensible System for Design and Execution of Scientific Workflows. In Proceedings of the 16th International Conference on Scientific and Statistical Database Management (SSDBM '04). IEEE Computer Society, Washington, DC, USA, 423--. https://doi.org/10.1109/SSDBM.2004.44Google ScholarGoogle ScholarCross RefCross Ref
  4. A. Alves, A. Arkin, S. Askary, C. Barreto, B. Bloch, Francisco Curbera, M. Ford, Y. Goland, A. Guzar, Neelakantan Kartha, C. Liu, and Rania Khalaf. 2007. Web Services Business Process Execution Language Version 2.0 (OASIS Standard). OASIS Standard, Vol. 11 (01 2007).Google ScholarGoogle Scholar
  5. Peter Amstutz, Nebojsa Tijanic, Stian Soiland-Reyes, John Kern, Luka Stojanovic, Tim Pierce, John Chilton, Maxim Mikheev, Samuel Lampa, Hervé Ménager, Scott Frazer, Venkat S. Malladi, and Michael R. Crusoe. 2015. Beyond Galaxy: portable workflows and tool definitions with the CWL. https://cesgo.genouest.org/resources/129Google ScholarGoogle Scholar
  6. Simon Andrews et al. 2010. FastQC: a quality control tool for high throughput sequence data.Google ScholarGoogle Scholar
  7. Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, Vol. 4, 3 (2009), 114--123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ankica Barivs ic, Vasco Amaral, Miguel Goul ao, and Bruno Barroca. 2014. Evaluating the usability of domain-specific languages. In Software Design and Development: Concepts, Methodologies, Tools, and Applications. IGI Global, 2120--2141.Google ScholarGoogle Scholar
  9. Ankica Barivsić, Pedro Monteiro, Vasco Amaral, Miguel Goul ao, and Miguel Monteiro. 2012. Patterns for Evaluating Usability of Domain-Specific Languages. In Proceedings of the 19th Conference on Pattern Languages of Programs (Tucson, Arizona) (PLoP '12). The Hillside Group, USA, Article 14, 34 pages.Google ScholarGoogle Scholar
  10. Adam Barker and Jano Van Hemert. 2007. Scientific workflow: a survey and research directions. In International Conference on Parallel Processing and Applied Mathematics. Springer, New York, NY, 746--753.Google ScholarGoogle Scholar
  11. Jon Bentley. 1986. Programming pearls: little languages. Commun. ACM, Vol. 29, 8 (1986), 711--721.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Nigel Bevan, Carol Barnum, Gilbert Cockton, Jakob Nielsen, Jared Spool, and Dennis Wixon. 2003. The magic number 5: is it enough for web testing?. In CHI'03 extended abstracts on Human factors in computing systems. ACM, New York, NY, USA, 698--699.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Alan F Blackwell. 1996. Metacognitive theories of visual programming: what do we think we are doing?. In Visual Languages, 1996. Proceedings., IEEE Symposium on. IEEE, 240--246.Google ScholarGoogle ScholarCross RefCross Ref
  14. Shawn Bowers and Bertram Lud"ascher. 2005. Actor-oriented design of scientific workflows. In International Conference on Conceptual Modeling. Springer, New York, NY, 369--384.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. John Brooke. 2013. SUS: a retrospective. Journal of usability studies, Vol. 8, 2 (2013), 29--40.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry, Vol. 189, 194 (1996), 4--7.Google ScholarGoogle Scholar
  17. Steven P Callahan, Juliana Freire, Emanuele Santos, Carlos E Scheidegger, Cláudio T Silva, and Huy T Vo. 2006. VisTrails: visualization meets data management. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data. ACM, New York, NY, USA, 745--747.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J Gregory Caporaso, Justin Kuczynski, Jesse Stombaugh, Kyle Bittinger, Frederic D Bushman, Elizabeth K Costello, Noah Fierer, Antonio Gonzalez Pena, Julia K Goodrich, Jeffrey I Gordon, et al. 2010. QIIME allows analysis of high-throughput community sequencing data. Nature methods, Vol. 7, 5 (2010), 335.Google ScholarGoogle Scholar
  19. Debasish Chakraborti, Banani Roy, Chanchal Roy, and Kevin Schneider. 2018. Optimized Storing of Workflow Outputs through Mining Association Rules. In 2018 IEEE International Conference on Big Data (Big Data). 508--515.Google ScholarGoogle ScholarCross RefCross Ref
  20. Debasish Chakroborti, Banani Roy, Amit Kumar Mondal, Golam Mostaeen, Ralph Deters, Chanchal K. Roy, and Schneider Kevin A. 2020. A Data Management Scheme for Micro-Level Modular Computation-intensive Programs in Big Data Platforms. In In: Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond, Studies in Big Data, Vol. 65. 1--20.Google ScholarGoogle Scholar
  21. Jinjun Chen and Wil van der Aalst. 2007. On scientific workflows. IEEE Computer Society's Technical Committee for Scalable Computing, Vol. 9 (2007).Google ScholarGoogle Scholar
  22. Michele Chinosi and Alberto Trombetta. 2012. BPMN: An introduction to the standard. Computer Standards & Interfaces, Vol. 34, 1 (2012), 124--134.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. David Churches, Gabor Gombas, Andrew Harrison, Jason Maassen, Craig Robinson, Matthew Shields, Ian Taylor, and Ian Wang. 2006. Programming Scientific and Distributed Workflow with Triana Services: Research Articles. Concurr. Comput. : Pract. Exper., Vol. 18, 10 (Aug. 2006), 1021--1037. https://doi.org/10.1002/cpe.v18:10Google ScholarGoogle ScholarCross RefCross Ref
  24. Paul Cleary, Matt Bolger, Lachlan Hetherton, Chris Rucinski, David Thomas, and Damien Watkins. 2014. Workspace: A Platform for Delivering Scientific Applications. Proceedings eResearch (2014), 4.Google ScholarGoogle Scholar
  25. Peter JA Cock, Tiago Antao, Jeffrey T Chang, Brad A Chapman, Cymon J Cox, Andrew Dalke, Iddo Friedberg, Thomas Hamelryck, Frank Kauff, Bartek Wilczynski, et al. 2009. Biopython: freely available Python tools for computational molecular biology and bioinformatics. Bioinformatics, Vol. 25, 11 (2009), 1422--1423.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Steve Cook, Gareth Jones, Stuart Kent, and Alan Cameron Wills. 2007. Domain-specific development with visual studio dsl tools .Pearson Education.Google ScholarGoogle Scholar
  27. Fredy Cuenca, Jan Van den Bergh, Kris Luyten, and Karin Coninx. 2014. A domain-specific textual language for rapid prototyping of multimodal interactive systems. In Proceedings of the 2014 ACM SIGCHI symposium on Engineering interactive computing systems. ACM, New York, NY, USA, 97--106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM, Vol. 51, 1 (2008), 107--113.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ewa Deelman, Karan Vahi, Gideon Juve, Mats Rynge, Scott Callaghan, Philip J Maechling, Rajiv Mayani, Weiwei Chen, Rafael Ferreira Da Silva, Miron Livny, et al. 2015. Pegasus, a workflow management system for science automation. Future Generation Computer Systems, Vol. 46 (2015), 17--35.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Paolo Di Tommaso, Maria Chatzou, Evan W Floden, Pablo Prieto Barja, Emilio Palumbo, and Cedric Notredame. 2017. Nextflow enables reproducible computational workflows. Nature biotechnology, Vol. 35, 4 (2017), 316--319.Google ScholarGoogle Scholar
  31. Marlon Dumas and Arthur ter Hofstede. 2001. UML Activity Diagrams as a Workflow Specification Language .Springer Berlin Heidelberg, Berlin, Heidelberg, 76--90. https://doi.org/10.1007/3--540--45441--1_7Google ScholarGoogle Scholar
  32. elephantlaboratories. 2019. Does anyone use CWL? Does it actually help you get work done? https://www.reddit.com/r/bioinformatics/comments/7gxsk0/does_anyone_use_cwl_does_it_actually_help_you_get/, visited 2019-07-08.Google ScholarGoogle Scholar
  33. Moritz Eysholdt and Heiko Behrens. 2010. Xtext: implement your language faster than the quick and dirty way. In Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companion. ACM, New York, NY, USA, 307--309.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Rayhan Ferdous, Banani Roy, Chanchal Roy, and Kevin Schneider. 2020. Workflow Provenance for Big Data: From Modelling to Reporting. In Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond, Studies in Big Data, Vol. 65. 1--18.Google ScholarGoogle Scholar
  35. Peter Forbrig, Anke Dittmar, and Mathias Kühn. 2018. A Textual Domain Specific Language for Task Models: Generating Code for CoTaL, CTTE, and HAMSTERS. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems. ACM, New York, NY, USA, 5.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Martin Fowler. 2010a. Domain-specific languages .Pearson Education.Google ScholarGoogle Scholar
  37. Martin Fowler. 2010b. Domain Specific Languages 1st ed.). Addison-Wesley Professional, Reading, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Debasish Ghosh. 2011. DSL for the uninitiated. Commun. ACM, Vol. 54, 7 (2011), 44--50.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Belinda Giardine, Cathy Riemer, Ross C Hardison, Richard Burhans, Laura Elnitski, Prachi Shah, Yi Zhang, Daniel Blankenberg, Istvan Albert, James Taylor, et al. 2005. Galaxy: a platform for interactive large-scale genome analysis. Genome research, Vol. 15, 10 (2005), 1451--1455.Google ScholarGoogle Scholar
  40. Yolanda Gil, Ewa Deelman, Mark Ellisman, Thomas Fahringer, Geoffrey Fox, Dennis Gannon, Carole Goble, et al. 2007. Examining the challenges of scientific workflows. Computer, Vol. 40, 12 (2007), 24--32.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. David J. Gilmore and Thomas R. G. Green. 1984. Comprehension and recall of miniature programs. International Journal of Man-Machine Studies, Vol. 21, 1 (1984), 31--48.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Carole A Goble et al. 2010. myExperiment: a repository and social network for the sharing of bioinformatics workflows. Nucleic acids research, Vol. 38, suppl_2 (2010), W677--W682.Google ScholarGoogle Scholar
  43. Jeremy Goecks, Anton Nekrutenko, et al. 2010. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome biology, Vol. 11, 8 (2010), R86.Google ScholarGoogle Scholar
  44. Katharina Görlach, Mirko Sonntag, Dimka Karastoyanova, Frank Leymann, and Michael Reiter. 2011. Conventional workflow technology for scientific simulation. In Guide to e-Science. Springer, New York, NY, 323--352.Google ScholarGoogle Scholar
  45. Thomas RG Green and Marian Petre. 1992. When visual programs are harder to read than textual programs. In Human-Computer Interaction: Tasks and Organisation, Proceedings of ECCE-6 (6th European Conference on Cognitive Ergonomics). GC van der Veer, MJ Tauber, S. Bagnarola and M. Antavolits. Rome, CUD. 167--180.Google ScholarGoogle Scholar
  46. Thomas RG Green, Marian Petre, and RKE Bellamy. 1991. Comprehensibility of visual and textual programs: A test of superlativism against the'match-mismatch'conjecture. ESP, Vol. 91, 743 (1991), 121--146.Google ScholarGoogle Scholar
  47. Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage publications Sage CA, Los Angeles, CA, 904--908.Google ScholarGoogle Scholar
  48. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.Google ScholarGoogle Scholar
  49. Petra Heinl, Stefan Horn, Stefan Jablonski, Jens Neeb, et al. 1999. A comprehensive approach to flexibility in workflow management systems. In ACM SIGSOFT Software Engineering Notes, Vol. 24. ACM, New York, NY, USA, 79--88.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. D Hollingsworth. 1995. Workflow Management Coalition: The Workflow Reference Model. Workflow Management Coalition, Vol. 68 (01 1995).Google ScholarGoogle Scholar
  51. David Hollingsworth et al. 2004. The workflow reference model: 10 years on. In Fujitsu Services, UK; Technical Committee Chair of WfMC. Citeseer, 295--312.Google ScholarGoogle Scholar
  52. Shawn Hoon, Kiran Kumar Ratnapu, Jer-ming Chia, Balamurugan Kumarasamy, Xiao Juguang, Michele Clamp, Arne Stabenau, Simon Potter, Laura Clarke, and Elia Stupka. 2003. Biopipe: a flexible framework for protocol-based bioinformatics analysis. Genome Research, Vol. 13, 8 (2003), 1904--1915.Google ScholarGoogle Scholar
  53. Paul Hudak. 1998. Modular domain specific languages and tools. In Software Reuse, 1998. Proceedings. Fifth International Conference on. IEEE, 134--142.Google ScholarGoogle ScholarCross RefCross Ref
  54. Kevin Jacobs and Kacper Surdy. 2016. Apache Flink: Distributed stream data processing. Technical Report.Google ScholarGoogle Scholar
  55. Frédéric Jouault, Jean Bézivin, and Ivan Kurtev. 2006. TCS:: a DSL for the specification of textual concrete syntaxes in model engineering. In Proceedings of the 5th international conference on Generative programming and component engineering. ACM, New York, NY, USA, 249--254.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Holden Karau. 2017. Unifying the open big data world: The possibilities$ast$ of apache BEAM. In 2017 IEEE International Conference on Big Data (Big Data). IEEE, 3981--3981.Google ScholarGoogle ScholarCross RefCross Ref
  57. Richard B Kieburtz, Laura McKinney, Jeffrey M Bell, James Hook, Alex Kotov, Jeffrey Lewis, Dino P Oliva, Tim Sheard, Ira Smith, and Lisa Walton. 1996. A software engineering experiment in software component generation. In Proceedings of the 18th international conference on Software engineering. IEEE Computer Society, 542--552.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Dimitrios S Kolovos, Richard F Paige, and Fiona AC Polack. 2006. Eclipse development tools for epsilon. In Eclipse Summit Europe, Eclipse Modeling Symposium, Vol. 20062. 200.Google ScholarGoogle Scholar
  59. Tomavz Kosar, Sudev Bohra, and Marjan Mernik. 2016. Domain-specific languages: A systematic mapping study. Information and Software Technology, Vol. 71 (2016), 77--91.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Johannes Köster and Sven Rahmann. 2012. Snakemake-a scalable bioinformatics workflow engine. Bioinformatics, Vol. 28, 19 (2012), 2520--2522.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. A. Lajmi, J. Martinez, and T. Ziadi. 2014. DSLFORGE: Textual modeling on the web. CEUR Workshop Proceedings, Vol. 1255 (01 2014), 25--29.Google ScholarGoogle Scholar
  62. Peter Lawrence (Ed.). 1997. Workflow Handbook 1997 .John Wiley & Sons, Inc., New York, NY, USA.Google ScholarGoogle Scholar
  63. Jeremy Leipzig. 2017. A review of bioinformatic pipeline frameworks. Briefings in bioinformatics, Vol. 18, 3 (2017), 530--536.Google ScholarGoogle Scholar
  64. James Jim R Lewis and Jeff Sauro. 2017. Revisiting the factor structure of the System Usability Scale. Journal of Usability Studies, Vol. 12, 4 (2017), 183--192.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. James R Lewis and Jeff Sauro. 2009. The factor structure of the system usability scale. In International conference on human centered design. Springer, New York, NY, 94--103.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Heng Li. 2013. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. ArXiv, Vol. 1303 (2013).Google ScholarGoogle Scholar
  67. Heng Li, Bob Handsaker, Alec Wysoker, Tim Fennell, Jue Ruan, Nils Homer, Gabor Marth, Goncalo Abecasis, and Richard Durbin. 2009. The sequence alignment/map format and SAMtools. Bioinformatics, Vol. 25, 16 (2009), 2078--2079.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. William Lidwell et al. 2010. Universal principles of design, revised and updated: 125 ways to enhance usability, influence perception, increase appeal, make better design decisions, and teach through design .Rockport Pub.Google ScholarGoogle Scholar
  69. Cui Lin, Shiyong Lu, Xubo Fei, Artem Chebotko, Darshan Pai, Zhaoqiang Lai, Farshad Fotouhi, and Jing Hua. 2009. A reference architecture for scientific workflow management systems and the VIEW SOA solution. IEEE Transactions on Services Computing, Vol. 2, 1 (2009), 79--92.Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Ji Liu, Esther Pacitti, Patrick Valduriez, and Marta Mattoso. 2015. A survey of data-intensive scientific workflow management. Journal of Grid Computing, Vol. 13, 4 (2015), 457--493.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Bertram Lud"ascher et al. 2009a. Scientific process automation and workflow management. In Scientific Data Management: Challenges, Technology, and Deployment. CRC press.Google ScholarGoogle Scholar
  72. Bertram Lud"ascher, Mathias Weske, Timothy McPhillips, and Shawn Bowers. 2009b. Scientific workflows: Business as usual?. In International Conference on Business Process Management. Springer, New York, NY, 31--47.Google ScholarGoogle Scholar
  73. Bertram Ludäscher, Ilkay Altintas, and Amarnath Gupta. 2003. Compiling Abstract Scientific Workflows into Web Service Workflows. In 15th International Conference on Scientific and Statistical Database Management, 2003., Vol. 2003. IEEE, 251--254. https://doi.org/10.1109/SSDM.2003.1214990Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Tanja Magovc and Steven L Salzberg. 2011. FLASH: fast length adjustment of short reads to improve genome assemblies. Bioinformatics, Vol. 27, 21 (2011), 2957--2963.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Ketan Maheshwari and Johan Montagnat. 2010. Scientific workflow development using both visual and script-based representation. In 2010 6th World Congress on Services. IEEE, 328--335.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Seema Maitrey and C.K. Jha. 2015. MapReduce: Simplified Data Analysis of Big Data. Procedia Computer Science, Vol. 57 (2015), 563 -- 571. https://doi.org/10.1016/j.procs.2015.07.392 3rd International Conference on Recent Trends in Computing 2015 (ICRTC-2015).Google ScholarGoogle ScholarCross RefCross Ref
  77. Shalil Majithia, Matthew Shields, Ian Taylor, and Ian Wang. 2004. Triana: A graphical web service composition and execution toolkit. In Web Services, 2004. Proceedings. IEEE International Conference on. IEEE, 514--521.Google ScholarGoogle ScholarCross RefCross Ref
  78. James Malone, Andy Brown, Allyson L Lister, Jon Ison, Duncan Hull, Helen Parkinson, and Robert Stevens. 2014. The Software Ontology (SWO): a resource for reproducibility in biomedical data analysis, curation and digital preservation. Journal of biomedical semantics, Vol. 5, 1 (2014), 25.Google ScholarGoogle ScholarCross RefCross Ref
  79. Daniel D McCracken and Edwin D Reilly. 2003. Backus-naur form (bnf). (2003).Google ScholarGoogle Scholar
  80. Paul McGuire. 2007. Getting started with pyparsing ." O'Reilly Media, Inc.", 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA.Google ScholarGoogle Scholar
  81. Marjan Mernik, Jan Heering, and Anthony M Sloane. 2005. When and how to develop domain-specific languages. ACM computing surveys (CSUR), Vol. 37, 4 (2005), 316--344.Google ScholarGoogle Scholar
  82. Golam Mostaeen, Banani Roy, Chanchal Roy, and Kevin Schneider. 2018. Fine-Grained Attribute Level Locking Scheme for Collaborative Scientific Workflow Development. In 2018 IEEE International Conference on Services Computing (SCC). 273--277.Google ScholarGoogle Scholar
  83. Golam Mostaeen, Banani Roy, Chanchal Roy, and Kevin Schneider. 2019. Designing for Real-Time Groupware Systems to Support Complex Scientific Data Analysis. Journal Proceedings of the ACM on Human-Computer Interaction, Vol. 3, EICS, Article 9 (June 2019), 28 pages.Google ScholarGoogle Scholar
  84. Jakob Nielsen. 1994. Usability engineering. Elsevier.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Jakob Nielsen. 2000. Why You Only Need to Test with 5 Users, Jakob Nielsen's Alertbox. "https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/"Google ScholarGoogle Scholar
  86. Shadi A Noghabi et al. 2017. Samza: stateful scalable stream processing at LinkedIn. Proceedings of the VLDB Endowment, Vol. 10, 12 (2017), 1634--1645.Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Andres Ojamaa, Hele-Mai Haav, and Jaan Penjam. 2015. Semi-automated generation of DSL meta models from formal domain ontologies. In Model and Data Engineering. Springer, New York, NY, 3--15.Google ScholarGoogle Scholar
  88. Chris Parnin, Eric Helms, Chris Atlee, Harley Boughton, Mark Ghattas, Andy Glover, James Holman, John Micco, et al. 2017. The top 10 adages in continuous deployment. IEEE Software, Vol. 34, 3 (2017), 86--95.Google ScholarGoogle ScholarDigital LibraryDigital Library
  89. Terence Parr. 2013. The definitive ANTLR 4 reference .Pragmatic Bookshelf.Google ScholarGoogle Scholar
  90. Maja Pesic, Helen Schonenberg, and Wil van der Aalst. 2010. Declarative workflow. In Modern Business Process Automation. Springer, 175--201.Google ScholarGoogle Scholar
  91. Marian Petre. 1995. Why Looking Isn't Always Seeing: Readership Skills and Graphical Programming. Commun. ACM, Vol. 38, 6 (June 1995), 33--44. https://doi.org/10.1145/203241.203251Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Carl Adam Petri. 1962. Kommunikation mit Automaten. Ph.D. Dissertation. Universität Hamburg.Google ScholarGoogle Scholar
  93. Akond Rahman et al. 2018. What questions do programmers ask about configuration as code?. In Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering. ACM, New York, NY, USA, 16--22.Google ScholarGoogle Scholar
  94. Anthony Rowe, Dimitrios Kalaitzopoulos, Michelle Osmond, Moustafa Ghanem, and Yike Guo. 2003. The discovery net system for high throughput bioinformatics. Bioinformatics, Vol. 19, suppl 1 (2003), i225--i231.Google ScholarGoogle ScholarCross RefCross Ref
  95. Banani Roy, Amit Kumar Mondal, Chanchal K Roy, Kevin A Schneider, and Kawser Wazed. 2017. Towards a reference architecture for cloud-based plant genotyping and phenotyping analysis frameworks. In 2017 IEEE International Conference on Software Architecture (ICSA). IEEE, 41--50.Google ScholarGoogle ScholarCross RefCross Ref
  96. Simon P Sadedin, Bernard Pope, and Alicia Oshlack. 2012. Bpipe: a tool for running and managing bioinformatics pipelines. Bioinformatics, Vol. 28, 11 (2012), 1525--1526.Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. Hiroaki Sakai et al. 2013. Rice Annotation Project Database (RAP-DB): an integrative and interactive database for rice genomics. Plant and Cell Physiology, Vol. 54, 2 (2013), e6--e6.Google ScholarGoogle ScholarCross RefCross Ref
  98. Jeff Sauro. 2011. Measuring usability with the system usability scale (SUS).Google ScholarGoogle Scholar
  99. Patrick D Schloss, Sarah L Westcott, Thomas Ryabin, Justine R Hall, Martin Hartmann, Emily B Hollister, Ryan A Lesniewski, Brian B Oakley, Donovan H Parks, Courtney J Robinson, et al. 2009. Introducing mothur: open-source, platform-independent, community-supported software for describing and comparing microbial communities. Appl. Environ. Microbiol., Vol. 75, 23 (2009), 7537--7541.Google ScholarGoogle ScholarCross RefCross Ref
  100. Helen Schonenberg, Ronny Mans, Nick Russell, Nataliya Mulyar, and Wil van der Aalst. 2008. Process flexibility: A survey of contemporary approaches. In Advances in enterprise engineering I. Springer, New York, NY, 16--30.Google ScholarGoogle Scholar
  101. Clare Sloggett, Nuwan Goonasekera, and Enis Afgan. 2013. BioBlend: automating pipeline analyses within Galaxy and CloudMan. Bioinformatics, Vol. 29, 13 (2013), 1685--1686.Google ScholarGoogle Scholar
  102. Barry Smith, Michael Ashburner, Cornelius Rosse, Jonathan Bard, William Bug, Werner Ceusters, Louis J Goldberg, Karen Eilbeck, Amelia Ireland, Christopher J Mungall, et al. 2007. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nature biotechnology, Vol. 25, 11 (2007), 1251.Google ScholarGoogle Scholar
  103. Kenia Sousa, Jean Vanderdonckt, Brian Henderson-Sellers, et al. 2012. Evaluating a graphical notation for modelling software development methodologies. Journal of Visual Languages & Computing, Vol. 23, 4 (2012), 195--212.Google ScholarGoogle ScholarDigital LibraryDigital Library
  104. Jared Spool and Will Schroeder. 2001. Testing web sites: Five users is nowhere near enough. In CHI'01 extended abstracts on Human factors in computing systems. ACM, New York, NY, USA, 285--286.Google ScholarGoogle ScholarDigital LibraryDigital Library
  105. Jonathan Sprinkle, Marjan Mernik, Juha-Pekka Tolvanen, and Diomidis Spinellis. 2009. Guest editors' introduction: What kinds of nails need a domain-specific hammer? IEEE software, Vol. 26, 4 (2009), 15--18.Google ScholarGoogle Scholar
  106. Robert Tairas, Marjan Mernik, et al. 2008. Using ontologies in the domain analysis of domain-specific languages. In International Conference on Model Driven Engineering Languages and Systems. Springer, New York, NY, 332--342.Google ScholarGoogle Scholar
  107. Gabor Terstyanszky, Tamas Kukla, Tamas Kiss, Peter Kacsuk, et al. 2014. Enabling scientific workflow sharing through coarse-grained interoperability. Future Generation Computer Systems, Vol. 37 (2014), 46--59.Google ScholarGoogle ScholarCross RefCross Ref
  108. Tom Tullis and Bill Albert. 2013. Chapter 6 - Self-Reported Metrics. In Measuring the User Experience (Second Edition) second edition ed.), Tom Tullis and Bill Albert (Eds.). Morgan Kaufmann, Boston, 121 -- 161. https://doi.org/10.1016/B978-0--12--415781--1.00006--6Google ScholarGoogle Scholar
  109. Jan Van den Bergh and Kris Luyten. 2017. DICE-R: Defining human-robot interaction with composite events. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems. ACM, 117--122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  110. Wil van der Aalst and Arthur ter Hofstede. 1999. Workflow Patterns Initiative. http://www.workflowpatterns.com Retrieved December 12, 2019 fromGoogle ScholarGoogle Scholar
  111. Wil van der Aalst and Arthur ter Hofstede. 2005. YAWL: yet another workflow language. Information systems, Vol. 30, 4 (2005), 245--275.Google ScholarGoogle Scholar
  112. Vinod Kumar Vavilapalli, Arun C Murthy, Chris Douglas, Sharad Agarwal, Mahadev Konar, Robert Evans, Thomas Graves, Jason Lowe, Hitesh Shah, Siddharth Seth, et al. 2013. Apache hadoop yarn: Yet another resource negotiator. In Proceedings of the 4th annual Symposium on Cloud Computing. ACM, New York, NY, USA, 5.Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. Markus Völter, Sebastian Benz, Christian Dietrich, et al. 2013. DSL Engineering - Designing, Implementing and Using Domain-Specific Languages .dslbook.org. http://www.dslbook.orgGoogle ScholarGoogle Scholar
  114. Tom White. 2009. Hadoop: The Definitive Guide 1st ed.). O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  115. Guido Wirtz, Mathias Weske, and Holger Giese. 2000. Extending UML with Workflow Modeling Capabilities .Springer Berlin Heidelberg, Berlin, Heidelberg, 30--41. https://doi.org/10.1007/10722620_3Google ScholarGoogle ScholarCross RefCross Ref
  116. Katherine Wolstencroft, Robert Haines, Fellows, et al. 2013. The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic acids research, Vol. 41, W1 (2013), W557--W561.Google ScholarGoogle Scholar
  117. Xiaorong Xiang and Gregory Madey. 2007. Improving the reuse of ScientificWorkflows and their by-products. In IEEE International Conference on Web Services (ICWS 2007). IEEE, 792--799.Google ScholarGoogle ScholarCross RefCross Ref
  118. Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster computing with working sets. HotCloud, Vol. 10, 10--10 (2010), 95.Google ScholarGoogle ScholarDigital LibraryDigital Library
  119. Matei Zaharia, Reynold S Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J Franklin, et al. 2016. Apache spark: a unified engine for big data processing. Commun. ACM, Vol. 59, 11 (2016), 56--65.Google ScholarGoogle ScholarDigital LibraryDigital Library
  120. Jiajie Zhang, Kassian Kobert, Tomávs Flouri, and Alexandros Stamatakis. 2013. PEAR: a fast and accurate Illumina Paired-End reAd mergeR. Bioinformatics, Vol. 30, 5 (2013), 614--620.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. VizSciFlow: A Visually Guided Scripting Framework for Supporting Complex Scientific Data Analysis

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!