skip to main content
research-article
Open Access

Creating Interactive Scientific Publications using Bindings

Published:13 June 2019Publication History
Skip Editorial Notes Section

Editorial Notes

The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected VoR was published on April 5, 2023. For reference purposes the VoR may still be accessed via the Supplemental Material section on this page.

Skip Abstract Section

Abstract

Many scientific publications report on computational results based on code and data, but even when code and data are published, the main text is usually provided in a separate, traditional format such as PDF. Since code, data, and text are not linked on a deep level, it is difficult for readers and reviewers to understand and retrace how the authors achieved a specific result that is reported in the main text, e.g. a figure, table, or number. In addition, to make use of new the opportunities afforded by data and code availability, such as re-running analyses with changed parameters, considerable effort is required. In order to overcome this issue and to enable more interactive publications that support scientists in more deeply exploring the reported results, we present the concept, implementation, and initial evaluation of bindings. A binding describes which data subsets, code lines, and parameters produce a specific result that is reported in the main text (e.g. a figure or number). Based on a prototypical implementation of these bindings, we propose a toolkit for authors to easily create interactive figures by connecting specific UI widgets (e.g. a slider) to parameters. In addition to inspecting code and data, readers can then manipulate the parameter and see how the results change. We evaluated the approach by applying it to a set of existing articles. The results provide initial evidence that the concept is feasible and applicable to many papers with moderate effort.

Skip Supplemental Material Section

Supplemental Material

References

  1. Muhammad Adnan, Mike Just, and Lynne Baillie. 2016. Investigating Time Series Visualisations to Improve the User Experience. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 5444--5455. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. JJ Allaire, Yihui Xie, Jonathan McPherson, Javier Luraschi, Kevin Ushey, Aron Atkins, Hadley Wickham, Joe Cheng, and Winston Chang. 2018. rmarkdown: Dynamic Documents for R. https://CRAN.R-project.org/package=rmarkdown R package version 1.10. Accessed: 2019-02--21.Google ScholarGoogle Scholar
  3. Giuliano Antoniol, Gerardo Canfora, Gerardo Casazza, Andrea De Lucia, and Ettore Merlo. 2002. Recovering traceability links between code and documentation. IEEE transactions on software engineering 28, 10 (2002), 970--983. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Alberto Bacchelli, Michele Lanza, and Romain Robbes. 2010. Linking e-mails and source code artifacts. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 1. ACM, 375--384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Nick Barnes. 2010. Publish your computer code: it is good enough. Nature News 467, 7317 (2010), 753--753.Google ScholarGoogle ScholarCross RefCross Ref
  6. Winston Chang, Joe Cheng, JJ Allaire, Yihui Xie, and Jonathan McPherson. 2018. shiny: Web Application Framework for R. https://CRAN.R-project.org/package=shiny R package version 1.1.0. Accessed: 2019-02--21.Google ScholarGoogle Scholar
  7. Fernando Chirigati, Rémi Rampin, Dennis Shasha, and Juliana Freire. 2016. Reprozip: Computational reproducibility with ease. In Proceedings of the 2016 International Conference on Management of Data. ACM, 2085--2088.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Julien Colomb and Björn Brembs. 2014. Sub-strains of Drosophila Canton-S differ markedly in their locomotor behavior. F1000Research 3 (2014).Google ScholarGoogle Scholar
  9. Benjamin M. Delory, Mao Li, Christopher N. Topp, and Guillaume Lobet. 2018. archiDART v3. 0: A new data analysis pipeline allowing the topological analysis of plant root systems. F1000Research 7 (2018).Google ScholarGoogle Scholar
  10. Francesco Dottori, Rui Figueiredo, Mario LV Martina, Daniela Molinari, A. Scorzini, et al. 2016. INSYDE: a synthetic, probabilistic flood damage model based on explicit cost analysis. Natural Hazards and Earth System Sciences 16 (2016), 2577--2591.Google ScholarGoogle ScholarCross RefCross Ref
  11. Pierre Dragicevic, Yvonne Jansen, Abhraneel Sarma, Matthew Kay, and Fanny Chevalier. 2019. Increasing the Transparency of Research Papers with Explorable Multiverse Analyses. In CHI 2019 - The ACM CHI Conference on Human Factors in Computing Systems. ACM.Google ScholarGoogle Scholar
  12. Bastian Drees, Angelina Kraft, and Thomas Koprucki. 2018. Reproducible and comprehensible research results through persistently linked and visualized numerical simulation data. Optical and Quantum Electronics 50, 2 (2018), 59.Google ScholarGoogle ScholarCross RefCross Ref
  13. Steve M Easterbrook. 2014. Open code for open science? Nature Geoscience 7, 11 (2014), 779.Google ScholarGoogle ScholarCross RefCross Ref
  14. Florian Echtler and Maximilian Häußler. 2018. Open Source, Open Science, and the Replication Crisis in HCI. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, alt02. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Gabriel Fils, Zhihao Yuan, Tanu Malik, et al. 2017. Sciunits: Reusable Research Objects. In e-Science (e-Science), 2017 IEEE 13th International Conference on. IEEE, 374--383.Google ScholarGoogle Scholar
  16. Robert Gentleman and Duncan Temple Lang. 2007. Statistical analyses and reproducible research. Journal of Computational and Graphical Statistics 16, 1 (2007), 1--23.Google ScholarGoogle ScholarCross RefCross Ref
  17. Virginia Gewin. 2016. Data sharing: An open mind on open data. Nature 529, 7584 (2016), 117--119.Google ScholarGoogle Scholar
  18. Yolanda Gil, Varun Ratnakar, and Daniel Garijo. 2015. OntoSoft: Capturing scientific software metadata. In Proceedings of the 8th International Conference on Knowledge Capture. ACM, 32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jeremy Goecks, Anton Nekrutenko, and James Taylor. 2010. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome biology 11, 8 (2010), R86.Google ScholarGoogle Scholar
  20. Steven N. Goodman, Daniele Fanelli, and John P. A. Ioannidis. 2016. What does research reproducibility mean? Science translational medicine 8, 341 (2016), 341ps12--341ps12.Google ScholarGoogle Scholar
  21. Miriam Greis, Hendrik Schuff, Marius Kleiner, Niels Henze, and Albrecht Schmidt. 2017. Input Controls for Entering Uncertain Data: Probability Distribution Sliders. Proceedings of the ACM on Human-Computer Interaction 1, 1 (2017), 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Stevan Harnad, Tim Brody, Francois Vallieres, Les Carr, Steve Hitchcock, Yves Gingras, Charles Oppenheim, Heinrich Stamerjohanns, and Eberhard R. Hilf. 2004. The access/impact problem and the green and gold roads to open access. Serials review 30, 4 (2004), 310--314.Google ScholarGoogle Scholar
  23. Anett Hoppe, Jascha Hagen, Helge Holzmann, Günter Kniesel, and Ralph Ewerth. 2018. An Analytics Tool for Exploring Scientific Software and Related Publications. In Digital Libraries for Open Knowledge, Eva Méndez, Fabio Crestani, Cristina Ribeiro, Gabriel David, and João Correia Lopes (Eds.). Springer International Publishing, Cham, 299--303.Google ScholarGoogle Scholar
  24. Tomi Kauppinen and Giovana Mira de Espindola. 2011. Linked Open Science-Communicating, Sharing and Evaluating Data, Methods and Results for Executable Papers.. In ICCS. 726--731.Google ScholarGoogle Scholar
  25. Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian E. Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica B. Hamrick, Jason Grout, Sylvain Corlay, et al. 2016. Jupyter Notebooks-a publishing format for reproducible computational workflows.. In ELPUB. 87--90.Google ScholarGoogle Scholar
  26. Donald Ervin Knuth. 1984. Literate programming. Comput. J. 27, 2 (1984), 97--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Markus Konkol and Christian Kray. 2018. In-depth examination of spatio-temporal figures in open reproducible research. Cartography and Geographic Information Science (2018).Google ScholarGoogle Scholar
  28. Markus Konkol, Christian Kray, and Max Pfeiffer. 2018. Computational reproducibility in geoscientific papers: Insights from a series of studies with geoscientists and a reproduction study. International Journal of Geographical Information Science (2018), 1--22.Google ScholarGoogle Scholar
  29. Friedrich Leisch. 2002. Sweave: Dynamic generation of statistical reports using literate data analysis. In Compstat. Springer, 575--580.Google ScholarGoogle Scholar
  30. Jon Mease. 2018. Bringing ipywidgets Support to plotly. py. (2018).Google ScholarGoogle Scholar
  31. Daniel Nüst, Carlos Granell, Barbara Hofer, Markus Konkol, Frank O. Ostermann, Rusne Sileryte, and Valentina Cerutti. 2018. Reproducible research and GIScience: an evaluation using AGILE conference papers. PeerJ 6 (2018), e5072.Google ScholarGoogle ScholarCross RefCross Ref
  32. Daniel Nüst, Markus Konkol, Edzer Pebesma, Christian Kray, Marc Schutzeichel, Holger Przibytzin, and Jörg Lorenz. 2017. Opening the publication process with executable research compendia. D-Lib Magazine 23, 1/2 (2017).Google ScholarGoogle ScholarCross RefCross Ref
  33. Jeroen Ooms. 2018. opencpu. https://CRAN.R-project.org/package=opencpu Accessed: 2019-02--21.Google ScholarGoogle Scholar
  34. Frank O. Ostermann and Carlos Granell. 2017. Advancing science with VGI: Reproducibility and replicability of recent studies using VGI. Transactions in GIS 21, 2 (2017), 224--237.Google ScholarGoogle ScholarCross RefCross Ref
  35. Jeffrey M. Perkel. 2018. Data visualization tools drive interactivity and reproducibility in online publishing. Nature 554, 7690 (2018), 133--134.Google ScholarGoogle Scholar
  36. Quan Pham, Tanu Malik, and Ian Foster. 2013. Using Provenance for Repeatability. TaPP 13 (2013), 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Quan Pham, Tanu Malik, Ian Foster, Roberto Di Lauro, and Raffaele Montella. 2012. SOLE: linking research papers with science objects. In International Provenance and Annotation Workshop. Springer, 203--208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. R Core Team. 2018. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org Accessed: 2019-02--21.Google ScholarGoogle Scholar
  39. Richard J Radke, Srinivas Andra, Omar Al-Kofahi, and Badrinath Roysam. 2005. Image change detection algorithms: a systematic survey. IEEE transactions on image processing 14, 3 (2005), 294--307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Thomas Rathfux, Roman Popp, and Hermann Kaindl. 2016. Adding custom widgets to model-driven GUI generation. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems. ACM, 16--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, and Eivind Hovig. 2013. Ten simple rules for reproducible computational research. PLoS computational biology 9, 10 (2013), e1003285.Google ScholarGoogle Scholar
  42. Carson Sievert, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2017. plotly: Create Interactive Web Graphics via 'plotly. js'. R package version 4.7. 1.Google ScholarGoogle Scholar
  43. Dalmeet Singh Chawla. 2015. 'Living figures' make their debut. Nature News 521, 7550 (2015), 112.Google ScholarGoogle ScholarCross RefCross Ref
  44. Stefan Steiniger and Erwan Bocher. 2009. An overview on current free and open source desktop GIS developments. International Journal of Geographical Information Science 23, 10 (2009), 1345--1370.Google ScholarGoogle ScholarCross RefCross Ref
  45. Victoria Stodden, Matthew S. Krafczyk, and Adhithya Bhaskar. 2018. Enabling the Verification of Computational Results: An Empirical Evaluation of Computational Reproducibility. In Proceedings of the First International Workshop on Practical Reproducible Evaluation of Computer Systems. ACM, 3.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Victoria Stodden, Marcia McNutt, David H. Bailey, Ewa Deelman, Yolanda Gil, Brooks Hanson, Michael A. Heroux, John P. A. Ioannidis, and Michela Taufer. 2016. Enhancing reproducibility for computational methods. Science 354, 6317 (2016), 1240--1241.Google ScholarGoogle Scholar
  47. Trestle Technology, LLC. 2018. plumber: An API Generator for R. https://CRAN.R-project.org/package=plumber R package version 0.4.6. Accessed: 2019-02--21.Google ScholarGoogle Scholar
  48. Ramnath Vaidyanathan, Yihui Xie, JJ Allaire, Joe Cheng, and Kenton Russell. 2016. htmlwidgets: HTML Widgets for R. https://CRAN.R-project.org/package=htmlwidgets R package version 1.3. Accessed: 2019-02--21.Google ScholarGoogle Scholar
  49. Bret Victor. 2018. Explorable Explanations. http://worrydream.com/#!/ExplorableExplanations. Accessed: 2019-02--21.Google ScholarGoogle Scholar
  50. Chat Wacharamanotham, Krishna Subramanian, Sarah Theres Völkel, and Jan Borchers. 2015. Statsplorer: Guiding novices in statistical analysis. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2693--2702. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. L.J. Welty, L.V. Rasmussen, A.S. Baldridge, and E. Whitley. 2016. StatTag. Galter Health Sciences Library, Chicago, Illinois, United States.Google ScholarGoogle Scholar
  52. Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, et al. 2016. The FAIR Guiding Principles for scientific data management and stewardship. Scientific data 3 (2016).Google ScholarGoogle Scholar
  53. Kanit Wongsuphasawat, Dominik Moritz, Arvind Satyanarayan, and Jeffrey Heer. 2018. Vega-Lite - A Grammar of Interactive Graphics. https://vega.github.io/vega-lite/. Accessed: 2019-02--21.Google ScholarGoogle Scholar

Index Terms

  1. Creating Interactive Scientific Publications using Bindings

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!