skip to main content

Digging for fold: synthesis-aided API discovery for Haskell

Published:13 November 2020Publication History
Skip Abstract Section

Abstract

We present Hoogle+, a web-based API discovery tool for Haskell. A Hoogle+ user can specify a programming task using either a type, a set of input-output tests, or both. Given a specification, the tool returns a list of matching programs composed from functions in popular Haskell libraries, and annotated with automatically-generated examples of their behavior. These features of Hoogle+ are powered by three novel techniques. First, to enable efficient type-directed synthesis from tests only, we develop an algorithm that infers likely type specifications from tests. Second, to return high-quality programs even with ambiguous specifications, we develop a technique that automatically eliminates meaningless and repetitive synthesis results. Finally, we show how to extend this elimination technique to automatically generate informative inputs that can be used to demonstrate program behavior to the user. To evaluate the effectiveness of Hoogle+ compared with traditional API search techniques, we perform a user study with 30 participants of varying Haskell proficiency. The study shows that programmers equipped with Hoogle+ generally solve tasks faster and were able to solve 50% more tasks overall.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

We present Hoogle+, a component-based synthesizer for Haskell, whose input specification can be a type, a set of input-output tests, or both, and returns a list of programs composed from popular Haskell libraries. Hoogle+ focuses on the end-to-end usability of synthesis. Usability depends on whether and how the programmer can specify their own intent and comprehend the tools results. Hoogle+ addresses the specification problem by allowing the user to specify goals via tests from which it automatically infers type signatures that aid rapid synthesis. Hoogle+ uses property based testing to ensure the user only sees useful and unique candidates, and generates example usages to show the functionality of each program helping the user comprehend them. We evaluate Hoogle+ with a user study with 30 participants of varying Haskell proficiency, showing programmers generally solve tasks faster with Hoogle+, and they were able to solve 50% more tasks when equipped with Hoogle+.

References

  1. Jong-hoon (David) An, Avik Chaudhuri, Jefrey S. Foster, and Michael Hicks. 2011. Dynamic inference of static types for ruby. In POPL. Austin, TX, USA, January 26-28, 2011, Thomas Ball and Mooly Sagiv (Eds.). ACM, 459-472.Google ScholarGoogle Scholar
  2. Lennart Augusstson. 2005. Djinn. https://github.com/augustss/djinn.Google ScholarGoogle Scholar
  3. Ambrose Bonnaire-Sergeant. 2019. Typed Clojure in Theory and Practice. Ph.D. Dissertation. Indiana University, Bloomington.Google ScholarGoogle Scholar
  4. Sarah E Chasins, Maria Mueller, and Rastislav Bodik. 2018. Rousillon: Scraping Distributed Hierarchical Web Data. In UIST 2018. 963-975.Google ScholarGoogle Scholar
  5. Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In PLDI '16 (PLDI '16). Association for Computing Machinery, New York, NY, USA, 341-354.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Ravi Chugh, Sorin Lerner, and Ranjit Jhala. 2011. Type Inference with Run-time Logs. In Workshop on Scripts to Programs (STOP).Google ScholarGoogle Scholar
  7. Koen Claessen and John Hughes. 2000. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00 (ICFP '00). Association for Computing Machinery, 268-279.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Nils Anders Danielsson and Patrik Jansson. 2004. Chasing Bottoms. In Mathematics of Program Construction (Lecture Notes in Computer Science), Dexter Kozen (Ed.). Springer, 85-109.Google ScholarGoogle Scholar
  9. Roberto Di Cosmo. 1993. Deciding Type isomorphisms in a type assignment framework. Journal of Functional Programming 3, 3 ( 1993 ), 485-525. http://www.dicosmo.org/Articles/JFP93.pdf Special Issue on ML.Google ScholarGoogle ScholarCross RefCross Ref
  10. Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. In CHI 2020. Association for Computing Machinery, New York, NY, USA, 1-12.Google ScholarGoogle Scholar
  11. Yu Feng, Ruben Martins, Yuepeng Wang, Isil Dillig, and Thomas W. Reps. 2017. Component-based synthesis for complex APIs. In POPL 2017.Google ScholarGoogle Scholar
  12. Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. In POPL 2011, Austin, TX, USA, January 26-28, 2011. 317-330.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Zheng Guo, Michael James, David Justo, Jiaxiao Zhou, Ziteng Wang, Ranjit Jhala, and Nadia Polikarpova. 2020. Program synthesis by type-guided abstraction refinement. Proc. ACM Program. Lang. 4, POPL ( 2020 ), 12 : 1-12 : 28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Tihomir Gvero, Viktor Kuncak, Ivan Kuraj, and Ruzica Piskac. 2013. Complete completion using types and weights. In PLDI 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. George T. Heineman, Jan Bessai, Boris Düdder, and Jakob Rehof. 2016. A Long and Winding Road Towards Modular Synthesis. In ISoLA 2016, Imperial, Corfu, Greece, October 10-14, 2016, Proceedings, Part I. 303-317.Google ScholarGoogle Scholar
  16. Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-guided component-based program synthesis. In ICSE '10, Vol. 1. ACM Press, 215. http://portal.acm.org/citation.cfm?doid= 1806799. 1806833Google ScholarGoogle Scholar
  17. Vu Le, Daniel Perelman, Oleksandr Polozov, Mohammad Raza, Abhishek Udupa, and Sumit Gulwani. 2017. Interactive Program Synthesis. CoRR abs/1703.03539 ( 2017 ). arXiv: 1703.03539 http://arxiv.org/abs/1703.03539Google ScholarGoogle Scholar
  18. David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid Mining: Helping to Navigate the API Jungle. In PLDI 2005.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Anders Miltner, Solomon Maina, Kathleen Fisher, Benjamin C. Pierce, David Walker, and Steve Zdancewic. 2019. Synthesizing Symmetric Lenses. Proc. ACM Program. Lang. 3, ICFP 2019, Article Article 95 ( July 2019 ), 28 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Neil Mitchell. 2004. Hoogle. https://www.haskell.org/hoogle/.Google ScholarGoogle Scholar
  21. Ulf Norell. 2008. Dependently Typed Programming in Agda. In AFP 2008, Heijen, The Netherlands, May 2008, Revised Lectures. 230-266.Google ScholarGoogle Scholar
  22. Hila Peleg, Sharon Shoham, and Eran Yahav. 2018. Programming not only by example. In ICSE 2018. ACM, 1114-1124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Daniel Perelman, Sumit Gulwani, Thomas Ball, and Dan Grossman. 2012. Type-directed completion of partial expressions. In PLDI '12, Beijing, China-June 11-16, 2012. 275-286.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Gordon Plotkin. 1970. Lattice Theoretic Properties of Subsumption. Edinburgh University, Department of Machine Intelligence and Perception. https://books.google.com/books?id=2p09cgAACAAJGoogle ScholarGoogle Scholar
  25. Oleksandr Polozov and Sumit Gulwani. 2015. Flashmeta: A framework for inductive program synthesis. ACM SIGPLAN Notices 50, 10 ( 2015 ), 107-126.Google ScholarGoogle Scholar
  26. Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code Completion with Statistical Language Models. SIGPLAN Not. 49, 6 ( June 2014 ), 419-428.Google ScholarGoogle Scholar
  27. John C. Reynolds. 1969. Transformational systems and the algebraic structure of atomic for-mulas.Google ScholarGoogle Scholar
  28. Colin Runciman, Matthew Naylor, and Fredrik Lindblad. 2008. Smallcheck and lazy smallcheck: automatic exhaustive testing for small values. In Haskell Symposium 2008 (Haskell '08). Association for Computing Machinery, 37-48.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Rishabh Singh and Sumit Gulwani. 2015. Predicting a Correct Program in Programming by Example. In CAV-27th International Conference, 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part I. 398-414.Google ScholarGoogle ScholarCross RefCross Ref
  30. Pawel Urzyczyn. 1997. Inhabitation in Typed Lambda-Calculi (A Syntactic Approach). In TLCA '97, Nancy, France, April 2-4, 1997, Proceedings. 373-389.Google ScholarGoogle ScholarCross RefCross Ref
  31. Philip Wadler and Stephen Blott. 1989. How to Make ad-hoc Polymorphism Less ad-hoc. In POPL 1989, Austin, Texas, USA, January 11-13, 1989. 60-76.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Digging for fold: synthesis-aided API discovery for Haskell

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!