skip to main content
research-article
Open Access

Fixpoints for the masses: programming with first-class Datalog constraints

Published:13 November 2020Publication History
Skip Abstract Section

Abstract

Datalog is a declarative logic programming language that has been used in a variety of applications, including big-data analytics, language processing, networking and distributed systems, and program analysis.

In this paper, we propose first-class Datalog constraints as a mechanism to construct, compose, and solve Datalog programs at run time. The benefits are twofold: We gain the full power of a functional programming language to operate on Datalog constraints-as-values, while simultaneously we can use Datalog where it really shines: to declaratively express and solve fixpoint problems.

We present an extension of the lambda calculus with first-class Datalog constraints, including its semantics and a type system with row polymorphism based on Hindley-Milner. We prove soundness of the type system and implement it as an extension of the Flix programming language.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

Datalog is a declarative logic programming language that has been used in a variety of applications, including big-data analytics, language processing, networking and distributed systems, and program analysis. In this paper, we propose first-class Datalog constraints as a mechanism to construct, compose, and solve Datalog programs at run time. The benefits are twofold: We gain the full power of a functional programming language to operate on Datalog constraints-as-values, while simultaneously we can use Datalog where it really shines: to declaratively express and solve fixpoint problems. We present an extension of the lambda calculus with first-class Datalog constraints, including its semantics and a type system with row polymorphism based on Hindley-Milner. We prove soundness of the type system and implement it as an extension of the Flix programming language.

References

  1. Peter Alvaro, Neil Conway, Joseph M Hellerstein, and William R Marczak. 2011. Consistency Analysis in Bloom: a CALM and Collected Approach. In Proc. Conference on Innovative Data Systems (CIDR).Google ScholarGoogle Scholar
  2. Peter Alvaro, William R Marczak, Neil Conway, Joseph M Hellerstein, David Maier, and Russell Sears. 2010. Dedalus: Datalog in time and space. In International Datalog 2.0 Workshop.Google ScholarGoogle Scholar
  3. Michael Arntzenius and Neel Krishnaswami. 2019. Seminaïve evaluation for a higher-order functional language. Proc. of ACM on Programming Languages Principles of Programming Languages (POPL) ( 2019 ).Google ScholarGoogle Scholar
  4. Michael Arntzenius and Neelakantan R Krishnaswami. 2016. Datafun: a functional Datalog. In Proc. International Conference on Functional Programming.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Pavel Avgustinov, Oege De Moor, Michael Peyton Jones, and Max Schäfer. 2016. QL: Object-oriented queries on relational data. In Proc. European Conference on Object-Oriented Programming (ECOOP 2016 ).Google ScholarGoogle Scholar
  6. Francois Bancilhon, David Maier, Yehoshua Sagiv, and Jefrey D Ullman. 1985. Magic Sets and Other Strange Ways to Implement Logic Programs. In Proc. Principles of Database Systems (PODS). https://doi.org/10.1145/6012.15399 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Aaron Bembenek and Stephen Chong. 2018. FormuLog: Datalog for static analysis involving logical formulae. arXiv preprint arXiv: 1809. 06274 ( 2018 ).Google ScholarGoogle Scholar
  8. Martin Bravenboer and Yannis Smaragdakis. 2009. Strictly Declarative Specification of Sophisticated Points-To Analyses. In Proc. Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). https://doi.org/10.1145/1640089. 1640108 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Stefano Ceri, Georg Gottlob, and Letizia Tanca. 1989. What You Always Wanted to Know About Datalog (and Never Dared to Ask). IEEE Transactions on Knowledge and Data Engineering (TKDE) ( 1989 ). https://doi.org/10.1109/69.43410 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Stefano Ceri, Georg Gottlob, and Letizia Tanca. 2012. Logic programming and databases. Springer Science & Business Media.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Jacques Cohen. 1990. Constraint Logic Programming Languages. Commun. ACM ( 1990 ). https://doi.org/10.1145/79204.79209 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Neil Conway, William R Marczak, Peter Alvaro, Joseph M Hellerstein, and David Maier. 2012. Logic and Lattices for Distributed Programming. In Proc. Symposium on Cloud Computing (SoCC). https://doi.org/10.1145/2391229.2391230 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Luis Damas and Robin Milner. 1982. Principal Type-Schemes for Functional Programs. In Proc. Symposium on Principles of Programming Languages (POPL).Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. O de Moor, G Gottlob, T Furche, and AJ Sellers (Eds.). 2011. Datalog Reloaded-First International Workshop, Datalog 2010. https://doi.org/10.1007/978-3-642-24206-9 Google ScholarGoogle ScholarCross RefCross Ref
  15. Matthias Felleisen, Robert Bruce Findler, and Matthew Flatt. 2009. Semantics Engineering with PLT Redex.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Melvin Fitting. 2002. Fixpoint Semantics for Logic Programming a Survey. Theoretical Computer Science (TCS) ( 2002 ). https://doi.org/10.1016/S0304-3975 ( 00 ) 00330-3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Michael Gelfond and Vladimir Lifschitz. 1988. The Stable Model Semantics for Logic Programming. In Proc. International Conference on Logic Programming (ICLP/SLP).Google ScholarGoogle Scholar
  18. Michael Gelfond and Vladimir Lifschitz. 1991. Classical Negation in Logic Programs and Disjunctive Databases. New Generation Computing ( 1991 ). https://doi.org/10.1007/BF03037169 Google ScholarGoogle ScholarCross RefCross Ref
  19. Goetz Graefe. 1993. Query Evaluation Techniques for Large Databases. ACM Computing Surveys (CSUR) ( 1993 ). https: //doi.org/10.1145/152610.152611 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Steve Gregory. 1987. Parallel Logic Programming in PARLOG: The Language and its Implementation. Addison-Wesley.Google ScholarGoogle Scholar
  21. Elnar Hajiyev, Mathieu Verbaere, and Oege De Moor. 2006. codeQuest: Scalable Source Code Queries with Datalog. In Proc. European Conference on Object-Oriented Programming (ECOOP). https://doi.org/10.1007/11785477_2 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Daniel Halperin, Victor Teixeira de Almeida, Lee Lee Choo, Shumo Chu, Paraschos Koutris, Dominik Moritz, Jennifer Ortiz, Vaspol Ruamviboonsuk, Jingjing Wang, Andrew Whitaker, et al. 2014. Demonstration of the Myria big data management service. In Proc. International Conference on Management of Data.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Fergus Henderson, Thomas Conway, Zoltan Somogyi, David Jefery, Peter Schachte, Simon Taylor, Chris Speirs, Tyson Dowd, Ralph Becket, and Mark Brown. 1996. The Mercury language reference manual. ( 1996 ).Google ScholarGoogle Scholar
  24. Shan Shan Huang, Todd Jefrey Green, and Boon Thau Loo. 2011. Datalog and Emerging Applications: An Interactive Tutorial. In Proc. Management of Data (SIGMOD). https://doi.org/10.1145/1989323.1989456 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Joxan Jafar and Jean-Louis Lassez. 1987. Constraint Logic Programming. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/41625.41635 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Joxan Jafar and Michael J Maher. 1994. Constraint Logic Programming: A Survey. Journal of Logic Programming ( 1994 ). https://doi.org/10.1016/ 0743-1066 ( 94 ) 90033-7 Google ScholarGoogle ScholarCross RefCross Ref
  27. Herbert Jordan, Bernhard Scholz, and Pavle Subotić. 2016. Souflé: On Synthesis of Program Analyzers. In International Conference on Computer Aided Verification.Google ScholarGoogle Scholar
  28. Herbert Jordan, Bernhard Scholz, and Pavle Subotic. 2018. Two concurrent data structures for eficient datalog query processing. In Proc. Symposium on Principles and Practice of Parallel Programming.Google ScholarGoogle Scholar
  29. George Kastrinis and Yannis Smaragdakis. 2013. Hybrid context-sensitivity for points-to analysis. Proc. International Conference on Programming Language Design and Implementation (PLDI) ( 2013 ).Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Ross D King. 2004. Applying Inductive Logic Programming to Predicting Gene Function. AI Magazine ( 2004 ).Google ScholarGoogle Scholar
  31. Kenneth Kunen. 1987. Negation in Logic Programming. Journal of Logic Programming ( 1987 ). https://doi.org/10.1016/ 0743-1066 ( 87 ) 90007-0 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Monica S Lam, John Whaley, V Benjamin Livshits, Michael C Martin, Dzintars Avots, Michael Carbin, and Christopher Unkel. 2005. Context-sensitive Program Analysis as Database Queries. In Proc. Principles of Database Systems (PODS). https://doi.org/10.1145/1065167.1065169 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Daan Leijen. 2005. Extensible records with scoped labels. Trends in Functional Programming ( 2005 ).Google ScholarGoogle Scholar
  34. Ninghui Li and John C Mitchell. 2003. Datalog with Constraints: A Foundation for Trust Management Languages. In Proc. Practical Aspects of Declarative Languages (PADL). https://doi.org/10.1007/3-540-36388-2_6 Google ScholarGoogle ScholarCross RefCross Ref
  35. Boon Thau Loo, Tyson Condie, Minos Garofalakis, David E Gay, Joseph M Hellerstein, Petros Maniatis, Raghu Ramakrishnan, Timothy Roscoe, and Ion Stoica. 2009. Declarative networking. Commun. ACM ( 2009 ).Google ScholarGoogle Scholar
  36. Magnus Madsen and Ondřej Lhoták. 2020. Fixpoints for the Masses: Programming with First-class Datalog Constraints. Technical Report CS-2020-05. University of Waterloo. https://cs.uwaterloo.ca/sites/ca.computer-science/files/uploads/ ifles/cs-2020-05.pdfGoogle ScholarGoogle Scholar
  37. Magnus Madsen, Ming-Ho Yee, and Ondřej Lhoták. 2016. From Datalog to Flix: A Declarative Language for Fixed Points on Lattices. In Programming Language Design and Implementation (PLDI).Google ScholarGoogle Scholar
  38. Jack Minker. 1988. Foundations of Deductive Databases and Logic Programming. Morgan Kaufmann.Google ScholarGoogle Scholar
  39. Raymond J Mooney. 1996. Inductive Logic Programming for Natural Language Processing. In International Conference on Inductive Logic Programming.Google ScholarGoogle Scholar
  40. Christos H. Papadimitriou. 1985. A note the expressive power of Prolog. Bulletin of the European Association for Theoretical Computer Science (EATCS) ( 1985 ).Google ScholarGoogle Scholar
  41. Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise Interprocedural Dataflow Analysis via Graph Reachability. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/199448.199462 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Mooly Sagiv, Thomas Reps, and Susan Horwitz. 1996. Precise Interprocedural Dataflow Analysis with Applications to Constant Propagation. Theoretical Computer Science (TCS) ( 1996 ). https://doi.org/10.1016/ 0304-3975 ( 96 ) 00072-2 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Max Schäfer and Oege de Moor. 2010. Type inference for datalog with complex type hierarchies. In Proc. Symposium on Principles of Programming Languages (POPL).Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Bernhard Scholz, Herbert Jordan, Pavle Subotić, and Till Westmann. 2016. On fast large-scale program analysis in datalog. In Proc. International Conference on Compiler Construction (CC).Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jiwon Seo, Stephen Guo, and Monica S Lam. 2013. SociaLite: Datalog extensions for eficient social network analysis. In International Conference Data Engineering (ICDE).Google ScholarGoogle Scholar
  46. Alexander Shkapsky, Mohan Yang, Matteo Interlandi, Hsuan Chiu, Tyson Condie, and Carlo Zaniolo. 2016. Big Data Analytics with Datalog Queries on Spark. In Proc. International Conference on Management of Data.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yannis Smaragdakis, George Balatsouras, and George Kastrinis. 2013. Set-based pre-processing for points-to analysis. In Proc. International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA).Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Yannis Smaragdakis and Martin Bravenboer. 2011. Using Datalog for Fast and Easy Program Analysis. In Datalog Reloaded. https://doi.org/10.1145/1926385.1926390 Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Yannis Smaragdakis, Martin Bravenboer, and Ondrej Lhoták. 2011. Pick Your Contexts Well: Understanding Object-Sensitivity. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/1925844.1926390 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Yannis Smaragdakis, George Kastrinis, and George Balatsouras. 2014. Introspective analysis: context-sensitivity, across the board. Proc. International Conference on Programming Language Design and Implementation (PLDI).Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Zoltan Somogyi, Fergus Henderson, and Thomas Conway. 1996. The Execution Algorithm of Mercury, an Eficient Purely Declarative Logic Programming Language. The Journal of Logic Programming ( 1996 ).Google ScholarGoogle Scholar
  52. Zoltan Somogyi, Fergus J Henderson, and Thomas Charles Conway. 1995. Mercury, an eficient purely declarative logic programming language. Australian Computer Science Communications ( 1995 ).Google ScholarGoogle Scholar
  53. Soufle Authors. 2018. Soufle. https://soufle-lang.github.io/ [Online; accessed 18-October-2018].Google ScholarGoogle Scholar
  54. Pavle Subotic, Herbert Jordan, Lijun Chang, Alan Fekete, and Bernhard Scholz. 2018. Automatic Index Selection for Large-Scale Datalog Computation. ( 2018 ).Google ScholarGoogle Scholar
  55. Jefrey D Ullman. 1984. Principles of Database Systems. Galgotia publications.Google ScholarGoogle Scholar
  56. Jefrey D. Ullman. 1988. Principles of Database and Knowledge-Base Systems.Google ScholarGoogle Scholar
  57. Todd L Veldhuizen. 2012. Leapfrog Triejoin: a worst-case optimal join algorithm. arXiv preprint arXiv:1210.0481 ( 2012 ).Google ScholarGoogle Scholar
  58. Andrew K Wright and Matthias Felleisen. 1994. A Syntactic Approach to Type Soundness. Information and computation ( 1994 ).Google ScholarGoogle Scholar
  59. David Zook, Emir Pasalic, and Beata Sarna-Starosta. 2009. Typed datalog. In International Symposium on Practical Aspects of Declarative Languages (PADL).Google ScholarGoogle Scholar

Index Terms

  1. Fixpoints for the masses: programming with first-class Datalog constraints

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Programming Languages
        Proceedings of the ACM on Programming Languages  Volume 4, Issue OOPSLA
        November 2020
        3108 pages
        EISSN:2475-1421
        DOI:10.1145/3436718
        Issue’s Table of Contents

        Copyright © 2020 Owner/Author

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 13 November 2020
        Published in pacmpl Volume 4, Issue OOPSLA

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!