skip to main content
research-article
Open Access

A Gradual Probabilistic Lambda Calculus

Published:06 April 2023Publication History
Skip Abstract Section

Abstract

Probabilistic programming languages have recently gained a lot of attention, in particular due to their applications in domains such as machine learning and differential privacy. To establish invariants of interest, many such languages include some form of static checking in the form of type systems. However, adopting such a type discipline can be cumbersome or overly conservative. Gradual typing addresses this problem by supporting a smooth transition between static and dynamic checking, and has been successfully applied for languages with different constructs and type abstractions. Nevertheless, its benefits have never been explored in the context of probabilistic languages. In this work, we present and formalize GPLC, a gradual source probabilistic lambda calculus. GPLC includes a binary probabilistic choice operator and allows programmers to gradually introduce/remove static type–and probability–annotations. The static semantics of GPLC heavily relies on the notion of probabilistic couplings, as required for defining several relations, such as consistency, precision, and consistent transitivity. The dynamic semantics of GPLC is given via elaboration to the target language TPLC, which features a distribution-based semantics interpreting programs as probability distributions over final values. Regarding the language metatheory, we establish that TPLC–and therefore also GPLC–is type safe and satisfies two of the so-called refined criteria for gradual languages, namely, that it is a conservative extension of a fully static variant and that it satisfies the gradual guarantee, behaving smoothly with respect to type precision.

References

  1. Martin Avanzini, Ugo Dal Lago, and Alexis Ghyselen. 2021. Type-Based Complexity Analysis of Probabilistic Functional Programs. In Proceedings of the 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS ’19). IEEE Press, Article 41, 13 pages. Google ScholarGoogle Scholar
  2. Arthur Azevedo de Amorim, Matt Fredrikson, and Limin Jia. 2020. Reconciling Noninterference and Gradual Typing. In Proceedings of the 2020 Symposium on Logic in Computer Science (LICS 2020). Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Felipe Bañados Schwerter, Ronald Garcia, and Éric Tanter. 2016. Gradual Type-and-Effect Systems. Journal of Functional Programming, 26 (2016), Sept., 19:1–19:69. Google ScholarGoogle Scholar
  4. Gilles Barthe, Benjamin Grégoire, and Santiago Zanella-Béguelin. 2009. Formal Certification of Code-Based Cryptographic Proofs. In 36th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’09). ACM, New York. 90–101. Google ScholarGoogle Scholar
  5. Giuseppe Castagna and Victor Lanvin. 2017. Gradual Typing with Union and Intersection Types. Proceedings of the ACM on Programming Languages, 1, ICFP (2017), Sept., 41:1–41:28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Giuseppe Castagna, Victor Lanvin, Tommaso Petrucciani, and Jeremy G. Siek. 2019. Gradual typing: a new perspective. 16:1–16:32. Google ScholarGoogle Scholar
  7. Guillaume Claret, Sriram K. Rajamani, Aditya V. Nori, Andrew D. Gordon, and Johannes Borgström. 2013. Bayesian Inference using Data Flow Analysis. In Proceedings of the 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). ACM, 92–102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Vincent Danos and Thomas Ehrhard. 2011. Probabilistic coherence spaces as a model of higher-order probabilistic computation. Information and Computation, 209, 6 (2011), 966–991. issn:0890-5401 https://doi.org/10.1016/j.ic.2011.02.001 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Yuxin Deng and Wenjie Du. 2011. Logical, Metric, and Algorithmic Characterisations of Probabilistic Bisimulation. CoRR, abs/1103.4577 (2011), arXiv:1103.4577. arxiv:1103.4577 Google ScholarGoogle Scholar
  10. Tim Disney and Cormac Flanagan. 2011. Gradual information flow typing. In International Workshop on Scripts to Programs. Google ScholarGoogle Scholar
  11. Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci., 9, 3-4 (2014), 211–407. https://doi.org/10.1561/0400000042 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Thomas Ehrhard, Michele Pagani, and Christine Tasson. 2017. Measurable Cones and Stable, Measurable Functions: A Model for Probabilistic Higher-Order Programming. Proc. ACM Program. Lang., 2, POPL (2017), Article 59, dec, 28 pages. https://doi.org/10.1145/3158147 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Luminous Fennell and Peter Thiemann. 2013. Gradual Security Typing with References. In Proceedings of the 26th Computer Security Foundations Symposium (CSF). 224–239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ronald Garcia. 2013. Calculating threesomes, with blame. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional programming. 417–428. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ronald Garcia, Alison M. Clark, and Éric Tanter. 2016. Abstracting Gradual Typing. In Proceedings of the 43rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2016), Rastislav Bodík and Rupak Majumdar (Eds.). ACM Press, St Petersburg, FL, USA. 429–442. See erratum: https://www.cs.ubc.ca/ rxg/agt-erratum.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Zoubin Ghahramani. 2015. Probabilistic Machine Learning and Artificial Intelligence. Nature, 521, 7553 (2015), 452–459. Google ScholarGoogle Scholar
  17. Shafi Goldwasser and Silvio Micali. 1984. Probabilistic Encryption. J. Comput. Sys. Sci., 28, 2 (1984), 270–299. Google ScholarGoogle ScholarCross RefCross Ref
  18. Noah D. Goodman, Vikash K. Mansinghka, Daniel Roy, Keith Bonawitz, and Joshua B. Tenenbaum. 2008. Church: A Language for Generative Models. In Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI’08). AUAI Press, 220–229. isbn:0974903949 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Noah D Goodman and Andreas Stuhlmüller. 2014. The Design and Implementation of Probabilistic Programming Languages. http://dippl.org Accessed: 2022-10-17. Google ScholarGoogle Scholar
  20. Andrew D. Gordon, Thomas A. Henzinger, Aditya V. Nori, and Sriram K. Rajamani. 2014. Probabilistic programming. In Proceedings of the on Future of Software Engineering, FOSE 2014. ACM, 167–181. Google ScholarGoogle Scholar
  21. David Herman, Aaron Tomb, and Cormac Flanagan. 2007. Space-efficient gradual typing. In In Trends in Functional Programming (TFP. Google ScholarGoogle Scholar
  22. David Herman, Aaron Tomb, and Cormac Flanagan. 2010. Space-efficient gradual typing. Higher-Order and Sympolic Computation, 23, 2 (2010), June, 167–189. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Xuejing Huang and Bruno C. d. S. Oliveira. 2020. A Type-Directed Operational Semantics For a Calculus with a Merge Operator. In ECOOP. Google ScholarGoogle Scholar
  24. Khurram A. Jafery and Jana Dunfield. 2017. Sums of Uncertainty: Refinements Go Gradual. 804–817. Google ScholarGoogle Scholar
  25. C. Jones and Gordon D. Plotkin. 1989. A probabilistic powerdomain of evaluations. [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science, 186–195. Google ScholarGoogle Scholar
  26. Oleg Kiselyov. 2016. Probabilistic Programming Language and its Incremental Evaluation. In Programming Languages and Systems - 14th Asian Symposium, APLAS 2016, Hanoi, Vietnam, November 21-23, 2016, Proceedings, Atsushi Igarashi (Ed.) (Lecture Notes in Computer Science, Vol. 10017). 357–376. https://doi.org/10.1007/978-3-319-47958-3_19 Google ScholarGoogle ScholarCross RefCross Ref
  27. Ugo Dal Lago and Charles Grellois. 2017. Probabilistic Termination by Monadic Affine Sized Typing. ACM Transactions on Programming Languages and Systems (TOPLAS), 41 (2017), 1 – 65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ugo Dal Lago and Margherita Zorzi. 2012. Probabilistic operational semantics for the lambda calculus. RAIRO Theor. Informatics Appl., 46, 3 (2012), 413–450. https://doi.org/10.1051/ita/2012012 Google ScholarGoogle ScholarCross RefCross Ref
  29. Ugo Dal Lago and Margherita Zorzi. 2012. Probabilistic operational semantics for the lambda calculus. ArXiv, abs/1104.0195 (2012). Google ScholarGoogle Scholar
  30. Tuan Anh Le, Atilim Gunes Baydin, and Frank D. Wood. 2017. Inference Compilation and Universal Probabilistic Programming. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, Aarti Singh and Xiaojin (Jerry) Zhu (Eds.) (Proceedings of Machine Learning Research, Vol. 54). PMLR, 1338–1348. http://proceedings.mlr.press/v54/le17a.html Google ScholarGoogle Scholar
  31. Nico Lehmann and Éric Tanter. 2017. Gradual Refinement Types. 775–788. Google ScholarGoogle Scholar
  32. Meven Lennon-Bertrand, Kenji Maillard, Nicolas Tabareau, and Éric Tanter. 2022. Gradualizing the Calculus of Inductive Constructions. ACM Transactions on Programming Languages and Systems, To appear. To be presented at POPL’22. Google ScholarGoogle Scholar
  33. Stefan Malewski, Michael Greenberg, and Éric Tanter. 2021. Gradually Structured Data. Proceedings of the ACM on Programming Languages, 5, OOPSLA (2021), Nov., 126:1–126:28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Max S. New, Daniel R. Licata, and Amal Ahmed. 2019. Gradual Type Theory. 15:1–15:31. Google ScholarGoogle Scholar
  36. Avi Pfeffer. 2010. Practical Probabilistic Programming. In Inductive Logic Programming - 20th International Conference, ILP 2010, Florence, Italy, June 27-30, 2010. Revised Papers, Paolo Frasconi and Francesca A. Lisi (Eds.) (Lecture Notes in Computer Science, Vol. 6489). Springer, 2–3. Google ScholarGoogle Scholar
  37. Luna Phipps-Costin, Carolyn Jane Anderson, Michael Greenberg, and Arjun Guha. 2021. Solver-based gradual type migration. Proc. ACM Program. Lang., 5, OOPSLA (2021), 1–27. https://doi.org/10.1145/3485488 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. 2017. Proceedings of the 44th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2017). ACM Press, Paris, France. Google ScholarGoogle Scholar
  39. 2019. Proceedings of the 46th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2019). Google ScholarGoogle Scholar
  40. Norman Ramsey and Avi Pfeffer. 2002. Stochastic lambda calculus and monads of probability distributions. In POPL ’02. Google ScholarGoogle Scholar
  41. Jason Reed and Benjamin C. Pierce. 2010. Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming (ICFP’10). ACM, 157–168. Google ScholarGoogle Scholar
  42. Amr Sabry and Matthias Felleisen. 1993. Reasoning about Programs in Continuation-Passing Style. In LISP AND SYMBOLIC COMPUTATION. 288–298. Google ScholarGoogle Scholar
  43. Nasser Saheb-Djahromi. 1978. Probabilistic LCF. In MFCS. Google ScholarGoogle Scholar
  44. Roberto Segala and Nancy A. Lynch. 1995. Probabilistic Simulations for Probabilistic Processes. Nord. J. Comput., 2, 2 (1995), 250–273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jeremy Siek and Walid Taha. 2006. Gradual Typing for Functional Languages. In Proceedings of the Scheme and Functional Programming Workshop. 81–92. Google ScholarGoogle Scholar
  46. Jeremy Siek, Peter Thiemann, and Phil Wadler. 2015. Blame and Coercion: Together Again for the First Time. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2015). ACM Press, Portland, OR, USA. 425–435. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Jeremy Siek and Philip Wadler. 2010. Threesomes, with and without blame. In Proceedings of the 37th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2010). ACM Press, Madrid, Spain. 365–376. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Jeremy G. Siek, Ronald Garcia, and Walid Taha. 2009. Exploring the Design Space of Higher-Order Casts. In ESOP. Google ScholarGoogle Scholar
  49. Jeremy G. Siek, Michael M. Vitousek, Matteo Cimini, and John Tang Boyland. 2015. Refined Criteria for Gradual Typing. In 1st Summit on Advances in Programming Languages (SNAPL 2015) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 32). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Asilomar, California, USA. 274–293. Google ScholarGoogle Scholar
  50. Jeremy G. Siek, Michael M. Vitousek, Matteo Cimini, Sam Tobin-Hochstadt, and Ronald Garcia. 2015. Monotonic References for Efficient Gradual Typing. In ESOP. Google ScholarGoogle Scholar
  51. Asumu Takikawa, T. Stephen Strickland, Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012. Gradual Typing for First-Class Classes. In Proceedings of the 27th ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications (OOPSLA 2012). ACM Press, Tucson, AZ, USA. 793–810. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Matías Toro, Ronald Garcia, and Éric Tanter. 2018. Type-Driven Gradual Security with References. ACM Transactions on Programming Languages and Systems, 40, 4 (2018), Nov., 16:1–16:55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Matías Toro and Éric Tanter. 2017. A Gradual Interpretation of Union Types. In Proceedings of the 24th Static Analysis Symposium (SAS 2017) (Lecture Notes in Computer Science, Vol. 10422). Springer-Verlag, New York City, NY, USA. 382–404. Google ScholarGoogle Scholar
  54. Matías Toro and Éric Tanter. 2020. Abstracting Gradual References. Science of Computer Programming, 197 (2020), Oct., 1–65. Google ScholarGoogle ScholarCross RefCross Ref
  55. Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, and David M. Blei. 2017. Deep Probabilistic Programming. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=Hy6b4Pqee Google ScholarGoogle Scholar
  56. Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. 2018. An Introduction to Probabilistic Programming. CoRR, abs/1809.10756 (2018), arXiv:1809.10756. arxiv:1809.10756 Google ScholarGoogle Scholar
  57. Philip Wadler and Robert Bruce Findler. 2009. Well-Typed Programs Can’t Be Blamed. In Proceedings of the 18th European Symposium on Programming Languages and Systems (ESOP 2009), Giuseppe Castagna (Ed.) (Lecture Notes in Computer Science, Vol. 5502). Springer-Verlag, York, UK. 1–16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Wenjia Ye, Bruno C. d. S. Oliveira, and Xuejing Huang. 2021. Type-Directed Operational Semantics for Gradual Typing. In ECOOP. Google ScholarGoogle Scholar

Index Terms

  1. A Gradual Probabilistic Lambda Calculus

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Article Metrics

            • Downloads (Last 12 months)170
            • Downloads (Last 6 weeks)57

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!