skip to main content
research-article
Open Access

Reasoning about “reasoning about reasoning”: semantics and contextual equivalence for probabilistic programs with nested queries and recursion

Published:12 January 2022Publication History
Skip Abstract Section

Abstract

Metareasoning can be achieved in probabilistic programming languages (PPLs) using agent models that recursively nest inference queries inside inference queries. However, the semantics of this powerful, reflection-like language feature has defied an operational treatment, much less reasoning principles for contextual equivalence. We give formal semantics to a core PPL with continuous distributions, scoring, general recursion, and nested queries. Unlike prior work, the presence of nested queries and general recursion makes it impossible to stratify the definition of a sampling-based operational semantics and that of a measure-theoretic semantics—the two semantics must be defined mutually recursively. A key yet challenging property we establish is that probabilistic programs have well-defined meanings: limits exist for the step-indexed measures they induce. Beyond a semantics, we offer relational reasoning principles for probabilistic programs making nested queries. We construct a step-indexed, biorthogonal logical-relations model. A soundness theorem establishes that logical relatedness implies contextual equivalence. We demonstrate the usefulness of the reasoning principles by proving novel equivalences of practical relevance—in particular, game-playing and decisionmaking agents. We mechanize our technical developments leading to the soundness proof using the Coq proof assistant. Nested queries are an important yet theoretically underdeveloped linguistic feature in PPLs; we are first to give them semantics in the presence of general recursion and to provide them with sound reasoning principles for contextual equivalence.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

This is the required 5-minute presentation advertising the POPL 2022 paper.

References

  1. Nathanael L. Ackerman, Cameron E. Freer, and Daniel M. Roy. 2011. Noncomputable Conditional Distributions. In Symp. on Logic In Computer Science (LICS). https://doi.org/10.1109/LICS. 2011.49 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Amal Ahmed. 2006. Step-Indexed Syntactic Logical Relations for Recursive and Quantified Types. In European Symp. on Programming (ESOP). https://doi.org/10.1007/11693024_6 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Andrew W. Appel and David McAllester. 2001. An Indexed Model of Recursive Types for Foundational Proof-Carrying Code. ACM Tran. on Programming Languages and Systems (TOPLAS) 23, 5 (Sept. 2001 ). https://doi.org/10.1145/504709.504712 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Richard E. Bellman. 1957. Dynamic Programming. Princeton University Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Benjamin Bichsel, Timon Gehr, and Martin Vechev. 2018. Fine-Grained Semantics for Probabilistic Programs. In European Symp. on Programming (ESOP). https://doi.org/10.1007/978-3-319-89884-1_6 Google ScholarGoogle ScholarCross RefCross Ref
  6. Aleš Bizjak and Lars Birkedal. 2015. Step-Indexed Logical Relations for Probability. In Int'l Conf. on Foundations of Software Science and Computation Structures (FoSSaCS). https://doi.org/10.1007/978-3-662-46678-0_18 Google ScholarGoogle ScholarCross RefCross Ref
  7. Johannes Borgström, Ugo Dal Lago, Andrew D. Gordon, and Marcin Szymczak. 2016. A Lambda-Calculus Foundation for Universal Probabilistic Programming. In ACM SIGPLAN Conf. on Functional Programming (ICFP). https://doi.org/10. 1145/2951913.2951942 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ryan Culpepper and Andrew Cobb. 2017. Contextual Equivalence for Probabilistic Programs with Continuous Random Variables and Scoring. In European Symp. on Programming (ESOP). https://doi.org/10.1007/978-3-662-54434-1_14 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Owain Evans, Andreas Stuhlmüller, John Salvatier, and Daniel Filan. 2017. Modeling Agents with Probabilistic Programs. https://agentmodels.org.Google ScholarGoogle Scholar
  10. Matthias Felleisen and Robert Hieb. 1992. The Revised Report on the Syntactic Theories of Sequential Control and State. Theoretical Computer Science 103, 2 ( 1992 ). https://doi.org/10.1016/ 0304-3975 ( 92 ) 90014-7 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cameron E. Freer, Daniel M. Roy, and Joshua B. Tenenbaum. 2014. Towards Common Sense Reasoning via Conditional Simulation: Legacies of Turing in Artificial Intelligence. Turing's Legacy 42 ( 2014 ). arXiv: 1212. 4799Google ScholarGoogle Scholar
  12. Samuel J. Gershman and Noah D. Goodman. 2014. Amortized Inference in Probabilistic Reasoning. In Annual Meeting of the Cognitive Science Society (CogSci). https://cogsci.mindmodeling.org/2014/papers/098/paper098.pdfGoogle ScholarGoogle Scholar
  13. Noah D. Goodman, Vikash K. Mansinghka, Daniel Roy, Keith Bonawitz, and Joshua B. Tenenbaum. 2008. Church: A Language for Generative Models. In Uncertainty in Artificial Intelligence (UAI). arXiv:1206.3255Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. 2017. A Convenient Category for Higher-Order Probability Theory. In Symp. on Logic In Computer Science (LICS). https://doi.org/10.1109/LICS. 2017.8005137 Google ScholarGoogle ScholarCross RefCross Ref
  15. Steven Holtzen, Todd Millstein, and Guy Van den Broeck. 2019. Symbolic Exact Inference for Discrete Probabilistic Programs. ( 2019 ). arXiv: 1904.02079Google ScholarGoogle Scholar
  16. Steven Holtzen, Guy Van den Broeck, and Todd Millstein. 2020. Scaling Exact Inference for Discrete Probabilistic Programs. Proc. of the ACM on Programming Languages (PACMPL) 4, OOPSLA ( 2020 ). https://doi.org/10.1145/3428208 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Ronald A. Howard. 1960. Dynamic Programming and Markov Processes. The MIT Press.Google ScholarGoogle Scholar
  18. Daniel Huang, Greg Morrisett, and Bas Spitters. 2020. An Application of Computable Distributions to the Semantics of Probabilistic Programs. Cambridge University Press. https://doi.org/10.1017/9781108770750.004 Google ScholarGoogle ScholarCross RefCross Ref
  19. Thomas Icard. 2017. Beyond Almost-Sure Termination. In Annual Meeting of the Cognitive Science Society (CogSci). https: //cogsci.mindmodeling.org/2017/papers/0430/paper0430.pdfGoogle ScholarGoogle Scholar
  20. Dexter Kozen. 1979. Semantics of Probabilistic Programs. In Annual Symposium on Foundations of Computer Science (SFCS). https://doi.org/10.1109/SFCS. 1979.38 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jun S. Liu and Rong Chen. 1998. Sequential Monte Carlo Methods for Dynamic Systems. Journal of the American Statistical Association 93, 443 ( 1998 ). https://doi.org/10.1080/01621459. 1998.10473765 Google ScholarGoogle ScholarCross RefCross Ref
  22. Theofrastos Mantadelis and Gerda Janssens. 2011. Nesting Probabilistic Inference. ( 2011 ). arXiv: 1112. 3785Google ScholarGoogle Scholar
  23. Ian Mason and Carolyn Talcott. 1991. Equivalence in Functional Languages with Efects. Journal of Functional Programming (JFP) 1, 3 ( 1991 ). https://doi.org/10.1017/S0956796800000125 Google ScholarGoogle ScholarCross RefCross Ref
  24. Micaela Mayero. 2001. Formalisation et automatisation de preuves en analyses réelle et numérique. Ph.D. Dissertation. Université Paris VI.Google ScholarGoogle Scholar
  25. James H. Morris, Jr. 1968. Lambda-Calculus Models of Programming Languages. Ph.D. Dissertation. Massachusetts Institute of Technology. http://hdl.handle. net/1721.1/64850Google ScholarGoogle Scholar
  26. Federico Olmedo, Friedrich Gretz, Nils Jansen, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Annabelle McIver. 2018. Conditioning in Probabilistic Programming. ACM Tran. on Programming Languages and Systems (TOPLAS) 40, 1 ( 2018 ). https://doi.org/10.1145/3156018 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Sungwoo Park, Frank Pfenning, and Sebastian Thrun. 2008. A Probabilistic Language Based on Sampling Functions. ACM Tran. on Programming Languages and Systems (TOPLAS) 31, 1 (Dec. 2008 ). https://doi.org/10.1145/1452044.1452048 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Andrew Pitts and Ian Stark. 1998. Operational Reasoning for Functions with Local State. In Higher Order Operational Techniques in Semantics. https://homepages.inf.ed.ac.uk/stark/operfl.pdfGoogle ScholarGoogle Scholar
  29. Tom Rainforth. 2018. Nesting Probabilistic Programs. In Uncertainty in Artificial Intelligence (UAI). arXiv: 1803.06328Google ScholarGoogle Scholar
  30. Iris Rubi Seaman, Jan-Willem van de Meent, and David Wingate. 2020. Nested Reasoning About Autonomous Agents Using Probabilistic Programs. ( 2020 ). arXiv: 1812.01569Google ScholarGoogle Scholar
  31. Brian Cantwell Smith. 1982. Procedural Reflection in Programming Languages. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  32. Rick Statman. 1985. Logical Relations and the Typed-calculus. Information and Control 65, 2 ( 1985 ). https://doi.org/10. 1016/S0019-9958 ( 85 ) 80001-2 Google ScholarGoogle ScholarCross RefCross Ref
  33. Sam Staton. 2017. Commutative Semantics for Probabilistic Programming. In European Symp. on Programming (ESOP). https://doi.org/10.1007/978-3-662-54434-1_32 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Sam Staton, Frank Wood, Hongseok Yang, Chris Heunen, and Ohad Kammar. 2016. Semantics for Probabilistic Programming: Higher-Order Functions, Continuous Distributions, and Soft Constraints. In Symp. on Logic In Computer Science (LICS). https://doi.org/10.1145/2933575.2935313 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Elias M. Stein and Rami Shakarchi. 2005. Real Analysis: Measure Theory, Integration, and Hilbert Spaces. Princeton University Press. https://doi.org/10.2307/j.ctvd58v18 Google ScholarGoogle ScholarCross RefCross Ref
  36. Andreas Stuhlmüller and Noah D. Goodman. 2012. A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs. ( 2012 ). arXiv: 1206. 3555Google ScholarGoogle Scholar
  37. Andreas Stuhlmüller and Noah D. Goodman. 2014. Reasoning about Reasoning by Nested Conditioning: Modeling Theory of Mind with Probabilistic Programs. Cognitive Systems Research 28 ( 2014 ). https://doi.org/10.1016/j.cogsys. 2013. 07.003 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Marcin Szymczak and Joost-Pieter Katoen. 2019. Weakest Preexpectation Semantics for Bayesian Inference. In Int'l School on Engineering Trustworthy Software Systems (SETSS). https://doi.org/10.1007/978-3-030-55089-9_3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Zenna Tavares, Xin Zhang, Edgar Minaysan, Javier Burroni, Rajesh Ranganath, and Armando Solar-Lezama. 2019. The Random Conditional Distribution for Higher-Order Probabilistic Inference. ( 2019 ). arXiv: 1903.10556Google ScholarGoogle Scholar
  40. David Tolpin, Yuan Zhou, and Hongseok Yang. 2021. Probabilistic Programs with Stochastic Conditioning. ( 2021 ). arXiv: 2010.00282Google ScholarGoogle Scholar
  41. Mitchell Wand. 1998. The Theory of Fexprs is Trivial. Lisp and Symbolic Computation 10, 3 ( 1998 ). https://doi.org/10.1023/A: 1007720632734 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Mitchell Wand, Ryan Culpepper, Theophilos Giannakopoulos, and Andrew Cobb. 2018. Contextual Equivalence for a Probabilistic Language with Continuous Random Variables and Recursion. Proc. of the ACM on Programming Languages (PACMPL) 2, ICFP ( 2018 ). https://doi.org/10.1145/3236782 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yizhou Zhang and Nada Amin. 2021. Semantics and Contextual Equivalence for Probabilistic Programs with Nested Queries and Recursion. Technical Report CS-2021-02. School of Computer Science, University of Waterloo. https://uwaterloo.ca/ computer-science/sites/ca.computer-science/files/uploads/files/cs-2021-02.pdfGoogle ScholarGoogle Scholar

Index Terms

  1. Reasoning about “reasoning about reasoning”: semantics and contextual equivalence for probabilistic programs with nested queries and recursion

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!