Abstract
We present a denotational semantics for higher-order probabilistic programs in terms of linear operators between Banach spaces. Our semantics is rooted in the classical theory of Banach spaces and their tensor products, but bears similarities with the well-known semantics of higher-order programs a la Scott through the use of ordered Banach spaces which allow definitions in terms of fixed points. Our semantics is a model of intuitionistic linear logic: it is based on a symmetric monoidal closed category of ordered Banach spaces which treats randomness as a linear resource, but by constructing an exponential comonad we can also accommodate non-linear reasoning. We apply our semantics to the verification of the classical Gibbs sampling algorithm.
Supplemental Material
- S. Abramsky, R. Blute, and P. Panangaden. 1999. Nuclear and trace ideals in tensored ★-categories. Journal of Pure and Applied Algebra 143, 1-3 (1999), 3–47.Google Scholar
Cross Ref
- C. Aliprantis and K. Border. 1999. Infinite dimensional analysis. Vol. 32006. Springer.Google Scholar
- C. D. Aliprantis and O. Burkinshaw. 2006. Positive operators. Vol. 119. Springer Science & Business Media.Google Scholar
- J. Borgström, U. Dal Lago, A. D. Gordon, and M. Szymczak. 2016. A lambda-calculus foundation for universal probabilistic programming. In ACM SIGPLAN Notices, Vol. 51. ACM, 33–46.Google Scholar
- J. T. Chang and D. Pollard. 1997. Conditioning as disintegration. Statistica Neerlandica 51, 3 (1997), 287–317.Google Scholar
Cross Ref
- P. Chaput, V. Danos, P. Panangaden, and G. Plotkin. 2014. Approximating Markov Processes by averaging. J. ACM 61, 1 (Jan. 2014).Google Scholar
Digital Library
- F. Clerc, F. Dahlqvist, V. Danos, and I. Garnier. 2017. Pointless Learning. In Foundations of Software Science and Computation Structures - 20th International Conference, FOSSACS 2017. Proceedings.Google Scholar
- R. Crubillé, T. Ehrhard, M. Pagani, and C. Tasson. 2017. The free exponential modality of probabilistic coherence spaces. In International Conference on Foundations of Software Science and Computation Structures. Springer, 20–35.Google Scholar
- F. Dahlqvist, V. Danos, I. Garnier, and A. Silva. 2018. Borel kernels and their Approximations, Categorically. In Mathematical Foundations of Programming Semantics (MFPS).Google Scholar
- E. B. Davies. 1968. The structure and ideal theory of the predual of a Banach lattice. Trans. Amer. Math. Soc. 131, 2 (1968), 544–555.Google Scholar
Cross Ref
- J. Dieudonné. 1951. Sur les espaces de Köthe. Jour. d’Analyse Math. 1, 1 (1951), 81–115.Google Scholar
- N. Dunford, J. T. Schwartz, W. G Bade, and R. G. Bartle. 1971. Linear operators I. Wiley-interscience New York.Google Scholar
- T. Ehrhard, M. Pagani, and C. Tasson. 2017. Measurable cones and stable, measurable functions: a model for probabilistic higher-order programming. Proceedings of the ACM on Programming Languages 2, POPL (2017), 59.Google Scholar
- T. Ehrhard, C. Tasson, and M. Pagani. 2014. Probabilistic coherence spaces are fully abstract for probabilistic PCF. In ACM SIGPLAN Notices, Vol. 49. ACM, 309–320.Google Scholar
- T. Eisner, B. Farkas, M. Haase, and R. Nagel. 2015. Operator theoretic aspects of ergodic theory. Vol. 272. Springer.Google Scholar
- D. H. Fremlin. 1972. Tensor products of Archimedean vector lattices. American Journal of Mathematics 94, 3 (1972), 777–798.Google Scholar
Cross Ref
- D. H. Fremlin. 1974. Tensor products of Banach lattices. Math. Ann. 211, 2 (1974), 87–106.Google Scholar
Cross Ref
- S. Geman and D. Geman. 1987. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. In Readings in computer vision. Elsevier, 564–584.Google Scholar
- M. Giry. 1982. A categorical approach to probability theory. In Categorical aspects of topology and analysis. Springer, 68–85.Google Scholar
- A. Grothendieck. 1955. Produits tensoriels topologiques et espaces nucléaires. Vol. 16. American Mathematical Soc.Google Scholar
- B. Hayes. 2001. Computing Science: Randomness as a Resource. American Scientist 89, 4 (2001), 300–304.Google Scholar
Cross Ref
- O. Heunen, C.and Kammar, S. Staton, and H. Yang. 2017. A convenient category for higher-order probability theory. In LICS. IEEE, 1–12.Google Scholar
- A. S. Kechris. 1995. Classical descriptive set theory. Graduate Text in Mathematics, Vol. 156. Springer.Google Scholar
- D. Kozen. 1981. Semantics of probabilistic programs. J. Comput. Syst. Sci. 22, 3 (June 1981), 328–350.Google Scholar
Cross Ref
- D. Kozen. 1985. A probabilistic PDL. J. Comput. Syst. Sci. 30, 2 (April 1985), 162–178.Google Scholar
Cross Ref
- D. Licata and E. Finster. 2014. Eilenberg-MacLane spaces in homotopy type theory. In LICS. ACM, 66.Google Scholar
- P. Mellies. 2009. Categorical semantics of linear logic. Panoramas et syntheses 27 (2009), 15–215.Google Scholar
- P. Melliès, N. Tabareau, and C. Tasson. 2018. An explicit formula for the free exponential modality of linear logic. Mathematical Structures in Computer Science 28, 7 (2018), 1253–1286.Google Scholar
- K. C. Min. 1983. An exponential law for regular ordered Banach spaces. Cahiers de Topologie et Géométrie Différentielle Catégoriques 24, 3 (1983), 279–298.Google Scholar
- R. A. Ryan. 2013. Introduction to tensor products of Banach spaces. Springer Science & Business Media.Google Scholar
- A. Ścibior, O. Kammar, M. Vákár, S. Staton, H. Yang, Y. Cai, K. Ostermann, S. K. Moss, C. Heunen, and Z. Ghahramani. 2017. Denotational validation of higher-order Bayesian inference. Proceedings of the ACM on Programming Languages 2, POPL (2017), 60.Google Scholar
- P. Selinger and B. Valiron. 2008. A linear-non-linear model for a computational call-by-value lambda calculus. In International Conference on Foundations of Software Science and Computational Structures. Springer, 81–96.Google Scholar
- S. Staton. 2017. Commutative semantics for probabilistic programming. In ESOP. Springer, 855–879.Google Scholar
- S. Staton, F. Wood, H. Yang, C. Heunen, and O. Kammar. 2016. Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints. In 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS). IEEE, 1–10.Google Scholar
- M. Vákár, O. Kammar, and S. Staton. 2019. A domain theory for statistical probabilistic programming. Proceedings of the ACM on Programming Languages 3, POPL (2019), 36.Google Scholar
- A. W. Wickstead. 2007. Regular operators between Banach lattices. In Positivity. Springer, 255–279.Google Scholar
- G. Wittstock. 1974. Ordered normed tensor products. In Foundations of quantum mechanics and ordered linear spaces. Springer, 67–84.Google Scholar
- Y. Wong and K. Ng. 1973. Partially ordered topological vector spaces. Oxford University Press.Google Scholar
- A. C. Zaanen. 2012. Introduction to operator theory in Riesz spaces. Springer Science & Business Media.Google Scholar
Index Terms
Semantics of higher-order probabilistic programs with conditioning
Recommendations
Higher-order probabilistic adversarial computations: categorical semantics and program logics
Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. These computations arise routinely in several domains, ...
Program Logic for Higher-Order Probabilistic Programs in Isabelle/HOL
Functional and Logic ProgrammingAbstractThe verification framework PPV (Probabilistic Program Verification) verifies functional probabilistic programs supporting higher-order functions, continuous distributions, and conditional inference. PPV is based on the theory of quasi-Borel spaces ...
Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization
Probabilistic programming provides a convenient lingua franca for writing succinct and rigorous descriptions of probabilistic models and inference tasks. Several probabilistic programming languages, including Anglican, Church or Hakaru, derive their ...






Comments