skip to main content
research-article

Slicing probabilistic programs

Published:09 June 2014Publication History
Skip Abstract Section

Abstract

Probabilistic programs use familiar notation of programming languages to specify probabilistic models. Suppose we are interested in estimating the distribution of the return expression r of a probabilistic program P. We are interested in slicing the probabilistic program P and obtaining a simpler program Sli(P) which retains only those parts of P that are relevant to estimating r, and elides those parts of P that are not relevant to estimating r. We desire that the Sli transformation be both correct and efficient. By correct, we mean that P and Sli(P) have identical estimates on r. By efficient, we mean that estimation over Sli(P) be as fast as possible.

We show that the usual notion of program slicing, which traverses control and data dependencies backward from the return expression r, is unsatisfactory for probabilistic programs, since it produces incorrect slices on some programs and sub-optimal ones on others. Our key insight is that in addition to the usual notions of control dependence and data dependence that are used to slice non-probabilistic programs, a new kind of dependence called observe dependence arises naturally due to observe statements in probabilistic programs.

We propose a new definition of Sli(P) which is both correct and efficient for probabilistic programs, by including observe dependence in addition to control and data dependences for computing slices. We prove correctness mathematically, and we demonstrate efficiency empirically. We show that by applying the Sli transformation as a pre-pass, we can improve the efficiency of probabilistic inference, not only in our own inference tool R2, but also in other systems for performing inference such as Church and Infer.NET.

References

  1. T. Ball, R. Majumdar, T. Millstein, and S. K. Rajamani. Automatic predicate abstraction of C programs. In Programming Language Design and Implementation (PLDI), pages 203--213, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. J. Borgström, A. D. Gordon, M. Greenberg, J. Margetson, and J. Van Gael. Measure transformer semantics for Bayesian machine learning. In European Symposium on Programming (ESOP), pages 77--96, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. Chaganty, A. Lal, A. V. Nori, and S. K. Rajamani. Combining relational learning with smt solvers using CEGAR. In Computer Aided Verification (CAV), pages 447--462, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. T. Chaganty, A. V. Nori, and S. K. Rajamani. Efficiently sampling probabilistic programs via program analysis. In Artificial Intelligence and Statistics (AISTATS), 2013.Google ScholarGoogle Scholar
  5. G. Claret, S. K. Rajamani, A. V. Nori, A. D. Gordon, and J. Borgström. Bayesian inference for probabilistic programs via symbolic execution. In Foundations of Software Engineering (FSE), 2013.Google ScholarGoogle Scholar
  6. T. M. Cover and J. Thomas. Elements of Information Theory. Wiley, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. R. Cytron, J. Ferrante, B. K. Rosen, M. N. Wegman, and F. K. Zadeck. An efficient method of computing static single assignment form. In Principles of Programming Languages (POPL), pages 25--35, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. D. L. Detlefs, K. R. M. Leino, G. Nelson, and J. B. Saxe. Extended static checking. Compaq SRC Research Report 159, 1998.Google ScholarGoogle Scholar
  9. J. Ferrante, K. J. Ottenstein, and J. D. Warren. The program dependence graph and its use in optimization. Transactions on Programming Languages and Systems (TOPLAS)., 9(3):319--349, 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. W. R. Gilks, A. Thomas, and D. J. Spiegelhalter. A language and program for complex Bayesian modelling. The Statistician, 43(1):169--177, 1994.Google ScholarGoogle Scholar
  11. N. D. Goodman, V. K. Mansinghka, D. M. Roy, K. Bonawitz, and J. B. Tenenbaum. Church: a language for generative models. In Uncertainty in Artificial Intelligence (UAI), pages 220--229, 2008.Google ScholarGoogle Scholar
  12. A. D. Gordon, M. Aizatulin, J. Borgström, G. Claret, T. Graepel, A. V. Nori, S. K. Rajamani, and C. Russo. A model-learner pattern for Bayesian reasoning. In Principles of Programming Languages (POPL), pages 403--416, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. J. Hatcliff, J. C. Corbett, M. B. Dwyer, S. Sokolowski, and H. Zheng. A formal study of slicing for multi-threaded programs with jvm concurrency primitives. In Static Analysis Symposium (SAS), pages 1--18, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. R. Herbrich, T. Minka, and T. Graepel. TrueSkill: A Bayesian skill rating system. In Neural Information Processing Systems (NIPS), pages 569--576, 2006.Google ScholarGoogle Scholar
  15. M. D. Hoffman and A. Gelman. The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, in press, 2013.Google ScholarGoogle Scholar
  16. S. Horwitz, T. W. Reps, and D. Binkley. Interprocedural slicing using dependence graphs. ACM Transactions on Programming Languages and Systems (TOPLAS), 12(1):26--60, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. O. Kiselyov and C. Shan. Monolingual probabilistic programming using generalized coroutines. In Uncertainty in Artificial Intelligence (UAI), pages 285--292, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. Kok, M. Sumner, M. Richardson, P. Singla, H. Poon, D. Lowd, and P. Domingos. The Alchemy system for statistical relational AI, 2007. http://alchemy.cs.washington.edu.Google ScholarGoogle Scholar
  19. D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. D. Koller, D. A. McAllester, and A. Pfeffer. Effective Bayesian inference for stochastic programs. In National Conference on Artificial Intelligence (AAAI), pages 740--747, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. J. C. MacKay. Information Theory, Inference & Learning Algorithms. Cambridge University Press, New York, NY, USA, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. P. Mardziel, S. Magill, M. Hicks, and M. Srivatsa. Dynamic enforcement of knowledge-based security policies using probabilistic abstract interpretation. Journal of Computer Security, January 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. T. Minka, J. Winn, J. Guiver, and A. Kannan. Infer.NET 2.3, Nov. 2009. Software available from http://research.microsoft.com/infernet.Google ScholarGoogle Scholar
  24. G. Nelson. A generalization of Dijkstra's calculus. Transactions on Programming Languages and Systems (TOPLAS)., 11(4):517--561, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. V. Nori, C.-K. Hur, S. K. Rajamani, and S. Samuel. The R2 project. 2014. http://research.microsoft.com/r2.Google ScholarGoogle Scholar
  26. A. Pfeffer. The design and implementation of IBAL: A general-purpose probabilistic language. In L. Getoor and B. Taskar, editors, Introduction to Statistical Relational Learning. MIT Press, 2007.Google ScholarGoogle Scholar
  27. J. Pfeffer. Probabilistic Reasoning in Intelligence Systems. Morgan Kaufmann, 1996.Google ScholarGoogle Scholar
  28. S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis of probabilistic programs: Inferring whole program properties from finitely many executions. In Programming Languages Design and Implementation (PLDI), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. M. Weiser. Program slicing. In International Conference on Software Engineering (ICSE), pages 439--449, 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Slicing probabilistic programs

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM SIGPLAN Notices
              ACM SIGPLAN Notices  Volume 49, Issue 6
              PLDI '14
              June 2014
              598 pages
              ISSN:0362-1340
              EISSN:1558-1160
              DOI:10.1145/2666356
              • Editor:
              • Andy Gill
              Issue’s Table of Contents
              • cover image ACM Conferences
                PLDI '14: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation
                June 2014
                619 pages
                ISBN:9781450327848
                DOI:10.1145/2594291

              Copyright © 2014 ACM

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 9 June 2014

              Check for updates

              Qualifiers

              • research-article

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!