skip to main content
article

Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs

Published:11 January 2016Publication History
Skip Abstract Section

Abstract

In this paper, we consider termination of probabilistic programs with real-valued variables. The questions concerned are: 1. qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the expected termination time is finite (finite termination); 2. quantitative ones that ask (i) to approximate the expected termination time (expectation problem) and (ii) to compute a bound B such that the probability to terminate after B steps decreases exponentially (concentration problem). To solve these questions, we utilize the notion of ranking supermartingales which is a powerful approach for proving termination of probabilistic programs. In detail, we focus on algorithmic synthesis of linear ranking-supermartingales over affine probabilistic programs (APP's) with both angelic and demonic non-determinism. An important subclass of APP's is LRAPP which is defined as the class of all APP's over which a linear ranking-supermartingale exists. Our main contributions are as follows. Firstly, we show that the membership problem of LRAPP (i) can be decided in polynomial time for APP's with at most demonic non-determinism, and (ii) is NP-hard and in PSPACE for APP's with angelic non-determinism; moreover, the NP-hardness result holds already for APP's without probability and demonic non-determinism. Secondly, we show that the concentration problem over LRAPP can be solved in the same complexity as for the membership problem of LRAPP. Finally, we show that the expectation problem over LRAPP can be solved in 2EXPTIME and is PSPACE-hard even for APP's without probability and non-determinism (i.e., deterministic programs). Our experimental results demonstrate the effectiveness of our approach to answer the qualitative and quantitative questions over APP's with at most demonic non-determinism.

References

  1. IBM ILOG CPLEX Optimizer. http://www- 01.ibm.com/software/integration/optimization/cplex-optimizer/, 2010.Google ScholarGoogle Scholar
  2. K. Azuma. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, 19(3):357–367, 1967.Google ScholarGoogle ScholarCross RefCross Ref
  3. C. Baier and J.-P. Katoen. Principles of model checking. MIT Press, 2008. ISBN 978-0-262-02649-9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. G. Bennett. Probability inequalities for the sum of independent random variables. Journal of the American Statistical Association, 57(297): 33–45, 1962.Google ScholarGoogle ScholarCross RefCross Ref
  5. P. Billingsley. Probability and Measure. Wiley, 3rd edition, 1995.Google ScholarGoogle Scholar
  6. A. Bockmayr and V. Weispfenning. Solving numerical constraints. In J. A. Robinson and A. Voronkov, editors, Handbook of Automated Reasoning (in 2 volumes), pages 751–842. Elsevier and MIT Press, 2001. ISBN 0-444-50813-9.Google ScholarGoogle Scholar
  7. O. Bournez and F. Garnier. Proving positive almost-sure termination. In RTA, pages 323–337, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. R. Bradley, Z. Manna, and H. B. Sipma. The polyranking principle. In ICALP, pages 1349–1361, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. R. Bradley, Z. Manna, and H. B. Sipma. Linear ranking with reachability. In K. Etessami and S. K. Rajamani, editors, Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005, Proceedings, volume 3576 of Lecture Notes in Computer Science, pages 491–504. Springer, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. ISBN 3-540-27231-3..Google ScholarGoogle Scholar
  11. T. Brázdil, J. Esparza, S. Kiefer, and A. Kuˇcera. Analyzing Probabilistic Pushdown Automata. FMSD, 43(2):124–163, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Canny. Some algebraic and geometric computations in pspace. In Proceedings of the twentieth annual ACM symposium on Theory of computing, pages 460–467. ACM, 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. A. Chakarov and S. Sankaranarayanan. Probabilistic program analysis with martingales. In N. Sharygina and H. Veith, editors, Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. Proceedings, volume 8044 of Lecture Notes in Computer Science, pages 511–526. Springer, 2013. ISBN 978-3-642-39798-1.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. K. Chatterjee, H. Fu, P. Novotný, and R. Hasheminezhad. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. CoRR, abs/1510.08517, 2015. URL http: //arxiv.org/abs/1510.08517. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. F. Chung and L. Lu. Concentration inequalities and martingale inequalities: A survey. Internet Mathematics, 3:79–127, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  16. M. Colón and H. Sipma. Synthesis of linear ranking functions. In T. Margaria and W. Yi, editors, Tools and Algorithms for the Construction and Analysis of Systems, 7th International Conference, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings, volume 2031 of Lecture Notes in Computer Science, pages 67–81. Springer, 2001. ISBN 3-540-41865-2.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. B. Cook, A. See, and F. Zuleger. Ramsey vs. lexicographic termination proving. In TACAS, pages 47–61, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. P. Cousot and R. Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In R. M. Graham, M. A. Harrison, and R. Sethi, editors, Conference Record of the Fourth ACM Symposium on Principles of Programming Languages, Los Angeles, California, USA, January 1977, pages 238–252. ACM, 1977.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1st edition, 2009. ISBN 0521884276, 9780521884273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. R. Durrett. Probability: Theory and Examples (Second Edition). Duxbury Press, 1996.Google ScholarGoogle Scholar
  21. J. Esparza, A. Kuˇcera, and R. Mayr. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In LICS, pages 117–126. IEEE, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Esparza, A. Gaiser, and S. Kiefer. Proving termination of probabilistic programs using patterns. In CAV, pages 123–138, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Farkas. A fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö, 12:457–472, 1894.Google ScholarGoogle Scholar
  24. L. M. F. Fioriti and H. Hermanns. Probabilistic termination: Soundness, completeness, and compositionality. In S. K. Rajamani and D. Walker, editors, Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, pages 489–501. ACM, 2015. ISBN 978-1-4503-3300-9.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. R. W. Floyd. Assigning meanings to programs. Mathematical Aspects of Computer Science, 19:19–33, 1967.Google ScholarGoogle ScholarCross RefCross Ref
  26. F. G. Foster. On the stochastic matrices associated with certain queuing processes. The Annals of Mathematical Statistics, 24(3):pp. 355–360, 1953.Google ScholarGoogle ScholarCross RefCross Ref
  27. S. Hart and M. Sharir. Concurrent probabilistic programs, or: How to schedule if you must. SIAM J. Comput., 14(4):991–1012, 1985.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301): 13–30, 1963.Google ScholarGoogle ScholarCross RefCross Ref
  29. H. Howard. Dynamic Programming and Markov Processes. MIT Press, 1960.Google ScholarGoogle Scholar
  30. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237– 285, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1):99–134, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. J. Katoen, A. McIver, L. Meinicke, and C. C. Morgan. Linear-invariant generation for probabilistic programs: - automated support for proofbased methods. In SAS, volume LNCS 6337, Springer, pages 390–406, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. J. Kemeny, J. Snell, and A. Knapp. Denumerable Markov Chains. D. Van Nostrand Company, 1966.Google ScholarGoogle Scholar
  34. H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. Temporal-logicbased reactive mission and motion planning. IEEE Transactions on Robotics, 25(6):1370–1381, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. M. Z. Kwiatkowska, G. Norman, and D. Parker. Prism 4.0: Verification of probabilistic real-time systems. In CAV, LNCS 6806, pages 585–591, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. C. S. Lee, N. D. Jones, and A. M. Ben-Amram. The size-change principle for program termination. In POPL, pages 81–92, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. C. McDiarmid. Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics, pages 195–248. 1998.Google ScholarGoogle Scholar
  38. A. McIver and C. Morgan. Developing and reasoning about probabilistic programs in pGCL. In PSSE, pages 123–155, 2004.Google ScholarGoogle Scholar
  39. A. McIver and C. Morgan. Abstraction, Refinement and Proof for Probabilistic Systems. Monographs in Computer Science. Springer, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. D. Monniaux. An abstract analysis of the probabilistic termination of programs. In P. Cousot, editor, Static Analysis, 8th International Symposium, SAS 2001, Paris, France, July 16-18, 2001, Proceedings, volume 2126 of Lecture Notes in Computer Science, pages 111–126. Springer, 2001. ISBN 3-540-42314-1.. URL http://dx.doi. org/10.1007/3-540-47764-0_7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1995. ISBN 0-521-47465-5, 9780521474658. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. T. S. Motzkin. Beiträge zur Theorie der linearen Ungleichungen (German). PhD thesis, Basel, Jerusalem, 1936.Google ScholarGoogle Scholar
  43. A. Paz. Introduction to probabilistic automata (Computer science and applied mathematics). Academic Press, 1971. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. A. Podelski and A. Rybalchenko. A complete method for the synthesis of linear ranking functions. In B. Steffen and G. Levi, editors, Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, January 11-13, 2004, Proceedings, volume 2937 of Lecture Notes in Computer Science, pages 239–251. Springer, 2004. ISBN 3-540-20803-8..Google ScholarGoogle Scholar
  45. M. Rabin. Probabilistic automata. Information and Control, 6:230–245, 1963.Google ScholarGoogle ScholarCross RefCross Ref
  46. J. S. Rosenthal. A First Look at Rigorous Probability Theory. World Scientific Publishing Company, 2nd edition, 2006.Google ScholarGoogle Scholar
  47. S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In PLDI, pages 447–458, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. A. Schrijver. Combinatorial Optimization - Polyhedra and Efficiency. Springer, 2003. ISBN 978-3-540-44389-6.Google ScholarGoogle Scholar
  49. M. Sharir, A. Pnueli, and S. Hart. Verification of probabilistic programs. SIAM J. Comput., 13(2):292–314, 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. K. Sohn and A. V. Gelder. Termination detection in logic programs using argument sizes. In D. J. Rosenkrantz, editor, Proceedings of the Tenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, USA, pages 216–226. ACM Press, 1991. ISBN 0-89791-430-9.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. A. Solar-Lezama, R. M. Rabbah, R. Bodík, and K. Ebcioglu. Programming by sketching for bit-streaming programs. In PLDI, pages 281–294, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. D. Williams. Probability with Martingales. Cambridge University Press, 1991.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 51, Issue 1
        POPL '16
        January 2016
        815 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/2914770
        • Editor:
        • Andy Gill
        Issue’s Table of Contents
        • cover image ACM Conferences
          POPL '16: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages
          January 2016
          815 pages
          ISBN:9781450335492
          DOI:10.1145/2837614

        Copyright © 2016 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 11 January 2016

        Check for updates

        Qualifiers

        • article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!