skip to main content
research-article
Free Access

Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs

Published:28 May 2018Publication History
Skip Abstract Section

Abstract

In this article, we consider the termination problem of probabilistic programs with real-valued variables. The questions concerned are: qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the expected termination time is finite (finite termination); and quantitative ones that ask (i) to approximate the expected termination time (expectation problem) and (ii) to compute a bound B such that the probability not to terminate after B steps decreases exponentially (concentration problem). To solve these questions, we utilize the notion of ranking supermartingales, which is a powerful approach for proving termination of probabilistic programs. In detail, we focus on algorithmic synthesis of linear ranking-supermartingales over affine probabilistic programs (Apps) with both angelic and demonic non-determinism. An important subclass of Apps is LRApp which is defined as the class of all Apps over which a linear ranking-supermartingale exists.

Our main contributions are as follows. Firstly, we show that the membership problem of LRApp (i) can be decided in polynomial time for Apps with at most demonic non-determinism, and (ii) is NP-hard and in PSPACE for Apps with angelic non-determinism. Moreover, the NP-hardness result holds already for Apps without probability and demonic non-determinism. Secondly, we show that the concentration problem over LRApp can be solved in the same complexity as for the membership problem of LRApp. Finally, we show that the expectation problem over LRApp can be solved in 2EXPTIME and is PSPACE-hard even for Apps without probability and non-determinism (i.e., deterministic programs). Our experimental results demonstrate the effectiveness of our approach to answer the qualitative and quantitative questions over Apps with at most demonic non-determinism.

Skip Supplemental Material Section

Supplemental Material

References

  1. K. Azuma. 1967. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal 19, 3 (1967), 357--367.Google ScholarGoogle ScholarCross RefCross Ref
  2. Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. MIT Press. I--XVII, 1--975 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Gilles Barthe, Thomas Espitau, Luis María Ferrer Fioriti, and Justin Hsu. 2016. Synthesizing probabilistic invariants via Doob’s decomposition. In Proceedings of the 28th International Conference of Computer Aided Verification (CAV’16), Part I (Lecture Notes in Computer Science), Swarat Chaudhuri and Azadeh Farzan (Eds.), Vol. 9779. Springer, 43--61.Google ScholarGoogle ScholarCross RefCross Ref
  4. George Bennett. 1962. Probability inequalities for the sum of independent random variables. J. Amer. Statist. Assoc. 57, 297 (1962), 33--45.Google ScholarGoogle ScholarCross RefCross Ref
  5. P Billingsley. 1995. Probability and Measure (3rd ed.). Wiley.Google ScholarGoogle Scholar
  6. Alexander Bockmayr and Volker Weispfenning. 2001. Solving numerical constraints. In Handbook of Automated Reasoning (in 2 volumes), John Alan Robinson and Andrei Voronkov (Eds.). Elsevier and MIT Press, 751--842.Google ScholarGoogle Scholar
  7. Olivier Bournez and Florent Garnier. 2005. Proving positive almost-sure termination. In RTA. 323--337. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Aaron R. Bradley, Zohar Manna, and Henny B. Sipma. 2005a. Linear ranking with reachability. In Proceedings of the 17th International Conference of Computer Aided Verification (CAV’05), (Lecture Notes in Computer Science), Kousha Etessami and Sriram K. Rajamani (Eds.), Vol. 3576. Springer, 491--504. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Aaron R. Bradley, Zohar Manna, and Henny B. Sipma. 2005b. The polyranking principle. In ICALP. 1349--1361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. T. Brázdil, J. Esparza, S. Kiefer, and A. Kučera. 2012. Analyzing probabilistic pushdown automata. FMSD 43, 2 (2012), 124--163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. John Canny. 1988. Some algebraic and geometric computations in PSPACE. In Proceedings of the 20th Annual ACM Symposium on Theory of Computing. ACM, 460--467. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic program analysis with martingales. In Proceedings of the 25th International Conference of Computer Aided Verification (CAV’13) (Lecture Notes in Computer Science), Natasha Sharygina and Helmut Veith (Eds.), Vol. 8044. Springer, 511--526.Google ScholarGoogle ScholarCross RefCross Ref
  13. Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’16), Rastislav Bodik and Rupak Majumdar (Eds.). ACM, 327--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Fan Chung and Linyuan Lu. 2011. Concentration inequalities and martingale inequalities: A survey. Internet Mathematics 3 (2011), 79--127.Google ScholarGoogle ScholarCross RefCross Ref
  15. Michael Colón and Henny Sipma. 2001. Synthesis of linear ranking functions. In Proceedings of the 7th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’01), (Lecture Notes in Computer Science Tiziana Margaria and Wang Yi (Eds.), Vol. 2031. Springer, 67--81.Google ScholarGoogle Scholar
  16. Byron Cook, Abigail See, and Florian Zuleger. 2013. Ramsey vs. lexicographic termination proving. In TACAS. 47--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Patrick Cousot and Radhia Cousot. 1977. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Conference Record of the 4th ACM Symposium on Principles of Programming Languages, Robert M. Graham, Michael A. Harrison, and Ravi Sethi (Eds.). ACM, 238--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Devdatt Dubhashi and Alessandro Panconesi. 2009. Concentration of Measure for the Analysis of Randomized Algorithms (1st ed.). Cambridge University Press, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. R. Durrett. 1996. Probability: Theory and Examples (2nd ed.). Duxbury Press.Google ScholarGoogle Scholar
  20. Javier Esparza, Andreas Gaiser, and Stefan Kiefer. 2012. Proving termination of probabilistic programs using patterns. In CAV. 123--138. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J Esparza, A Kučera, and R Mayr. 2005. Quantitative analysis of probabilistic pushdown automata: Expectations and variances. In LICS. IEEE, 117--126. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Farkas. 1894. A fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö 12 (1894), 457--472.Google ScholarGoogle Scholar
  23. Luis María Ferrer Fioriti and Holger Hermanns. 2015. Probabilistic termination: Soundness, completeness, and compositionality. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’15), Sriram K. Rajamani and David Walker (Eds.). ACM, 489--501. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Robert W. Floyd. 1967. Assigning meanings to programs. Mathematical Aspects of Computer Science 19 (1967), 19--33.Google ScholarGoogle ScholarCross RefCross Ref
  25. F. G. Foster. 1953. On the stochastic matrices associated with certain queuing processes. The Annals of Mathematical Statistics 24, 3 (1953), 355--360.Google ScholarGoogle ScholarCross RefCross Ref
  26. Sergiu Hart and Micha Sharir. 1985. Concurrent probabilistic programs, or: How to schedule if you must. SIAM Journal of Computing 14, 4 (1985), 991--1012.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Wassily Hoeffding. 1963. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58, 301 (1963), 13--30.Google ScholarGoogle ScholarCross RefCross Ref
  28. H. Howard. 1960. Dynamic Programming and Markov Processes. MIT Press.Google ScholarGoogle Scholar
  29. IBM 2010. IBM ILOG CPLEX Optimizer. Retrieved from http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/.Google ScholarGoogle Scholar
  30. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 1 (1998), 99--134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. L. P. Kaelbling, M. L. Littman, and A. W. Moore. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996), 237--285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Joost-Pieter Katoen, Annabelle McIver, Larissa Meinicke, and Carroll C. Morgan. 2010. Linear-invariant generation for probabilistic programs: Automated support for proof-based methods. In SAS, Vol. LNCS 6337, Springer. 390--406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. J.G. Kemeny, J.L. Snell, and A.W. Knapp. 1966. Denumerable Markov Chains. D. Van Nostrand Company.Google ScholarGoogle Scholar
  34. H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. 2009. Temporal-logic-based reactive mission and motion planning. IEEE Transactions on Robotics 25, 6 (2009), 1370--1381. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marta Z. Kwiatkowska, Gethin Norman, and David Parker. 2011. PRISM 4.0: Verification of probabilistic real-time systems. In CAV (LNCS 6806). 585--591. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Chin Soon Lee, Neil D. Jones, and Amir M. Ben-Amram. 2001. The size-change principle for program termination. In POPL. 81--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Colin McDiarmid. 1998. Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics. 195--248.Google ScholarGoogle Scholar
  38. Annabelle McIver and Carroll Morgan. 2004. Developing and reasoning about probabilistic programs in pGCL. In PSSE. 123--155.Google ScholarGoogle Scholar
  39. Annabelle McIver and Carroll Morgan. 2005. Abstraction, Refinement and Proof for Probabilistic Systems. Springer.Google ScholarGoogle Scholar
  40. S. Meyn and R.L. Tweedie. 2009. Markov Chains and Stochastic Stability. Cambridge. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. David Monniaux. 2001. An abstract analysis of the probabilistic termination of programs. In Proceedings of the 8th International Symposium on Static Analysis (SAS’01), SAS 2001, Paris, France, July 16-18, 2001, (Lecture Notes in Computer Science), Patrick Cousot (Ed.), Vol. 2126. Springer, 111--126. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. T. S. Motzkin. 1936. Beiträge zur Theorie der Linearen Ungleichungen (German). Ph.D. Dissertation. Basel, Jerusalem.Google ScholarGoogle Scholar
  44. Martin J. Osborne and Ariel Rubinstein. 1994. A Course in Game Theory.Google ScholarGoogle Scholar
  45. A. Paz. 1971. Introduction to Probabilistic Automata (Computer Science and Applied Mathematics). Academic Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Andreas Podelski and Andrey Rybalchenko. 2004. A complete method for the synthesis of linear ranking functions. In Proceedings of the 5th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI’04), (Lecture Notes in Computer Science), Bernhard Steffen and Giorgio Levi (Eds.), Vol. 2937. Springer, 239--251.Google ScholarGoogle ScholarCross RefCross Ref
  47. M. O. Rabin. 1963. Probabilistic automata. Information and Control 6 (1963), 230--245.Google ScholarGoogle ScholarCross RefCross Ref
  48. Jeffrey S. Rosenthal. 2006. A First Look at Rigorous Probability Theory (2nd ed.). World Scientific Publishing Company.Google ScholarGoogle ScholarCross RefCross Ref
  49. Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static analysis for probabilistic programs: Inferring whole program properties from finitely many paths. In PLDI. 447--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Alexander Schrijver. 1999. Theory of Linear and Integer Programming. Wiley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Alexander Schrijver. 2003. Combinatorial Optimization—Polyhedra and Efficiency. Springer.Google ScholarGoogle Scholar
  52. Micha Sharir, Amir Pnueli, and Sergiu Hart. 1984. Verification of probabilistic programs. SIAM J. Comput. 13, 2 (1984), 292--314. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Kirack Sohn and Allen Van Gelder. 1991. Termination detection in logic programs using argument sizes. In Proceedings of the 10th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, Daniel J. Rosenkrantz (Ed.). ACM Press, 216--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Armando Solar-Lezama, Rodric M. Rabbah, Rastislav Bodík, and Kemal Ebcioglu. 2005. Programming by sketching for bit-streaming programs. In PLDI. 281--294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Stephen A. Vavasis. 1992. Approximation algorithms for indefinite quadratic programming. Mathematical Programming 57 (1992), 279--311. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. David Williams. 1991. Probability with Martingales. Cambridge University Press.Google ScholarGoogle Scholar

Index Terms

  1. Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Programming Languages and Systems
      ACM Transactions on Programming Languages and Systems  Volume 40, Issue 2
      June 2018
      223 pages
      ISSN:0164-0925
      EISSN:1558-4593
      DOI:10.1145/3229520
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 28 May 2018
      • Accepted: 1 December 2017
      • Revised: 1 October 2017
      • Received: 1 February 2016
      Published in toplas Volume 40, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!