Abstract
In this article, we consider the termination problem of probabilistic programs with real-valued variables. The questions concerned are: qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the expected termination time is finite (finite termination); and quantitative ones that ask (i) to approximate the expected termination time (expectation problem) and (ii) to compute a bound B such that the probability not to terminate after B steps decreases exponentially (concentration problem). To solve these questions, we utilize the notion of ranking supermartingales, which is a powerful approach for proving termination of probabilistic programs. In detail, we focus on algorithmic synthesis of linear ranking-supermartingales over affine probabilistic programs (Apps) with both angelic and demonic non-determinism. An important subclass of Apps is LRApp which is defined as the class of all Apps over which a linear ranking-supermartingale exists.
Our main contributions are as follows. Firstly, we show that the membership problem of LRApp (i) can be decided in polynomial time for Apps with at most demonic non-determinism, and (ii) is NP-hard and in PSPACE for Apps with angelic non-determinism. Moreover, the NP-hardness result holds already for Apps without probability and demonic non-determinism. Secondly, we show that the concentration problem over LRApp can be solved in the same complexity as for the membership problem of LRApp. Finally, we show that the expectation problem over LRApp can be solved in 2EXPTIME and is PSPACE-hard even for Apps without probability and non-determinism (i.e., deterministic programs). Our experimental results demonstrate the effectiveness of our approach to answer the qualitative and quantitative questions over Apps with at most demonic non-determinism.
Supplemental Material
Available for Download
- K. Azuma. 1967. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal 19, 3 (1967), 357--367.Google Scholar
Cross Ref
- Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. MIT Press. I--XVII, 1--975 pages.Google Scholar
Digital Library
- Gilles Barthe, Thomas Espitau, Luis María Ferrer Fioriti, and Justin Hsu. 2016. Synthesizing probabilistic invariants via Doob’s decomposition. In Proceedings of the 28th International Conference of Computer Aided Verification (CAV’16), Part I (Lecture Notes in Computer Science), Swarat Chaudhuri and Azadeh Farzan (Eds.), Vol. 9779. Springer, 43--61.Google Scholar
Cross Ref
- George Bennett. 1962. Probability inequalities for the sum of independent random variables. J. Amer. Statist. Assoc. 57, 297 (1962), 33--45.Google Scholar
Cross Ref
- P Billingsley. 1995. Probability and Measure (3rd ed.). Wiley.Google Scholar
- Alexander Bockmayr and Volker Weispfenning. 2001. Solving numerical constraints. In Handbook of Automated Reasoning (in 2 volumes), John Alan Robinson and Andrei Voronkov (Eds.). Elsevier and MIT Press, 751--842.Google Scholar
- Olivier Bournez and Florent Garnier. 2005. Proving positive almost-sure termination. In RTA. 323--337. Google Scholar
Digital Library
- Aaron R. Bradley, Zohar Manna, and Henny B. Sipma. 2005a. Linear ranking with reachability. In Proceedings of the 17th International Conference of Computer Aided Verification (CAV’05), (Lecture Notes in Computer Science), Kousha Etessami and Sriram K. Rajamani (Eds.), Vol. 3576. Springer, 491--504. Google Scholar
Digital Library
- Aaron R. Bradley, Zohar Manna, and Henny B. Sipma. 2005b. The polyranking principle. In ICALP. 1349--1361. Google Scholar
Digital Library
- T. Brázdil, J. Esparza, S. Kiefer, and A. Kučera. 2012. Analyzing probabilistic pushdown automata. FMSD 43, 2 (2012), 124--163. Google Scholar
Digital Library
- John Canny. 1988. Some algebraic and geometric computations in PSPACE. In Proceedings of the 20th Annual ACM Symposium on Theory of Computing. ACM, 460--467. Google Scholar
Digital Library
- Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic program analysis with martingales. In Proceedings of the 25th International Conference of Computer Aided Verification (CAV’13) (Lecture Notes in Computer Science), Natasha Sharygina and Helmut Veith (Eds.), Vol. 8044. Springer, 511--526.Google Scholar
Cross Ref
- Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’16), Rastislav Bodik and Rupak Majumdar (Eds.). ACM, 327--342. Google Scholar
Digital Library
- Fan Chung and Linyuan Lu. 2011. Concentration inequalities and martingale inequalities: A survey. Internet Mathematics 3 (2011), 79--127.Google Scholar
Cross Ref
- Michael Colón and Henny Sipma. 2001. Synthesis of linear ranking functions. In Proceedings of the 7th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’01), (Lecture Notes in Computer Science Tiziana Margaria and Wang Yi (Eds.), Vol. 2031. Springer, 67--81.Google Scholar
- Byron Cook, Abigail See, and Florian Zuleger. 2013. Ramsey vs. lexicographic termination proving. In TACAS. 47--61. Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 1977. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Conference Record of the 4th ACM Symposium on Principles of Programming Languages, Robert M. Graham, Michael A. Harrison, and Ravi Sethi (Eds.). ACM, 238--252. Google Scholar
Digital Library
- Devdatt Dubhashi and Alessandro Panconesi. 2009. Concentration of Measure for the Analysis of Randomized Algorithms (1st ed.). Cambridge University Press, New York, NY. Google Scholar
Digital Library
- R. Durrett. 1996. Probability: Theory and Examples (2nd ed.). Duxbury Press.Google Scholar
- Javier Esparza, Andreas Gaiser, and Stefan Kiefer. 2012. Proving termination of probabilistic programs using patterns. In CAV. 123--138. Google Scholar
Digital Library
- J Esparza, A Kučera, and R Mayr. 2005. Quantitative analysis of probabilistic pushdown automata: Expectations and variances. In LICS. IEEE, 117--126. Google Scholar
Digital Library
- J. Farkas. 1894. A fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö 12 (1894), 457--472.Google Scholar
- Luis María Ferrer Fioriti and Holger Hermanns. 2015. Probabilistic termination: Soundness, completeness, and compositionality. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’15), Sriram K. Rajamani and David Walker (Eds.). ACM, 489--501. Google Scholar
Digital Library
- Robert W. Floyd. 1967. Assigning meanings to programs. Mathematical Aspects of Computer Science 19 (1967), 19--33.Google Scholar
Cross Ref
- F. G. Foster. 1953. On the stochastic matrices associated with certain queuing processes. The Annals of Mathematical Statistics 24, 3 (1953), 355--360.Google Scholar
Cross Ref
- Sergiu Hart and Micha Sharir. 1985. Concurrent probabilistic programs, or: How to schedule if you must. SIAM Journal of Computing 14, 4 (1985), 991--1012.Google Scholar
Digital Library
- Wassily Hoeffding. 1963. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58, 301 (1963), 13--30.Google Scholar
Cross Ref
- H. Howard. 1960. Dynamic Programming and Markov Processes. MIT Press.Google Scholar
- IBM 2010. IBM ILOG CPLEX Optimizer. Retrieved from http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/.Google Scholar
- L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 1 (1998), 99--134. Google Scholar
Digital Library
- L. P. Kaelbling, M. L. Littman, and A. W. Moore. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996), 237--285. Google Scholar
Digital Library
- Joost-Pieter Katoen, Annabelle McIver, Larissa Meinicke, and Carroll C. Morgan. 2010. Linear-invariant generation for probabilistic programs: Automated support for proof-based methods. In SAS, Vol. LNCS 6337, Springer. 390--406. Google Scholar
Digital Library
- J.G. Kemeny, J.L. Snell, and A.W. Knapp. 1966. Denumerable Markov Chains. D. Van Nostrand Company.Google Scholar
- H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. 2009. Temporal-logic-based reactive mission and motion planning. IEEE Transactions on Robotics 25, 6 (2009), 1370--1381. Google Scholar
Digital Library
- Marta Z. Kwiatkowska, Gethin Norman, and David Parker. 2011. PRISM 4.0: Verification of probabilistic real-time systems. In CAV (LNCS 6806). 585--591. Google Scholar
Digital Library
- Chin Soon Lee, Neil D. Jones, and Amir M. Ben-Amram. 2001. The size-change principle for program termination. In POPL. 81--92. Google Scholar
Digital Library
- Colin McDiarmid. 1998. Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics. 195--248.Google Scholar
- Annabelle McIver and Carroll Morgan. 2004. Developing and reasoning about probabilistic programs in pGCL. In PSSE. 123--155.Google Scholar
- Annabelle McIver and Carroll Morgan. 2005. Abstraction, Refinement and Proof for Probabilistic Systems. Springer.Google Scholar
- S. Meyn and R.L. Tweedie. 2009. Markov Chains and Stochastic Stability. Cambridge. Google Scholar
Digital Library
- David Monniaux. 2001. An abstract analysis of the probabilistic termination of programs. In Proceedings of the 8th International Symposium on Static Analysis (SAS’01), SAS 2001, Paris, France, July 16-18, 2001, (Lecture Notes in Computer Science), Patrick Cousot (Ed.), Vol. 2126. Springer, 111--126. Google Scholar
Digital Library
- Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press, New York, NY. Google Scholar
Digital Library
- T. S. Motzkin. 1936. Beiträge zur Theorie der Linearen Ungleichungen (German). Ph.D. Dissertation. Basel, Jerusalem.Google Scholar
- Martin J. Osborne and Ariel Rubinstein. 1994. A Course in Game Theory.Google Scholar
- A. Paz. 1971. Introduction to Probabilistic Automata (Computer Science and Applied Mathematics). Academic Press. Google Scholar
Digital Library
- Andreas Podelski and Andrey Rybalchenko. 2004. A complete method for the synthesis of linear ranking functions. In Proceedings of the 5th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI’04), (Lecture Notes in Computer Science), Bernhard Steffen and Giorgio Levi (Eds.), Vol. 2937. Springer, 239--251.Google Scholar
Cross Ref
- M. O. Rabin. 1963. Probabilistic automata. Information and Control 6 (1963), 230--245.Google Scholar
Cross Ref
- Jeffrey S. Rosenthal. 2006. A First Look at Rigorous Probability Theory (2nd ed.). World Scientific Publishing Company.Google Scholar
Cross Ref
- Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static analysis for probabilistic programs: Inferring whole program properties from finitely many paths. In PLDI. 447--458. Google Scholar
Digital Library
- Alexander Schrijver. 1999. Theory of Linear and Integer Programming. Wiley. Google Scholar
Digital Library
- Alexander Schrijver. 2003. Combinatorial Optimization—Polyhedra and Efficiency. Springer.Google Scholar
- Micha Sharir, Amir Pnueli, and Sergiu Hart. 1984. Verification of probabilistic programs. SIAM J. Comput. 13, 2 (1984), 292--314. Google Scholar
Digital Library
- Kirack Sohn and Allen Van Gelder. 1991. Termination detection in logic programs using argument sizes. In Proceedings of the 10th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, Daniel J. Rosenkrantz (Ed.). ACM Press, 216--226. Google Scholar
Digital Library
- Armando Solar-Lezama, Rodric M. Rabbah, Rastislav Bodík, and Kemal Ebcioglu. 2005. Programming by sketching for bit-streaming programs. In PLDI. 281--294. Google Scholar
Digital Library
- Stephen A. Vavasis. 1992. Approximation algorithms for indefinite quadratic programming. Mathematical Programming 57 (1992), 279--311. Google Scholar
Digital Library
- David Williams. 1991. Probability with Martingales. Cambridge University Press.Google Scholar
Index Terms
Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs
Recommendations
Modular verification for almost-sure termination of probabilistic programs
In this work, we consider the almost-sure termination problem for probabilistic programs that asks whether a given probabilistic program terminates with probability 1. Scalable approaches for program analysis often rely on modularity as their ...
Lexicographic ranking supermartingales: an efficient approach to termination of probabilistic programs
Probabilistic programs extend classical imperative programs with real-valued random variables and random branching. The most basic liveness property for such programs is the termination property. The qualitative (aka almost-sure) termination problem ...
Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs
POPL '16: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming LanguagesIn this paper, we consider termination of probabilistic programs with real-valued variables. The questions concerned are: 1. qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the ...






Comments