Abstract
In this paper, we consider termination of probabilistic programs with real-valued variables. The questions concerned are: 1. qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the expected termination time is finite (finite termination); 2. quantitative ones that ask (i) to approximate the expected termination time (expectation problem) and (ii) to compute a bound B such that the probability to terminate after B steps decreases exponentially (concentration problem). To solve these questions, we utilize the notion of ranking supermartingales which is a powerful approach for proving termination of probabilistic programs. In detail, we focus on algorithmic synthesis of linear ranking-supermartingales over affine probabilistic programs (APP's) with both angelic and demonic non-determinism. An important subclass of APP's is LRAPP which is defined as the class of all APP's over which a linear ranking-supermartingale exists. Our main contributions are as follows. Firstly, we show that the membership problem of LRAPP (i) can be decided in polynomial time for APP's with at most demonic non-determinism, and (ii) is NP-hard and in PSPACE for APP's with angelic non-determinism; moreover, the NP-hardness result holds already for APP's without probability and demonic non-determinism. Secondly, we show that the concentration problem over LRAPP can be solved in the same complexity as for the membership problem of LRAPP. Finally, we show that the expectation problem over LRAPP can be solved in 2EXPTIME and is PSPACE-hard even for APP's without probability and non-determinism (i.e., deterministic programs). Our experimental results demonstrate the effectiveness of our approach to answer the qualitative and quantitative questions over APP's with at most demonic non-determinism.
- IBM ILOG CPLEX Optimizer. http://www- 01.ibm.com/software/integration/optimization/cplex-optimizer/, 2010.Google Scholar
- K. Azuma. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, 19(3):357–367, 1967.Google Scholar
Cross Ref
- C. Baier and J.-P. Katoen. Principles of model checking. MIT Press, 2008. ISBN 978-0-262-02649-9. Google Scholar
Digital Library
- G. Bennett. Probability inequalities for the sum of independent random variables. Journal of the American Statistical Association, 57(297): 33–45, 1962.Google Scholar
Cross Ref
- P. Billingsley. Probability and Measure. Wiley, 3rd edition, 1995.Google Scholar
- A. Bockmayr and V. Weispfenning. Solving numerical constraints. In J. A. Robinson and A. Voronkov, editors, Handbook of Automated Reasoning (in 2 volumes), pages 751–842. Elsevier and MIT Press, 2001. ISBN 0-444-50813-9.Google Scholar
- O. Bournez and F. Garnier. Proving positive almost-sure termination. In RTA, pages 323–337, 2005. Google Scholar
Digital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. The polyranking principle. In ICALP, pages 1349–1361, 2005. Google Scholar
Digital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. Linear ranking with reachability. In K. Etessami and S. K. Rajamani, editors, Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005, Proceedings, volume 3576 of Lecture Notes in Computer Science, pages 491–504. Springer, 2005. Google Scholar
Digital Library
- ISBN 3-540-27231-3..Google Scholar
- T. Brázdil, J. Esparza, S. Kiefer, and A. Kuˇcera. Analyzing Probabilistic Pushdown Automata. FMSD, 43(2):124–163, 2012. Google Scholar
Digital Library
- J. Canny. Some algebraic and geometric computations in pspace. In Proceedings of the twentieth annual ACM symposium on Theory of computing, pages 460–467. ACM, 1988. Google Scholar
Digital Library
- A. Chakarov and S. Sankaranarayanan. Probabilistic program analysis with martingales. In N. Sharygina and H. Veith, editors, Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. Proceedings, volume 8044 of Lecture Notes in Computer Science, pages 511–526. Springer, 2013. ISBN 978-3-642-39798-1.. Google Scholar
Digital Library
- K. Chatterjee, H. Fu, P. Novotný, and R. Hasheminezhad. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. CoRR, abs/1510.08517, 2015. URL http: //arxiv.org/abs/1510.08517. Google Scholar
Digital Library
- F. Chung and L. Lu. Concentration inequalities and martingale inequalities: A survey. Internet Mathematics, 3:79–127, 2011.Google Scholar
Cross Ref
- M. Colón and H. Sipma. Synthesis of linear ranking functions. In T. Margaria and W. Yi, editors, Tools and Algorithms for the Construction and Analysis of Systems, 7th International Conference, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings, volume 2031 of Lecture Notes in Computer Science, pages 67–81. Springer, 2001. ISBN 3-540-41865-2.. Google Scholar
Digital Library
- B. Cook, A. See, and F. Zuleger. Ramsey vs. lexicographic termination proving. In TACAS, pages 47–61, 2013. Google Scholar
Digital Library
- P. Cousot and R. Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In R. M. Graham, M. A. Harrison, and R. Sethi, editors, Conference Record of the Fourth ACM Symposium on Principles of Programming Languages, Los Angeles, California, USA, January 1977, pages 238–252. ACM, 1977.. Google Scholar
Digital Library
- D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1st edition, 2009. ISBN 0521884276, 9780521884273. Google Scholar
Digital Library
- R. Durrett. Probability: Theory and Examples (Second Edition). Duxbury Press, 1996.Google Scholar
- J. Esparza, A. Kuˇcera, and R. Mayr. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In LICS, pages 117–126. IEEE, 2005. Google Scholar
Digital Library
- J. Esparza, A. Gaiser, and S. Kiefer. Proving termination of probabilistic programs using patterns. In CAV, pages 123–138, 2012. Google Scholar
Digital Library
- J. Farkas. A fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö, 12:457–472, 1894.Google Scholar
- L. M. F. Fioriti and H. Hermanns. Probabilistic termination: Soundness, completeness, and compositionality. In S. K. Rajamani and D. Walker, editors, Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, pages 489–501. ACM, 2015. ISBN 978-1-4503-3300-9.. Google Scholar
Digital Library
- R. W. Floyd. Assigning meanings to programs. Mathematical Aspects of Computer Science, 19:19–33, 1967.Google Scholar
Cross Ref
- F. G. Foster. On the stochastic matrices associated with certain queuing processes. The Annals of Mathematical Statistics, 24(3):pp. 355–360, 1953.Google Scholar
Cross Ref
- S. Hart and M. Sharir. Concurrent probabilistic programs, or: How to schedule if you must. SIAM J. Comput., 14(4):991–1012, 1985.Google Scholar
Digital Library
- W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301): 13–30, 1963.Google Scholar
Cross Ref
- H. Howard. Dynamic Programming and Markov Processes. MIT Press, 1960.Google Scholar
- L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237– 285, 1996. Google Scholar
Digital Library
- L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1):99–134, 1998. Google Scholar
Digital Library
- J. Katoen, A. McIver, L. Meinicke, and C. C. Morgan. Linear-invariant generation for probabilistic programs: - automated support for proofbased methods. In SAS, volume LNCS 6337, Springer, pages 390–406, 2010. Google Scholar
Digital Library
- J. Kemeny, J. Snell, and A. Knapp. Denumerable Markov Chains. D. Van Nostrand Company, 1966.Google Scholar
- H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. Temporal-logicbased reactive mission and motion planning. IEEE Transactions on Robotics, 25(6):1370–1381, 2009. Google Scholar
Digital Library
- M. Z. Kwiatkowska, G. Norman, and D. Parker. Prism 4.0: Verification of probabilistic real-time systems. In CAV, LNCS 6806, pages 585–591, 2011. Google Scholar
Digital Library
- C. S. Lee, N. D. Jones, and A. M. Ben-Amram. The size-change principle for program termination. In POPL, pages 81–92, 2001. Google Scholar
Digital Library
- C. McDiarmid. Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics, pages 195–248. 1998.Google Scholar
- A. McIver and C. Morgan. Developing and reasoning about probabilistic programs in pGCL. In PSSE, pages 123–155, 2004.Google Scholar
- A. McIver and C. Morgan. Abstraction, Refinement and Proof for Probabilistic Systems. Monographs in Computer Science. Springer, 2005. Google Scholar
Digital Library
- D. Monniaux. An abstract analysis of the probabilistic termination of programs. In P. Cousot, editor, Static Analysis, 8th International Symposium, SAS 2001, Paris, France, July 16-18, 2001, Proceedings, volume 2126 of Lecture Notes in Computer Science, pages 111–126. Springer, 2001. ISBN 3-540-42314-1.. URL http://dx.doi. org/10.1007/3-540-47764-0_7. Google Scholar
Digital Library
- R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1995. ISBN 0-521-47465-5, 9780521474658. Google Scholar
Digital Library
- T. S. Motzkin. Beiträge zur Theorie der linearen Ungleichungen (German). PhD thesis, Basel, Jerusalem, 1936.Google Scholar
- A. Paz. Introduction to probabilistic automata (Computer science and applied mathematics). Academic Press, 1971. Google Scholar
Digital Library
- A. Podelski and A. Rybalchenko. A complete method for the synthesis of linear ranking functions. In B. Steffen and G. Levi, editors, Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, January 11-13, 2004, Proceedings, volume 2937 of Lecture Notes in Computer Science, pages 239–251. Springer, 2004. ISBN 3-540-20803-8..Google Scholar
- M. Rabin. Probabilistic automata. Information and Control, 6:230–245, 1963.Google Scholar
Cross Ref
- J. S. Rosenthal. A First Look at Rigorous Probability Theory. World Scientific Publishing Company, 2nd edition, 2006.Google Scholar
- S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In PLDI, pages 447–458, 2013. Google Scholar
Digital Library
- A. Schrijver. Combinatorial Optimization - Polyhedra and Efficiency. Springer, 2003. ISBN 978-3-540-44389-6.Google Scholar
- M. Sharir, A. Pnueli, and S. Hart. Verification of probabilistic programs. SIAM J. Comput., 13(2):292–314, 1984. Google Scholar
Digital Library
- K. Sohn and A. V. Gelder. Termination detection in logic programs using argument sizes. In D. J. Rosenkrantz, editor, Proceedings of the Tenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, USA, pages 216–226. ACM Press, 1991. ISBN 0-89791-430-9.. Google Scholar
Digital Library
- A. Solar-Lezama, R. M. Rabbah, R. Bodík, and K. Ebcioglu. Programming by sketching for bit-streaming programs. In PLDI, pages 281–294, 2005. Google Scholar
Digital Library
- D. Williams. Probability with Martingales. Cambridge University Press, 1991.Google Scholar
Cross Ref
Index Terms
Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs
Recommendations
Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs
In this article, we consider the termination problem of probabilistic programs with real-valued variables. The questions concerned are: qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) ...
Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs
POPL '16: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming LanguagesIn this paper, we consider termination of probabilistic programs with real-valued variables. The questions concerned are: 1. qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) whether the ...
Stochastic invariants for probabilistic termination
POPL '17Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with ...






Comments