Abstract
Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with probability 1 (almost-sure termination). A powerful approach for this qualitative problem is the notion of ranking supermartingales with respect to a given set of invariants. The quantitative problem (probabilistic termination) asks for bounds on the termination probability, and this problem has not been addressed yet. A fundamental and conceptual drawback of the existing approaches to address probabilistic termination is that even though the supermartingales consider the probabilistic behaviour of the programs, the invariants are obtained completely ignoring the probabilistic aspect (i.e., the invariants are obtained considering all behaviours with no information about the probability).
In this work we address the probabilistic termination problem for linear-arithmetic probabilistic programs with nondeterminism. We formally define the notion of stochastic invariants, which are constraints along with a probability bound that the constraints hold. We introduce a concept of repulsing supermartingales. First, we show that repulsing supermartingales can be used to obtain bounds on the probability of the stochastic invariants. Second, we show the effectiveness of repulsing supermartingales in the following three ways: (1) With a combination of ranking and repulsing supermartingales we can compute lower bounds on the probability of termination; (2) repulsing supermartingales provide witnesses for refutation of almost-sure termination; and (3) with a combination of ranking and repulsing supermartingales we can establish persistence properties of probabilistic programs.
Along with our conceptual contributions, we establish the following computational results: First, the synthesis of a stochastic invariant which supports some ranking supermartingale and at the same time admits a repulsing supermartingale can be achieved via reduction to the existential first-order theory of reals, which generalizes existing results from the non-probabilistic setting. Second, given a program with "strict invariants" (e.g., obtained via abstract interpretation) and a stochastic invariant, we can check in polynomial time whether there exists a linear repulsing supermartingale w.r.t. the stochastic invariant (via reduction to LP). We also present experimental evaluation of our approach on academic examples.
- IBM ILOG CPLEX Optimizer. http://www-01.ibm.com/software/integration/optimization/cplexoptimizer/, 2010.Google Scholar
- A. Abate, J.-P. Katoen, J. Lygeros, and M. Prandini. Approximate Model Checking of Stochastic Hybrid Systems. European Journal of Control, 16(6):624–641, 2010. ISSN 0947-3580.Google Scholar
Cross Ref
- M. F. Atig, A. Bouajjani, M. Emmi, and A. Lal. Detecting Fair Nontermination in Multithreaded Programs, pages 210–226. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-31424- 7. Google Scholar
Digital Library
- K. Azuma. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, Second Series, 19(3):357–367, 1967.Google Scholar
- C. Baier and J.-P. Katoen. Principles of model checking. MIT Press, 2008. ISBN 978-0-262-02649-9. Google Scholar
Digital Library
- G. Barthe, T. Espitau, L. M. F. Fioriti, and J. Hsu. Synthesizing Probabilistic Invariants via Doob’s Decomposition. In Computer Aided Verification - 28th International Conference, CAV 2016, Toronto, ON, Canada, July 17-23, 2016, Proceedings, Part I, pages 43–61, 2016.Google Scholar
- M. Berkelaar, K. Eikland, and P. Notebaert. lp_solve: Open source (Mixed-Integer) Linear Programming system.Google Scholar
- P. Billingsley. Probability and Measure. 1995.Google Scholar
- O. Bournez and F. Garnier. Proving Positive Almost-Sure Termination. In RTA, pages 323–337, 2005. Google Scholar
Digital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. The Polyranking Principle. In ICALP, pages 1349–1361, 2005. Google Scholar
Digital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. Linear Ranking with Reachability. In K. Etessami and S. K. Rajamani, editors, Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005, Proceedings, volume 3576 of Lecture Notes in Computer Science, pages 491–504. Springer, 2005. Google Scholar
Digital Library
- ISBN 3-540-27231-3.Google Scholar
- T. Brázdil, V. Brožek, K. Etessami, and A. Kuˇcera. Approximating the termination value of one-counter MDPs and stochastic games. Inf. Comput., 222:121–138, 2013. Google Scholar
Digital Library
- T. Brázdil, S. Kiefer, and A. Kuˇcera. Efficient Analysis of Probabilistic Programs with an Unbounded Counter. J. ACM, 61(6):41:1–41:35, Dec. 2014. ISSN 0004-5411. Google Scholar
Digital Library
- T. Brázdil, S. Kiefer, A. Kuˇcera, P. Novotný, and J.-P. Katoen. Zero-Reachability in Probabilistic Multi-Counter Automata. In Proceedings of LICS 2014, 2014. Google Scholar
Digital Library
- A. Chakarov and S. Sankaranarayanan. Probabilistic Program Analysis with Martingales. In N. Sharygina and H. Veith, editors, Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. Proceedings, volume 8044 of Lecture Notes in Computer Science, pages 511–526. Springer, 2013. ISBN 978-3-642-39798-1.Google Scholar
- A. Chakarov and S. Sankaranarayanan. Expectation Invariants for Probabilistic Program Loops as Fixed Points, pages 85–100. Springer International Publishing, 2014. ISBN 978-3-319-10936-7.Google Scholar
- A. Chakarov, Y.-L. Voronin, and S. Sankaranarayanan. Deductive Proofs of Almost Sure Persistence and Recurrence Properties, pages 260–279. Springer Berlin Heidelberg, Berlin, Heidelberg, 2016. ISBN 978-3-662-49674-9. Google Scholar
Digital Library
- K. Chatterjee, H. Fu, and A. K. Goharshady. Termination Analysis of Probabilistic Programs through Positivstellensatz’s. CoRR, abs/1604.07169, 2016.Google Scholar
- K. Chatterjee, H. Fu, P. Novotný, and R. Hasheminezhad. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In R. Bodík and R. Majumdar, editors, Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016, pages 327–342. ACM, 2016. ISBN 978-1-4503-3549-2. Google Scholar
Digital Library
- K. Chatterjee, P. Novotný, and Ð. Žikeli´c. Stochastic Invariants for Probabilistic Termination. CoRR, abs/1611.01063, 2016. Google Scholar
Digital Library
- H.-Y. Chen, B. Cook, C. Fuhs, K. Nimkar, and P. O’Hearn. Proving Nontermination via Safety, pages 156–171. Springer Berlin Heidelberg, Berlin, Heidelberg, 2014. ISBN 978-3-642-54862-8.Google Scholar
- M. Colón and H. Sipma. Synthesis of Linear Ranking Functions. In T. Margaria and W. Yi, editors, Tools and Algorithms for the Construction and Analysis of Systems, 7th International Conference, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings, volume 2031 of Lecture Notes in Computer Science, pages 67–81. Springer, 2001. ISBN 3-540-41865-2. Google Scholar
Digital Library
- M. A. Colón and H. B. Sipma. Practical Methods for Proving Program Termination, pages 442–454. Springer Berlin Heidelberg, Berlin, Heidelberg, 2002. ISBN 978-3-540-45657-5.Google Scholar
- M. A. Colón, S. Sankaranarayanan, and H. B. Sipma. Linear invariant generation using non-linear constraint solving. In International Conference on Computer Aided Verification, pages 420–432. Springer, 2003.Google Scholar
- B. Cook, A. See, and F. Zuleger. Ramsey vs. Lexicographic Termination Proving. In TACAS, pages 47–61, 2013. Google Scholar
Digital Library
- P. Cousot and R. Cousot. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In R. M. Graham, M. A. Harrison, and R. Sethi, editors, Conference Record of the Fourth ACM Symposium on Principles of Programming Languages, Los Angeles, California, USA, January 1977, pages 238–252. ACM, 1977. Google Scholar
Digital Library
- D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. 2009. Google Scholar
Digital Library
- D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1st edition, 2009. ISBN 0521884276, 9780521884273. Google Scholar
Digital Library
- R. Durrett. Probability: Theory and Examples (Second Edition). Duxbury Press, 1996.Google Scholar
- J. Esparza, A. Gaiser, and S. Kiefer. Proving Termination of Probabilistic Programs Using Patterns. In CAV, pages 123–138, 2012. Google Scholar
Digital Library
- J. Farkas. A Fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö, 12:457–472, 1894.Google Scholar
- Y. A. Feldman. A decidable propositional dynamic logic with explicit probabilities. Information and Control, 63(1):11–38, 1984. ISSN 0019-9958. Google Scholar
Digital Library
- Y. A. Feldman and D. Harel. A probabilistic dynamic logic. In Proceedings of the fourteenth annual ACM Symposium on Theory of computing, pages 181–195. ACM, 1982. Google Scholar
Digital Library
- L. M. F. Fioriti and H. Hermanns. Probabilistic Termination: Soundness, Completeness, and Compositionality. In S. K. Rajamani and D. Walker, editors, Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, pages 489–501. ACM, 2015. ISBN 978-1-4503-3300-9. Google Scholar
Digital Library
- R. W. Floyd. Assigning meanings to programs. Mathematical Aspects of Computer Science, 19:19–33, 1967.Google Scholar
Cross Ref
- F. G. Foster. On the Stochastic Matrices Associated with Certain Queuing Processes. The Annals of Mathematical Statistics, 24(3):pp. 355–360, 1953.Google Scholar
Cross Ref
- A. D. Gordon, T. A. Henzinger, A. V. Nori, and S. K. Rajamani. Probabilistic programming. In Proceedings of the on Future of Software Engineering, pages 167–181. ACM, 2014. Google Scholar
Digital Library
- F. Gretz, J.-P. Katoen, and A. McIver. Prinsys - On a Quest for Probabilistic Loop Invariants. In Quantitative Evaluation of Systems - 10th International Conference, QEST 2013, Buenos Aires, Argentina, August 27-30, 2013. Proceedings, pages 193–208, 2013. Google Scholar
Digital Library
- A. Gupta, T. A. Henzinger, R. Majumdar, A. Rybalchenko, and R.-G. Xu. Proving Non-termination. SIGPLAN Not., 43(1):147–158, Jan. 2008. ISSN 0362-1340. Google Scholar
Digital Library
- S. Hart and M. Sharir. Concurrent Probabilistic Programs, Or: How to Schedule if You Must. SIAM J. Comput., 14(4):991–1012, 1985.Google Scholar
Digital Library
- H. Hermanns, B. Wachter, and L. Zhang. Probabilistic CEGAR. In CAV, LNCS 5123, pages 162–175. Springer, 2008. Google Scholar
Digital Library
- H. Howard. Dynamic Programming and Markov Processes. MIT Press, 1960.Google Scholar
- L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237– 285, 1996. Google Scholar
Digital Library
- L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1):99–134, 1998. Google Scholar
Cross Ref
- B. L. Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. Weakest Precondition Reasoning for Expected Run–Times of Probabilistic Programs, pages 364–389. Springer Berlin Heidelberg, Berlin, Heidelberg, 2016. ISBN 978-3-662-49498-1.Google Scholar
- J.-P. Katoen, A. McIver, L. Meinicke, and C. C. Morgan. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In SAS, volume LNCS 6337, Springer, pages 390–406, 2010. Google Scholar
Digital Library
- M. Kattenbelt, M. Kwiatkowska, G. Norman, and D. Parker. Abstraction refinement for probabilistic software. In International Workshop on Verification, Model Checking, and Abstract Interpretation, pages 182–197. Springer, 2009. Google Scholar
Digital Library
- J. Kemeny, J. Snell, and A. Knapp. Denumerable Markov Chains. D. Van Nostrand Company, 1966.Google Scholar
- D. Kozen. Semantics of Probabilistic Programs. Journal of Computer and System Sciences, 22(3):328–350, 1981. ISSN 0022-0000.Google Scholar
Cross Ref
- D. Kozen. A Probabilistic PDL. In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, STOC ’83, pages 291–297, New York, NY, USA, 1983. ACM. ISBN 0-89791-099-0. Google Scholar
Digital Library
- H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. Temporal-Logic-Based Reactive Mission and Motion Planning. IEEE Transactions on Robotics, 25(6):1370–1381, 2009. Google Scholar
Digital Library
- M. Z. Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: Verification of Probabilistic Real-Time Systems. In CAV, LNCS 6806, pages 585–591, 2011. Google Scholar
Digital Library
- D. Larraz, K. Nimkar, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Proving Non-termination Using Max-SMT, pages 779–796. Springer International Publishing, Cham, 2014. ISBN 978-3-319- 08867-9. Google Scholar
Digital Library
- C. S. Lee, N. D. Jones, and A. M. Ben-Amram. The size-change principle for program termination. In POPL, pages 81–92, 2001. Google Scholar
Digital Library
- A. McIver and C. Morgan. Developing and Reasoning About Probabilistic Programs in pGCL. In PSSE, pages 123–155, 2004.Google Scholar
- A. McIver and C. Morgan. Abstraction, Refinement and Proof for Probabilistic Systems. Monographs in Computer Science. Springer, 2005. Google Scholar
Digital Library
- D. Monniaux. An Abstract Analysis of the Probabilistic Termination of Programs. In P. Cousot, editor, Static Analysis, 8th International Symposium, SAS 2001, Paris, France, July 16-18, 2001, Proceedings, volume 2126 of Lecture Notes in Computer Science, pages 111–126. Springer, 2001. ISBN 3-540-42314-1. Google Scholar
Digital Library
- R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1995. ISBN 0-521-47465-5, 9780521474658. Google Scholar
Digital Library
- A. Paz. Introduction to probabilistic automata (Computer science and applied mathematics). Academic Press, 1971. Google Scholar
Digital Library
- A. Podelski and A. Rybalchenko. A Complete Method for the Synthesis of Linear Ranking Functions. In B. Steffen and G. Levi, editors, Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, January 11-13, 2004, Proceedings, volume 2937 of Lecture Notes in Computer Science, pages 239–251. Springer, 2004. ISBN 3-540-20803-8.Google Scholar
- V. R. Pratt. Semantical consideration on floyo-hoare logic. In Foundations of Computer Science, 1976., 17th Annual Symposium on, pages 109–121, Oct 1976. Google Scholar
Digital Library
- M. Rabin. Probabilistic automata. Information and Control, 6:230–245, 1963.Google Scholar
Cross Ref
- J. H. Reif. Logics for Probabilistic Programming (Extended Abstract). In Proceedings of the Twelfth Annual ACM Symposium on Theory of Computing, STOC ’80, pages 8–13, New York, NY, USA, 1980. ACM. Google Scholar
Digital Library
- ISBN 0-89791-017-6.Google Scholar
- A. Sampson, P. Panchekha, T. Mytkowicz, K. S. McKinley, D. Grossman, and L. Ceze. Expressing and verifying probabilistic assertions. In M. F. P. O’Boyle and K. Pingali, editors, ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014, page 14. ACM, 2014. ISBN 978-1-4503-2784-8. Google Scholar
Digital Library
- S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static Analysis for Probabilistic Programs: Inferring Whole Program Properties from Finitely Many Paths. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, pages 447–458, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2014-6. Google Scholar
Digital Library
- S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In PLDI, pages 447–458, 2013. Google Scholar
Digital Library
- M. Sharir, A. Pnueli, and S. Hart. Verification of Probabilistic Programs. SIAM J. Comput., 13(2):292–314, 1984. Google Scholar
Digital Library
- K. Sohn and A. V. Gelder. Termination Detection in Logic Programs using Argument Sizes. In D. J. Rosenkrantz, editor, Proceedings of the Tenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, USA, pages 216–226. ACM Press, 1991. ISBN 0-89791-430-9. Google Scholar
Digital Library
- H. Velroyen and P. Rümmer. Non-termination Checking for Imperative Programs, pages 154–170. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. ISBN 978-3-540-79124-9. Google Scholar
Digital Library
- D. Williams. Probability with Martingales. 1991.Google Scholar
Cross Ref
Index Terms
Stochastic invariants for probabilistic termination
Recommendations
Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs
In this article, we consider the termination problem of probabilistic programs with real-valued variables. The questions concerned are: qualitative ones that ask (i) whether the program terminates with probability 1 (almost-sure termination) and (ii) ...
Lexicographic ranking supermartingales: an efficient approach to termination of probabilistic programs
Probabilistic programs extend classical imperative programs with real-valued random variables and random branching. The most basic liveness property for such programs is the termination property. The qualitative (aka almost-sure) termination problem ...
Stochastic invariants for probabilistic termination
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesTermination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with ...







Comments