skip to main content
research-article
Open Access

Ranking and Repulsing Supermartingales for Reachability in Randomized Programs

Authors Info & Claims
Published:08 June 2021Publication History
Skip Abstract Section

Abstract

Computing reachability probabilities is a fundamental problem in the analysis of randomized programs. This article aims at a comprehensive and comparative account of various martingale-based methods for over- and under-approximating reachability probabilities. Based on the existing works that stretch across different communities (formal verification, control theory, etc.), we offer a unifying account. In particular, we emphasize the role of order-theoretic fixed points—a classic topic in computer science—in the analysis of randomized programs. This leads us to two new martingale-based techniques, too. We also make an experimental comparison using our implementation of template-based synthesis algorithms for those martingales.

References

  1. Alessandro Abate, Joost-Pieter Katoen, John Lygeros, and Maria Prandini. 2010. Approximate model checking of stochastic hybrid systems. Eur. J. Cont. 16, 6 (2010), 624–641.Google ScholarGoogle ScholarCross RefCross Ref
  2. Sheshansh Agrawal, Krishnendu Chatterjee, and Petr Novotný. 2018. Lexicographic ranking supermartingales: An efficient approach to termination of probabilistic programs. Proc. ACM Prog. Lang. 2, POPL (2018), 34:1–34:32.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Christophe Alias, Alain Darte, Paul Feautrier, and Laure Gonnord. 2010. Multi-dimensional rankings, program termination, and complexity bounds of flowchart programs. In Proceedings of the 17th International Static Analysis Symposium, (SAS’10) (Lecture Notes in Computer Science), Radhia Cousot and Matthieu Martel (Eds.), Vol. 6337. Springer, 117–133. DOI:https://doi.org/10.1007/978-3-642-15769-1_8Google ScholarGoogle ScholarCross RefCross Ref
  4. Krzysztof R. Apt and Gordon D. Plotkin. 1986. Countable nondeterminism and random assignment. J. ACM 33, 4 (1986), 724–767. DOI:https://doi.org/10.1145/6490.6494Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. André Arnold and Damian Niwiński. 2001. Rudiments of -Calculus. Elsevier.Google ScholarGoogle Scholar
  6. Martin Avanzini, Ugo Dal Lago, and Akihisa Yamada. 2018. On probabilistic term rewriting. In Proceedings of the International Symposium on Functional and Logic Programming (FLOPS’18) (Lecture Notes in Computer Science), John P. Gallagher and Martin Sulzmann (Eds.), Vol. 10818. Springer, 132–148.Google ScholarGoogle ScholarCross RefCross Ref
  7. Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. The MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Dimitri P. Bertsekas and Steven E. Shreve. 2007. Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific.Google ScholarGoogle Scholar
  9. Rastislav Bodík and Rupak Majumdar (Eds.). 2016. In Proceedings of the ACM SIGPLAN Symposium on Principles of Programming Languages. ACM.Google ScholarGoogle Scholar
  10. Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic program analysis with martingales. In Proceedings of the International Conference on Computer Aided Verification (LNCS), Natasha Sharygina and Helmut Veith (Eds.), Vol. 8044. Springer, 511–526.Google ScholarGoogle ScholarCross RefCross Ref
  11. Aleksandar Chakarov, Yuen-Lam Voronin, and Sriram Sankaranarayanan. 2016. Deductive proofs of almost sure persistence and recurrence properties. In Proceedings of the 22nd International Conference on Tools and Algorithms for the Construction and Analysis of Systems, (TACAS’16) (LNCS), Marsha Chechik and Jean-François Raskin (Eds.), Vol. 9636. Springer, 260–279.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Krishnendu Chatterjee and Hongfei Fu. 2017. Termination of nondeterministic recursive probabilistic programs. CoRR abs/1701.02944 (2017).Google ScholarGoogle Scholar
  13. Krishnendu Chatterjee, Hongfei Fu, and Amir Kafshdar Goharshady. 2016. Termination analysis of probabilistic programs through Positivstellensatz’s. In Proceedings of the International Conference on Computer Aided Verification (LNCS), Swarat Chaudhuri and Azadeh Farzan (Eds.), Vol. 9779. Springer, 3–22.Google ScholarGoogle ScholarCross RefCross Ref
  14. Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In Proceedings of the ACM SIGPLAN Symposium on Principles of Programming Languages.327–342.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Krishnendu Chatterjee, Petr Novotný, and Dorde Zikelic. 2017. Stochastic invariants for probabilistic termination. In Proceedings of the ACM SIGPLAN Symposium on Principles of Programming Languages (POPL’17), Giuseppe Castagna and Andrew D. Gordon (Eds.). ACM, 145–160.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Radhia Cousot and Patrick Cousot. 1979. Constructive versions of Tarski’s fixed point theorems. Pacific J. Math. 82, 1 (1979), 43–57.Google ScholarGoogle ScholarCross RefCross Ref
  17. Luis María Ferrer Fioriti and Holger Hermanns. 2015. Probabilistic termination: Soundness, completeness, and compositionality. In Proceedings of the 42nd SIGPLAN-SIGACT Symposium on Principles of Programming Languages, (POPL’15), Sriram K. Rajamani and David Walker (Eds.). ACM, 489–501. DOI:https://doi.org/10.1145/2676726.2677001Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Makhorin. 2008. GLPK–GNU Linear Programming Kit. http://www.gnu.org/software/glpk/.Google ScholarGoogle Scholar
  19. Andrew D. Gordon, Thomas A. Henzinger, Aditya V. Nori, and Sriram K. Rajamani. 2014. Probabilistic programming. In Proceedings of the Conference on the Future of Software Engineering, (FOSE’14), James D. Herbsleb and Matthew B. Dwyer (Eds.). ACM, 167–181.Google ScholarGoogle Scholar
  20. Ichiro Hasuo, Shunsuke Shimizu, and Corina Cîrstea. 2016. Lattice-theoretic progress measures and coalgebraic model checking. In Proceedings of the ACM SIGPLAN Symposium on Principles of Programming Languages.. 718–732.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Roger A. Horn and Charles R. Johnson. 2012. Matrix Analysis (2nd ed.). Cambridge University Press, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Marcin Jurdzinski. 2000. Small progress measures for solving parity games. In Proceedings of the International Symposium on Theoretical Aspects of Computer Science (STACS’00) (LNCS), Horst Reichel and Sophie Tison (Eds.), Vol. 1770. Springer, 290–301.Google ScholarGoogle ScholarCross RefCross Ref
  23. Benjamin Lucien Kaminski, Joost-Pieter Katoen, Christoph Matheja, and Federico Olmedo. 2016. Weakest precondition reasoning for expected run-times of probabilistic programs. In Proceedings of the 25th European Symposium on Programming Languages and Systems, (ESOP’16), Held as Part of the European Joint Conferences on Theory and Practice of Software, (ETAPS’16) (Lecture Notes in Computer Science), Peter Thiemann (Ed.), Vol. 9632. Springer, 364–389.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Joost-Pieter Katoen, Annabelle McIver, Larissa Meinicke, and Carroll C. Morgan. 2010. Linear-invariant generation for probabilistic programs: Automated support for proof-based methods. In Proceedings of the 17th International Symposium on Static Analysis (SAS’10). 390–406.Google ScholarGoogle Scholar
  25. Dexter Kozen. 1985. A probabilistic PDL. J. Comput. Syst. Sci. 30, 2 (1985), 162–178. DOI:https://doi.org/10.1016/0022-0000(85)90012-1Google ScholarGoogle Scholar
  26. Satoshi Kura, Natsuki Urabe, and Ichiro Hasuo. 2019. Tail probabilities for randomized program runtimes via martingales for higher moments. In Proceedings of the 25th International Conference, on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’19), Held as Part of the European Joint Conferences on Theory and Practice of Software, (ETAPS’19) (Lecture Notes in Computer Science), Tomás Vojnar and Lijun Zhang (Eds.), Vol. 11428. Springer, 135–153. DOI:https://doi.org/10.1007/978-3-030-17465-1_8Google ScholarGoogle Scholar
  27. Annabelle McIver and Carroll Morgan. 2004. Abstraction, Refinement and Proof for Probabilistic Systems (Monographs in Computer Science). SpringerVerlag.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Annabelle McIver and Carroll Morgan. 2004. Developing and reasoning about probabilistic programs in pGCL. In Proceedings of the 1st Pernambuco Summer School on Software Engineering: Refinement Techniques in Software Engineering (PSSE’04) (LNCS), Ana Cavalcanti, Augusto Sampaio, and Jim Woodcock (Eds.), Vol. 3167. Springer, 123–155.Google ScholarGoogle Scholar
  29. Annabelle McIver, Carroll Morgan, Benjamin Lucien Kaminski, and Joost-Pieter Katoen. 2018. A new proof rule for almost-sure termination. Proc. ACM Prog. Lang. 2, POPL (2018), 33:1–33:28. DOI:https://doi.org/10.1145/3158121Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Frederick Mosteller. 2012. Fifty Challenging Problems in Probability with Solutions. Dover Publications.Google ScholarGoogle Scholar
  31. Arkadi Nemirovski and Cornelis Roos. 2008. Motzkin transposition theorem. Encyclopedia of Optimization (2nd edition), C. A. Floudas and P. M. Pardalos (Eds.). Springer, 235–2348. https://doi.org/10.1007/978-0-387-74759-0_405.Google ScholarGoogle Scholar
  32. Van Chan Ngo, Quentin Carbonneaux, and Jan Hoffmann. 2018. Bounded expectations: Resource analysis for probabilistic programs. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, (PLDI’18), Jeffrey S. Foster and Dan Grossman (Eds.). ACM, 496–512. DOI:https://doi.org/10.1145/3192366.3192394Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Stephen Prajna, Ali Jadbabaie, and George J. Pappas. 2004. Stochastic safety verification using barrier certificates. In Proceedings of the 43rd IEEE Conference on Decision and Control. IEEE, Piscataway, NJ, 929–934.Google ScholarGoogle Scholar
  34. Konrad Schmüdgen. 1991. The K-moment problem for compact semi-algebraic sets. Math. Ann. 289, 1 (01 Mar. 1991), 203–206.Google ScholarGoogle Scholar
  35. Alexander Schrijver. 1998. Theory of Linear and Integer Programming. Wiley.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. K. C. Toh, M. J. Todd, and R. H. Tutuncu, SDPT3–a Matlab software package for semidefinite programming. Optimization Methods and Software 11 (1999), 545–581.Google ScholarGoogle Scholar
  37. Antonis Papachristodoulou, James Anderson, Giorgio Valmorbida, Stephen Prajna, Pete Seiler, and Pablo A. Parrilo. 2013. SOSTOOLS Version 3.00 Sum of Squares Optimization Toolbox for MATLAB. CoRR abs/1310.4716.Google ScholarGoogle Scholar
  38. Jacob Steinhardt and Russ Tedrake. 2012. Finite-time regional verification of stochastic non-linear systems. Int. J. Robot. Res. 31, 7 (2012), 901–923.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Toru Takisaka, Yuichiro Oyabu, Natsuki Urabe, and Ichiro Hasuo. 2018. Ranking and repulsing supermartingales for reachability in probabilistic programs. In Proceedings of the 16th International Symposium on Automated Technology for Verification and Analysis, (ATVA’18) (Lecture Notes in Computer Science), Shuvendu K. Lahiri and Chao Wang (Eds.), Vol. 11138. Springer, 476–493. DOI:https://doi.org/10.1007/978-3-030-01090-4_28Google ScholarGoogle ScholarCross RefCross Ref
  40. Alfred Tarski. 1951. A Decision Method for Elementary Algebra and Geometry. University of California Press, Berkeley.Google ScholarGoogle Scholar
  41. Natsuki Urabe, Masaki Hara, and Ichiro Hasuo. 2017. Categorical liveness checking by corecursive algebras. In Proceedings of the ACM-IEEE Symposium on Logic in Computer Science (LICS’17). IEEE Computer Society, 1–12.Google ScholarGoogle ScholarCross RefCross Ref
  42. Moshe Y. Vardi. 1995. An automata-theoretic approach to linear temporal logic. In Proceedings of the Banff Higher Order Workshop (Lecture Notes in Computer Science), Faron Moller and Graham M. Birtwistle (Eds.), Vol. 1043. Springer, 238–266.Google ScholarGoogle Scholar

Index Terms

  1. Ranking and Repulsing Supermartingales for Reachability in Randomized Programs

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Programming Languages and Systems
        ACM Transactions on Programming Languages and Systems  Volume 43, Issue 2
        June 2021
        197 pages
        ISSN:0164-0925
        EISSN:1558-4593
        DOI:10.1145/3470134
        Issue’s Table of Contents

        Copyright © 2021 Owner/Author

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 8 June 2021
        • Accepted: 1 February 2021
        • Revised: 1 January 2021
        • Received: 1 May 2019
        Published in toplas Volume 43, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!