skip to main content
research-article
Open Access

On continuation-passing transformations and expected cost analysis

Published:19 August 2021Publication History
Skip Abstract Section

Abstract

We define a continuation-passing style (CPS) translation for a typed λ-calculus with probabilistic choice, unbounded recursion, and a tick operator — for modeling cost. The target language is a (non-probabilistic) λ-calculus, enriched with a type of extended positive reals and a fixpoint operator. We then show that applying the CPS transform of an expression M to the continuation λ v. 0 yields the expected cost of M. We also introduce a formal system for higher-order logic, called EHOL, prove it sound, and show it can derive tight upper bounds on the expected cost of classic examples, including Coupon Collector and Random Walk. Moreover, we relate our translation to Kaminski et al.’s ert-calculus, showing that the latter can be recovered by applying our CPS translation to (a generalization of) the classic embedding of imperative programs into λ-calculus. Finally, we prove that the CPS transform of an expression can also be used to compute pre-expectations and to reason about almost sure termination.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

Presentation video of our ICFP'21 paper "On Continuation-Passing Transformations and Expected Cost Analysis". We define a continuation-passing style (CPS) translation for a typed λ-calculus with probabilistic choice, unbounded recursion, and a tick operator — for modeling cost. The target language is a (non-probabilistic) λ-calculus, enriched with a type of extended positive reals and a fixpoint operator. We then show that applying the CPS transform of an expression M to the continuation λ v. 0 yields the expected cost of M. We also introduce a formal system for higher-order logic, called EHOL, prove it sound, and show it can derive tight upper bounds on the expected cost of classic examples.

3473592.mp4

Presentation Videos

References

  1. S. Agrawal, K. Chatterjee, and P. Novotný. 2018. Lexicographic Ranking Supermartingales: An Efficient Approach to Termination of Probabilistic Programs. PACMPL, 2, POPL (2018), 34:1–34:32. https://doi.org/10.1145/3158122 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. A. Aguirre, G. Barthe, M. Gaboardi, D. Garg, and P.-Y. Strub. 2017. A Relational Logic for Higher-order Programs. PACMPL, 1, ICFP (2017), 21:1–21:29. https://doi.org/10.1145/3110265 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Elvira Albert, Puri Arenas, Samir Genaim, German Puebla, and Damiano Zanardini. 2012. Cost analysis of object-oriented bytecode programs. Theor. Comput. Sci., 413, 1 (2012), 142–159. https://doi.org/10.1016/j.tcs.2011.07.009 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Andrew W. Appel. 1991. Compiling with Continuations. Cambridge University Press. https://doi.org/10.1017/CBO9780511609619 Google ScholarGoogle ScholarCross RefCross Ref
  5. R. Atkey. 2011. Amortised Resource Analysis with Separation Logic. Logical Methods in Computer Science, 7, 2 (2011), https://doi.org/10.2168/LMCS-7(2:17)2011 Google ScholarGoogle ScholarCross RefCross Ref
  6. M. Avanzini, U. Dal Lago, and A. Ghyselen. 2019. Type-Based Complexity Analysis of Probabilistic Functional Programs. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785725 Google ScholarGoogle ScholarCross RefCross Ref
  7. M. Avanzini, U. Dal Lago, and G. Moser. 2015. Analysing the Complexity of Functional Programs: Higher-Order Meets First-Order. In Proc. of 20 ICFP. ACM, 152–164. https://doi.org/10.1145/2784731.2784753 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Avanzini, U. Dal Lago, and A. Yamada. 2019. On Probabilistic Term Rewriting. SCP, 185 (2019), 102338.Google ScholarGoogle Scholar
  9. M. Avanzini, G. Moser, and M. Schaper. 2020. A Modular Cost Analysis for Probabilistic Programs. PACMPL, 4, OOPSLA (2020), 172:1–172:30. https://doi.org/10.1145/3428240 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. G. Barthe, B. Grégoire, and S. Z. Béguelin. 2009. Formal Certification of Code-based Cryptographic Proofs. In Proc. of 36 POPL. 90–101. https://doi.org/10.1145/1480881.1480894 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. K. Batz, B. L. Kaminski, J.-P. Katoen, and C Matheja. 2021. Relatively Complete Verification of Probabilistic Programs: An Expressive Language for Expectation-based Reasoning. PACMPL, 5, POPL (2021), 1–30. https://doi.org/10.1145/3434320 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. O. Bournez and F. Garnier. 2005. Proving Positive Almost-Sure Termination. In Proc. of 16 RTA (LNCS, Vol. 3467). Springer, 323–337. https://doi.org/10.1007/978-3-540-32033-3_24 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. Brázdil, S. Kiefer, A. Kucera, and I.H. Vareková. 2015. Runtime Analysis of Probabilistic Programs with Unbounded Recursion. J. Comput. Syst. Sci., 81, 1 (2015), 288–310. https://doi.org/10.1016/j.jcss.2014.06.005 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. A. Bucciarelli, A. Piperno, and I. Salvo. 2003. Intersection Types and lambda-Definability. MSCS, 13, 1 (2003), 15–53. https://doi.org/10.1017/S0960129502003833 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. O. Celiku and A. McIver. 2005. Compositional Specification and Analysis of Cost-Based Properties in Probabilistic Programs. In Proc. International Symposium of Formal Methods Europe (LNCS, Vol. 3582). Springer, 107–122. https://doi.org/10.1007/11526841_9 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Chakarov and S. Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In Proc. of 25 CAV (LNCS, Vol. 8044). Springer, 511–526. https://doi.org/10.1007/978-3-642-39799-8_34 Google ScholarGoogle ScholarCross RefCross Ref
  17. A. Chakarov and S. Sankaranarayanan. 2014. Expectation Invariants for Probabilistic Program Loops as Fixed Points. In Proc. of 21 SAS. 85–100. https://doi.org/10.1007/978-3-319-10936-7_6 Google ScholarGoogle ScholarCross RefCross Ref
  18. K. Chatterjee, H. Fu, and A. K. Goharshady. 2016. Termination Analysis of Probabilistic Programs Through Positivstellensatz’s. In Proc. of 28 CAV (LNCS, Vol. 9779). Springer, 3–22. https://doi.org/10.1007/978-3-319-41528-4_1 Google ScholarGoogle ScholarCross RefCross Ref
  19. K. Chatterjee, H. Fu, and A. Murhekar. 2017. Automated Recurrence Analysis for Almost-Linear Expected-Runtime Bounds. In Proc. of 29 CAV (LNCS, Vol. 10426). Springer, 118–139. https://doi.org/10.1007/978-3-319-63387-9_6 Google ScholarGoogle ScholarCross RefCross Ref
  20. K. Chatterjee, P. Novotný, and D. Zikelic. 2017. Stochastic Invariants for Probabilistic Termination. In Proc. of 44 POPL. ACM, 145–160. https://doi.org/10.1145/3009837.3009873 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. P. Cousot and R. Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proc. of 4 POPL. ACM, 238–252. https://doi.org/10.1145/512950.512973 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. W. Cutler, D. R. Licata, and N. Danner. 2020. Denotational Recurrence Extraction for Amortized Analysis. PACMPL, 4, ICFP (2020), 97:1–97:29. https://doi.org/10.1145/3408979 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. U. Dal Lago, C. Faggian, and S. Ronchi Della Rocca. 2021. Intersection Types and (Positive) Almost-sure Termination. PACMPL, 5, POPL (2021), 1–32. https://doi.org/10.1145/3434313 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ugo Dal Lago and Marco Gaboardi. 2011. Linear Dependent Types and Relative Completeness. Log. Methods Comput. Sci., 8, 4 (2011).Google ScholarGoogle Scholar
  25. U. Dal Lago and C. Grellois. 2017. Probabilistic Termination by Monadic Affine Sized Typing. In Proc. of 26 ESOP (LNCS). Springer, 393–419. https://doi.org/10.1145/3293605 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. U. Dal Lago and M. Zorzi. 2012. Probabilistic Operational Semantics for the Lambda Calculus. RAIRO - TIA, 46, 3 (2012), 413–450. https://doi.org/10.1051/ita/2012012 Google ScholarGoogle ScholarCross RefCross Ref
  27. N. A. Danielsson. 2008. Lightweight Semiformal Time Complexity Analysis for Purely Functional Data Structures. In Proc. of 35 POPL. ACM, 133–144. https://doi.org/10.1145/1328438.1328457 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. N. Danner, D. R. Licata, and Ramyaa. 2015. Denotational Cost Semantics for Functional Languages with Inductive Types. In Proc. of 20 ICFP. ACM, 140–151. https://doi.org/10.1145/2784731.2784749 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Oliver Danvy and Andrzex Filinski. 1992. Representing Control: a Study of the CPS Transformation. Mathematical Structures in Computer Science, 2, 4 (1992), 361–391. https://doi.org/10.1017/S0960129500001535 Google ScholarGoogle ScholarCross RefCross Ref
  30. O. Danvy and L. R. Nielsen. 2003. A First-Order One-pass CPS Transformation. TCS, 308, 1-3 (2003), 239–257. https://doi.org/10.1016/S0304-3975(02)00733-8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. De Leeuw, E. Moore, C. Shannon, and N. Shapiro. 1956. Computability by Probabilistic Machines. Automata Studies, 34 (1956), 183–198.Google ScholarGoogle ScholarCross RefCross Ref
  32. T. Ehrhard, M. Pagani, and C. Tasson. 2014. Probabilistic Coherence Spaces are Fully Abstract for Probabilistic PCF. In POPL. ACM. https://doi.org/10.1145/2535838.2535865 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. T. Ehrhard, M. Pagani, and C. Tasson. 2018. Full Abstraction for Probabilistic PCF. J. ACM, 65, 4 (2018), 23:1–23:44. https://doi.org/10.1145/3164540 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Esparza, A. Kucera, and R. Mayr. 2005. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In Proc. of 20 LICS. 117–126. https://doi.org/10.1109/LICS.2005.39 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Andrzej Filinski. 1994. Representing Monads. In Conference Record of POPL’94: 21st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Portland, Oregon, USA, January 17-21, 1994. ACM Press, 446–457.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. T. Gehr, S. Misailovic, and M. Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In Proc. of 28 CAV. 62–83. https://doi.org/10.1007/978-3-319-41528-4_4 Google ScholarGoogle ScholarCross RefCross Ref
  37. S. Goldwasser and S. Micali. 1984. Probabilistic Encryption. JCSS, 28, 2 (1984), 270–299. https://doi.org/10.1016/0022-0000(84)90070-9 Google ScholarGoogle ScholarCross RefCross Ref
  38. N. D. Goodman, V. K. Mansinghka, D. M. Roy, K. Bonawitz, and J. B. Tenenbaum. 2008. Church: A Language for Generative Models. In UAI. 220–229.Google ScholarGoogle Scholar
  39. J. Goubault-Larrecq. 2019. A Probabilistic and Non-Deterministic Call-by-Push-Value Language. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785809 Google ScholarGoogle ScholarCross RefCross Ref
  40. C. Heunen, O. Kammar, S. Staton, and H. Yang. 2017. A Convenient Category for Higher-order Probability Theory. In Proc. of 32 LICS. IEEE Computer Society, 1–12. https://doi.org/10.1109/LICS.2017.8005137 Google ScholarGoogle ScholarCross RefCross Ref
  41. J. Hoffmann, K. Aehlig, and M. Hofmann. 2012. Resource Aware ML. In Proc. of 24 CAV (LNCS, Vol. 7358). Springer, Heidelberg, DE. 781–786. https://doi.org/10.1007/978-3-642-31424-7_64 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. J. T. Gill III. 1974. Computational Complexity of Probabilistic Turing Machines. In Proceedings of STOC 1974. ACM, 91–95. https://doi.org/10.1145/800119.803889 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. S. Jost, K. Hammond, H.-W. Loidl, and M. Hofmann. 2010. Static Determination of Quantitative Resource Usage for Higher-order Programs. In Proc. of 37 POPL. ACM, 223–236. https://doi.org/10.1145/1706299.1706327 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. A. Jung and R. Tix. 1998. The Troublesome Probabilistic Powerdomain. ENTCS, 13 (1998), 70–91. https://doi.org/10.1016/S1571-0661(05)80216-6 Google ScholarGoogle ScholarCross RefCross Ref
  45. B. L. Kaminski and J.-P. Katoen. 2017. A Weakest Pre-expectation Semantics for Mixed-sign Expectations. In Proc. of 32 LICS. IEEE, 1–12. https://doi.org/10.1109/LICS.2017.8005153 Google ScholarGoogle ScholarCross RefCross Ref
  46. B. Lucien Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2016. Weakest Precondition Reasoning for Expected Run-Times of Probabilistic Programs. In Proc. of 25 ESOP (LNCS, Vol. 9632). Springer, 364–389. https://doi.org/10.1145/3208102 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. B. L. Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2018. Weakest Precondition Reasoning for Expected Runtimes of Randomized Algorithms. JACM, 65, 5 (2018), 30:1–30:68.Google ScholarGoogle Scholar
  48. B. L. Kaminski and J.-P. Katoen. 2015. On the Hardness of Almost-Sure Termination. In MFCS 2015, Part I. 307–318. https://doi.org/10.1007/978-3-662-48057-1_24 Google ScholarGoogle ScholarCross RefCross Ref
  49. J.-P. Katoen. 2016. The Probabilistic Model Checking Landscape. In Proc. of 31 LICS. 31–45. https://doi.org/10.1145/2933575.2934574 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. J.-P. Katoen, A. McIver, L. Meinicke, and C.C. Morgan. 2010. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In Proc. of 17 SAS. 390–406. https://doi.org/10.1007/978-3-642-15769-1_24 Google ScholarGoogle ScholarCross RefCross Ref
  51. D. Kozen. 1981. Semantics of Probabilistic Programs. J. Comput. Syst. Sci., 22, 3 (1981), 328–350.Google ScholarGoogle ScholarCross RefCross Ref
  52. D. Kozen. 1985. A Probabilistic PDL. JCSC, 30, 2 (1985), 162 – 178. https://doi.org/10.1016/0022-0000(85)90012-1 Google ScholarGoogle ScholarCross RefCross Ref
  53. A. McIver, C. Morgan, B. L. Kaminski, and J-P Katoen. 2018. A New Proof Rule for Almost-sure Termination. PACMPL, 2, POPL (2018), 33:1–33:28. https://doi.org/10.1145/3158121 Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. D. Monniaux. 2001. An Abstract Analysis of the Probabilistic Termination of Programs. In Proc. of 8 SAS (LNCS, Vol. 2126). Springer, 111–126. https://doi.org/10.1007/3-540-47764-0_7 Google ScholarGoogle ScholarCross RefCross Ref
  55. Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. P. Narayanan, J. Carette, W. Romano, C. Shan, and R. Zinkov. 2016. Probabilistic Inference by Program Transformation in Hakaru (System Description). In Proc. of 13 FLOPS (LNCS, Vol. 9613). Springer, 62–79. https://doi.org/10.1007/978-3-319-29604-3_5 Google ScholarGoogle ScholarCross RefCross Ref
  57. N. C. Ngo, Q. Carbonneaux, and J. Hoffmann. 2018. Bounded Expectations: Resource Analysis for Probabilistic Programs. In Proc. of 39 PLDI. ACM, 496–512. https://doi.org/10.1145/3192366.3192394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Hanne Riis Nielson. 1987. A Hoare-Like Proof System for Analysing the Computation Time of Programs. Sci. Comput. Program., 9, 2 (1987), 107–136. https://doi.org/10.1016/0167-6423(87)90029-3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. F. Olmedo, B. L. Kaminski, J.-P. Katoen, and C. Matheja. 2016. Reasoning About Recursive Probabilistic Programs. In Proc. of 31 LICS. 672–681. https://doi.org/10.1145/2933575.2935317 Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. G.D. Plotkin. 1975. Call-by-name, Call-by-value and the λ -calculus. TCS, 1, 2 (1975), 125–159. https://doi.org/10.1016/0304-3975(75)90017-1 Google ScholarGoogle ScholarCross RefCross Ref
  61. M. O. Rabin. 1963. Probabilistic Automata. Information and Control, 6, 3 (1963), 230–245.Google ScholarGoogle ScholarCross RefCross Ref
  62. I. Radicek, G. Barthe, M. Gaboardi, D. Garg, and F. Zuleger. 2018. Monadic Refinements for Relational Cost Analysis. PACMPL, 2, POPL (2018), 36:1–36:32. https://doi.org/10.1145/3158124 Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. N. Ramsey and A. Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In Proc. of 29 POPL. ACM, 154–165. https://doi.org/10.1145/503272.503288 Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. M. Rosendahl. 1989. Automatic Complexity Analysis. In FPCA. 144–156. https://doi.org/10.1145/99370.99381 Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. N. Saheb-Djahromi. 1978. Probabilistic LCF. In MFCS. 442–451. https://doi.org/10.1007/3-540-08921-7_92 Google ScholarGoogle ScholarCross RefCross Ref
  66. E. S. Santos. 1969. Probabilistic Turing Machines and Computability. Proc. of AMS, 22, 3 (1969), 704–710.Google ScholarGoogle ScholarCross RefCross Ref
  67. A. Ścibior, O. Kammar, M. Vákár, S. Staton, H. Yang, Y. Cai, K. Ostermann, S. K. Moss, C. Heunen, and Z. Ghahramani. 2018. Denotational Validation of Higher-order Bayesian Inference. PACMPL, 2, POPL (2018), 60:1–60:29. https://doi.org/10.1145/3158148 Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. R. Sedgewick and P. Flajolet. 1996. An Introduction to the Analysis of Algorithms. Addison-Wesley-Longman. isbn:978-0-201-40009-0Google ScholarGoogle Scholar
  69. T. Takisaka, Y. Oyabu, N. Urabe, and I. Hasuo. 2018. Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs. In Proc. of 16 ATVA (LNCS, Vol. 11138). Springer, 476–493. https://doi.org/10.1007/978-3-030-01090-4_28 Google ScholarGoogle ScholarCross RefCross Ref
  70. J. Tassarotti and R. Harper. 2018. Verified Tail Bounds for Randomized Programs. In Proc. of 9 ITP (LNCS, Vol. 10895). Springer, 560–578. https://doi.org/10.1007/978-3-030-01090-4_28 Google ScholarGoogle ScholarCross RefCross Ref
  71. D. Tolpin, J.-W. van de Meent, H. Yang, and F. D. Wood. 2016. Design and Implementation of Probabilistic Programming Language Anglican. In Proc. of 28 IFL. ACM, 6:1–6:12. https://doi.org/10.1145/3064899.3064910 Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. D. Wang, J. Hoffmann, and T. W. Reps. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. In Proc. of 39 PLDI. 513–528. https://doi.org/10.1145/3192366.3192408 Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. D. Wang, D. M. Kahn, and J. Hoffmann. 2020. Raising Expectations: Automating Expected Cost Analysis with Types. PACMPL, 4, ICFP (2020), 110:1–110:31. https://doi.org/10.1145/3408992 Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. P. Wang, H. Fu, A. K. Goharshady, K. Chatterjee, X. Qin, and W. Shi. 2019. Cost Analysis of Nondeterministic Probabilistic Programs. In Proc. of 40 PLDI. ACM, 204–220. https://doi.org/10.1145/3314221.3314581 Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. B. Wegbreit. 1975. Mechanical Program Analysis. Comm. ACM, 18, 9 (1975), 528–539. https://doi.org/10.1145/361002.361016 Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. G. Winskel. 1993. The Formal Semantics of Programming Languages. MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. On continuation-passing transformations and expected cost analysis

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image Proceedings of the ACM on Programming Languages
            Proceedings of the ACM on Programming Languages  Volume 5, Issue ICFP
            August 2021
            1011 pages
            EISSN:2475-1421
            DOI:10.1145/3482883
            Issue’s Table of Contents

            Copyright © 2021 Owner/Author

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 19 August 2021
            Published in pacmpl Volume 5, Issue ICFP

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!