Abstract
We define a continuation-passing style (CPS) translation for a typed λ-calculus with probabilistic choice, unbounded recursion, and a tick operator — for modeling cost. The target language is a (non-probabilistic) λ-calculus, enriched with a type of extended positive reals and a fixpoint operator. We then show that applying the CPS transform of an expression M to the continuation λ v. 0 yields the expected cost of M. We also introduce a formal system for higher-order logic, called EHOL, prove it sound, and show it can derive tight upper bounds on the expected cost of classic examples, including Coupon Collector and Random Walk. Moreover, we relate our translation to Kaminski et al.’s ert-calculus, showing that the latter can be recovered by applying our CPS translation to (a generalization of) the classic embedding of imperative programs into λ-calculus. Finally, we prove that the CPS transform of an expression can also be used to compute pre-expectations and to reason about almost sure termination.
Supplemental Material
- S. Agrawal, K. Chatterjee, and P. Novotný. 2018. Lexicographic Ranking Supermartingales: An Efficient Approach to Termination of Probabilistic Programs. PACMPL, 2, POPL (2018), 34:1–34:32. https://doi.org/10.1145/3158122 Google Scholar
Digital Library
- A. Aguirre, G. Barthe, M. Gaboardi, D. Garg, and P.-Y. Strub. 2017. A Relational Logic for Higher-order Programs. PACMPL, 1, ICFP (2017), 21:1–21:29. https://doi.org/10.1145/3110265 Google Scholar
Digital Library
- Elvira Albert, Puri Arenas, Samir Genaim, German Puebla, and Damiano Zanardini. 2012. Cost analysis of object-oriented bytecode programs. Theor. Comput. Sci., 413, 1 (2012), 142–159. https://doi.org/10.1016/j.tcs.2011.07.009 Google Scholar
Digital Library
- Andrew W. Appel. 1991. Compiling with Continuations. Cambridge University Press. https://doi.org/10.1017/CBO9780511609619 Google Scholar
Cross Ref
- R. Atkey. 2011. Amortised Resource Analysis with Separation Logic. Logical Methods in Computer Science, 7, 2 (2011), https://doi.org/10.2168/LMCS-7(2:17)2011 Google Scholar
Cross Ref
- M. Avanzini, U. Dal Lago, and A. Ghyselen. 2019. Type-Based Complexity Analysis of Probabilistic Functional Programs. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785725 Google Scholar
Cross Ref
- M. Avanzini, U. Dal Lago, and G. Moser. 2015. Analysing the Complexity of Functional Programs: Higher-Order Meets First-Order. In Proc. of 20 ICFP. ACM, 152–164. https://doi.org/10.1145/2784731.2784753 Google Scholar
Digital Library
- M. Avanzini, U. Dal Lago, and A. Yamada. 2019. On Probabilistic Term Rewriting. SCP, 185 (2019), 102338.Google Scholar
- M. Avanzini, G. Moser, and M. Schaper. 2020. A Modular Cost Analysis for Probabilistic Programs. PACMPL, 4, OOPSLA (2020), 172:1–172:30. https://doi.org/10.1145/3428240 Google Scholar
Digital Library
- G. Barthe, B. Grégoire, and S. Z. Béguelin. 2009. Formal Certification of Code-based Cryptographic Proofs. In Proc. of 36 POPL. 90–101. https://doi.org/10.1145/1480881.1480894 Google Scholar
Digital Library
- K. Batz, B. L. Kaminski, J.-P. Katoen, and C Matheja. 2021. Relatively Complete Verification of Probabilistic Programs: An Expressive Language for Expectation-based Reasoning. PACMPL, 5, POPL (2021), 1–30. https://doi.org/10.1145/3434320 Google Scholar
Digital Library
- O. Bournez and F. Garnier. 2005. Proving Positive Almost-Sure Termination. In Proc. of 16 RTA (LNCS, Vol. 3467). Springer, 323–337. https://doi.org/10.1007/978-3-540-32033-3_24 Google Scholar
Digital Library
- T. Brázdil, S. Kiefer, A. Kucera, and I.H. Vareková. 2015. Runtime Analysis of Probabilistic Programs with Unbounded Recursion. J. Comput. Syst. Sci., 81, 1 (2015), 288–310. https://doi.org/10.1016/j.jcss.2014.06.005 Google Scholar
Digital Library
- A. Bucciarelli, A. Piperno, and I. Salvo. 2003. Intersection Types and lambda-Definability. MSCS, 13, 1 (2003), 15–53. https://doi.org/10.1017/S0960129502003833 Google Scholar
Digital Library
- O. Celiku and A. McIver. 2005. Compositional Specification and Analysis of Cost-Based Properties in Probabilistic Programs. In Proc. International Symposium of Formal Methods Europe (LNCS, Vol. 3582). Springer, 107–122. https://doi.org/10.1007/11526841_9 Google Scholar
Digital Library
- A. Chakarov and S. Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In Proc. of 25 CAV (LNCS, Vol. 8044). Springer, 511–526. https://doi.org/10.1007/978-3-642-39799-8_34 Google Scholar
Cross Ref
- A. Chakarov and S. Sankaranarayanan. 2014. Expectation Invariants for Probabilistic Program Loops as Fixed Points. In Proc. of 21 SAS. 85–100. https://doi.org/10.1007/978-3-319-10936-7_6 Google Scholar
Cross Ref
- K. Chatterjee, H. Fu, and A. K. Goharshady. 2016. Termination Analysis of Probabilistic Programs Through Positivstellensatz’s. In Proc. of 28 CAV (LNCS, Vol. 9779). Springer, 3–22. https://doi.org/10.1007/978-3-319-41528-4_1 Google Scholar
Cross Ref
- K. Chatterjee, H. Fu, and A. Murhekar. 2017. Automated Recurrence Analysis for Almost-Linear Expected-Runtime Bounds. In Proc. of 29 CAV (LNCS, Vol. 10426). Springer, 118–139. https://doi.org/10.1007/978-3-319-63387-9_6 Google Scholar
Cross Ref
- K. Chatterjee, P. Novotný, and D. Zikelic. 2017. Stochastic Invariants for Probabilistic Termination. In Proc. of 44 POPL. ACM, 145–160. https://doi.org/10.1145/3009837.3009873 Google Scholar
Digital Library
- P. Cousot and R. Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proc. of 4 POPL. ACM, 238–252. https://doi.org/10.1145/512950.512973 Google Scholar
Digital Library
- J. W. Cutler, D. R. Licata, and N. Danner. 2020. Denotational Recurrence Extraction for Amortized Analysis. PACMPL, 4, ICFP (2020), 97:1–97:29. https://doi.org/10.1145/3408979 Google Scholar
Digital Library
- U. Dal Lago, C. Faggian, and S. Ronchi Della Rocca. 2021. Intersection Types and (Positive) Almost-sure Termination. PACMPL, 5, POPL (2021), 1–32. https://doi.org/10.1145/3434313 Google Scholar
Digital Library
- Ugo Dal Lago and Marco Gaboardi. 2011. Linear Dependent Types and Relative Completeness. Log. Methods Comput. Sci., 8, 4 (2011).Google Scholar
- U. Dal Lago and C. Grellois. 2017. Probabilistic Termination by Monadic Affine Sized Typing. In Proc. of 26 ESOP (LNCS). Springer, 393–419. https://doi.org/10.1145/3293605 Google Scholar
Digital Library
- U. Dal Lago and M. Zorzi. 2012. Probabilistic Operational Semantics for the Lambda Calculus. RAIRO - TIA, 46, 3 (2012), 413–450. https://doi.org/10.1051/ita/2012012 Google Scholar
Cross Ref
- N. A. Danielsson. 2008. Lightweight Semiformal Time Complexity Analysis for Purely Functional Data Structures. In Proc. of 35 POPL. ACM, 133–144. https://doi.org/10.1145/1328438.1328457 Google Scholar
Digital Library
- N. Danner, D. R. Licata, and Ramyaa. 2015. Denotational Cost Semantics for Functional Languages with Inductive Types. In Proc. of 20 ICFP. ACM, 140–151. https://doi.org/10.1145/2784731.2784749 Google Scholar
Digital Library
- Oliver Danvy and Andrzex Filinski. 1992. Representing Control: a Study of the CPS Transformation. Mathematical Structures in Computer Science, 2, 4 (1992), 361–391. https://doi.org/10.1017/S0960129500001535 Google Scholar
Cross Ref
- O. Danvy and L. R. Nielsen. 2003. A First-Order One-pass CPS Transformation. TCS, 308, 1-3 (2003), 239–257. https://doi.org/10.1016/S0304-3975(02)00733-8 Google Scholar
Digital Library
- K. De Leeuw, E. Moore, C. Shannon, and N. Shapiro. 1956. Computability by Probabilistic Machines. Automata Studies, 34 (1956), 183–198.Google Scholar
Cross Ref
- T. Ehrhard, M. Pagani, and C. Tasson. 2014. Probabilistic Coherence Spaces are Fully Abstract for Probabilistic PCF. In POPL. ACM. https://doi.org/10.1145/2535838.2535865 Google Scholar
Digital Library
- T. Ehrhard, M. Pagani, and C. Tasson. 2018. Full Abstraction for Probabilistic PCF. J. ACM, 65, 4 (2018), 23:1–23:44. https://doi.org/10.1145/3164540 Google Scholar
Digital Library
- J. Esparza, A. Kucera, and R. Mayr. 2005. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In Proc. of 20 LICS. 117–126. https://doi.org/10.1109/LICS.2005.39 Google Scholar
Digital Library
- Andrzej Filinski. 1994. Representing Monads. In Conference Record of POPL’94: 21st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Portland, Oregon, USA, January 17-21, 1994. ACM Press, 446–457.Google Scholar
Digital Library
- T. Gehr, S. Misailovic, and M. Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In Proc. of 28 CAV. 62–83. https://doi.org/10.1007/978-3-319-41528-4_4 Google Scholar
Cross Ref
- S. Goldwasser and S. Micali. 1984. Probabilistic Encryption. JCSS, 28, 2 (1984), 270–299. https://doi.org/10.1016/0022-0000(84)90070-9 Google Scholar
Cross Ref
- N. D. Goodman, V. K. Mansinghka, D. M. Roy, K. Bonawitz, and J. B. Tenenbaum. 2008. Church: A Language for Generative Models. In UAI. 220–229.Google Scholar
- J. Goubault-Larrecq. 2019. A Probabilistic and Non-Deterministic Call-by-Push-Value Language. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785809 Google Scholar
Cross Ref
- C. Heunen, O. Kammar, S. Staton, and H. Yang. 2017. A Convenient Category for Higher-order Probability Theory. In Proc. of 32 LICS. IEEE Computer Society, 1–12. https://doi.org/10.1109/LICS.2017.8005137 Google Scholar
Cross Ref
- J. Hoffmann, K. Aehlig, and M. Hofmann. 2012. Resource Aware ML. In Proc. of 24 CAV (LNCS, Vol. 7358). Springer, Heidelberg, DE. 781–786. https://doi.org/10.1007/978-3-642-31424-7_64 Google Scholar
Digital Library
- J. T. Gill III. 1974. Computational Complexity of Probabilistic Turing Machines. In Proceedings of STOC 1974. ACM, 91–95. https://doi.org/10.1145/800119.803889 Google Scholar
Digital Library
- S. Jost, K. Hammond, H.-W. Loidl, and M. Hofmann. 2010. Static Determination of Quantitative Resource Usage for Higher-order Programs. In Proc. of 37 POPL. ACM, 223–236. https://doi.org/10.1145/1706299.1706327 Google Scholar
Digital Library
- A. Jung and R. Tix. 1998. The Troublesome Probabilistic Powerdomain. ENTCS, 13 (1998), 70–91. https://doi.org/10.1016/S1571-0661(05)80216-6 Google Scholar
Cross Ref
- B. L. Kaminski and J.-P. Katoen. 2017. A Weakest Pre-expectation Semantics for Mixed-sign Expectations. In Proc. of 32 LICS. IEEE, 1–12. https://doi.org/10.1109/LICS.2017.8005153 Google Scholar
Cross Ref
- B. Lucien Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2016. Weakest Precondition Reasoning for Expected Run-Times of Probabilistic Programs. In Proc. of 25 ESOP (LNCS, Vol. 9632). Springer, 364–389. https://doi.org/10.1145/3208102 Google Scholar
Digital Library
- B. L. Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2018. Weakest Precondition Reasoning for Expected Runtimes of Randomized Algorithms. JACM, 65, 5 (2018), 30:1–30:68.Google Scholar
- B. L. Kaminski and J.-P. Katoen. 2015. On the Hardness of Almost-Sure Termination. In MFCS 2015, Part I. 307–318. https://doi.org/10.1007/978-3-662-48057-1_24 Google Scholar
Cross Ref
- J.-P. Katoen. 2016. The Probabilistic Model Checking Landscape. In Proc. of 31 LICS. 31–45. https://doi.org/10.1145/2933575.2934574 Google Scholar
Digital Library
- J.-P. Katoen, A. McIver, L. Meinicke, and C.C. Morgan. 2010. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In Proc. of 17 SAS. 390–406. https://doi.org/10.1007/978-3-642-15769-1_24 Google Scholar
Cross Ref
- D. Kozen. 1981. Semantics of Probabilistic Programs. J. Comput. Syst. Sci., 22, 3 (1981), 328–350.Google Scholar
Cross Ref
- D. Kozen. 1985. A Probabilistic PDL. JCSC, 30, 2 (1985), 162 – 178. https://doi.org/10.1016/0022-0000(85)90012-1 Google Scholar
Cross Ref
- A. McIver, C. Morgan, B. L. Kaminski, and J-P Katoen. 2018. A New Proof Rule for Almost-sure Termination. PACMPL, 2, POPL (2018), 33:1–33:28. https://doi.org/10.1145/3158121 Google Scholar
Digital Library
- D. Monniaux. 2001. An Abstract Analysis of the Probabilistic Termination of Programs. In Proc. of 8 SAS (LNCS, Vol. 2126). Springer, 111–126. https://doi.org/10.1007/3-540-47764-0_7 Google Scholar
Cross Ref
- Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press.Google Scholar
Digital Library
- P. Narayanan, J. Carette, W. Romano, C. Shan, and R. Zinkov. 2016. Probabilistic Inference by Program Transformation in Hakaru (System Description). In Proc. of 13 FLOPS (LNCS, Vol. 9613). Springer, 62–79. https://doi.org/10.1007/978-3-319-29604-3_5 Google Scholar
Cross Ref
- N. C. Ngo, Q. Carbonneaux, and J. Hoffmann. 2018. Bounded Expectations: Resource Analysis for Probabilistic Programs. In Proc. of 39 PLDI. ACM, 496–512. https://doi.org/10.1145/3192366.3192394 Google Scholar
Digital Library
- Hanne Riis Nielson. 1987. A Hoare-Like Proof System for Analysing the Computation Time of Programs. Sci. Comput. Program., 9, 2 (1987), 107–136. https://doi.org/10.1016/0167-6423(87)90029-3 Google Scholar
Digital Library
- F. Olmedo, B. L. Kaminski, J.-P. Katoen, and C. Matheja. 2016. Reasoning About Recursive Probabilistic Programs. In Proc. of 31 LICS. 672–681. https://doi.org/10.1145/2933575.2935317 Google Scholar
Digital Library
- G.D. Plotkin. 1975. Call-by-name, Call-by-value and the λ -calculus. TCS, 1, 2 (1975), 125–159. https://doi.org/10.1016/0304-3975(75)90017-1 Google Scholar
Cross Ref
- M. O. Rabin. 1963. Probabilistic Automata. Information and Control, 6, 3 (1963), 230–245.Google Scholar
Cross Ref
- I. Radicek, G. Barthe, M. Gaboardi, D. Garg, and F. Zuleger. 2018. Monadic Refinements for Relational Cost Analysis. PACMPL, 2, POPL (2018), 36:1–36:32. https://doi.org/10.1145/3158124 Google Scholar
Digital Library
- N. Ramsey and A. Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In Proc. of 29 POPL. ACM, 154–165. https://doi.org/10.1145/503272.503288 Google Scholar
Digital Library
- M. Rosendahl. 1989. Automatic Complexity Analysis. In FPCA. 144–156. https://doi.org/10.1145/99370.99381 Google Scholar
Digital Library
- N. Saheb-Djahromi. 1978. Probabilistic LCF. In MFCS. 442–451. https://doi.org/10.1007/3-540-08921-7_92 Google Scholar
Cross Ref
- E. S. Santos. 1969. Probabilistic Turing Machines and Computability. Proc. of AMS, 22, 3 (1969), 704–710.Google Scholar
Cross Ref
- A. Ścibior, O. Kammar, M. Vákár, S. Staton, H. Yang, Y. Cai, K. Ostermann, S. K. Moss, C. Heunen, and Z. Ghahramani. 2018. Denotational Validation of Higher-order Bayesian Inference. PACMPL, 2, POPL (2018), 60:1–60:29. https://doi.org/10.1145/3158148 Google Scholar
Digital Library
- R. Sedgewick and P. Flajolet. 1996. An Introduction to the Analysis of Algorithms. Addison-Wesley-Longman. isbn:978-0-201-40009-0Google Scholar
- T. Takisaka, Y. Oyabu, N. Urabe, and I. Hasuo. 2018. Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs. In Proc. of 16 ATVA (LNCS, Vol. 11138). Springer, 476–493. https://doi.org/10.1007/978-3-030-01090-4_28 Google Scholar
Cross Ref
- J. Tassarotti and R. Harper. 2018. Verified Tail Bounds for Randomized Programs. In Proc. of 9 ITP (LNCS, Vol. 10895). Springer, 560–578. https://doi.org/10.1007/978-3-030-01090-4_28 Google Scholar
Cross Ref
- D. Tolpin, J.-W. van de Meent, H. Yang, and F. D. Wood. 2016. Design and Implementation of Probabilistic Programming Language Anglican. In Proc. of 28 IFL. ACM, 6:1–6:12. https://doi.org/10.1145/3064899.3064910 Google Scholar
Digital Library
- D. Wang, J. Hoffmann, and T. W. Reps. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. In Proc. of 39 PLDI. 513–528. https://doi.org/10.1145/3192366.3192408 Google Scholar
Digital Library
- D. Wang, D. M. Kahn, and J. Hoffmann. 2020. Raising Expectations: Automating Expected Cost Analysis with Types. PACMPL, 4, ICFP (2020), 110:1–110:31. https://doi.org/10.1145/3408992 Google Scholar
Digital Library
- P. Wang, H. Fu, A. K. Goharshady, K. Chatterjee, X. Qin, and W. Shi. 2019. Cost Analysis of Nondeterministic Probabilistic Programs. In Proc. of 40 PLDI. ACM, 204–220. https://doi.org/10.1145/3314221.3314581 Google Scholar
Digital Library
- B. Wegbreit. 1975. Mechanical Program Analysis. Comm. ACM, 18, 9 (1975), 528–539. https://doi.org/10.1145/361002.361016 Google Scholar
Digital Library
- G. Winskel. 1993. The Formal Semantics of Programming Languages. MIT Press.Google Scholar
Digital Library
Index Terms
On continuation-passing transformations and expected cost analysis
Recommendations
Central moment analysis for cost accumulators in probabilistic programs
PLDI 2021: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and ImplementationFor probabilistic programs, it is usually not possible to automatically derive exact information about their properties, such as the distribution of states at a given program point. Instead, one can attempt to derive approximations, such as upper bounds ...
Production Cost Analysis of Plastic Woven Sack Manufacturer
IMMS '20: Proceedings of the 3rd International Conference on Information Management and Management ScienceThe plastic woven sack industry in Thailand is increasingly growing and competitive due to a lot of operators and price competition. The accuracy of production cost is vital for decision making in management, effect to selling price and impact the ...
Differential cost analysis with simultaneous potentials and anti-potentials
PLDI 2022: Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and ImplementationWe present a novel approach to differential cost analysis that, given a program revision, attempts to statically bound the difference in resource usage, or cost, between the two program versions. Differential cost analysis is particularly interesting ...






Comments