Abstract
We enhance refinement types with mechanisms to reason about relational properties of probabilistic computations. Our mechanisms, which are inspired from probabilistic couplings, are applicable to a rich set of probabilistic properties, including expected sensitivity, which ensures that the distance between outputs of two probabilistic computations can be controlled from the distance between their inputs. We implement our mechanisms in the type system of Liquid Haskell and we use them to formally verify Haskell implementations of two classic machine learning algorithms: Temporal Difference (TD) reinforcement learning and stochastic gradient descent (SGD). We formalize a fragment of our system for discrete distributions and we prove soundness with respect to a set-theoretical semantics.
- Chike Abuah, David Darais, and Joseph P. Near. 2021. Solo: Enforcing Differential Privacy Without Fancy Types. In CoRR. arxiv:2105.01632
Google Scholar
- Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, Shin-ya Katsumata, and Tetsuya Sato. 2021. Higher-Order Probabilistic Adversarial Computations: Categorical Semantics and Program Logics. In ICFP. https://doi.org/10.1145/3473598
Google Scholar
Digital Library
- Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Pierre-Yves Strub. 2017. A Relational Logic for Higher-Order Programs. In ICFP. https://doi.org/10.1145/3110265
Google Scholar
Digital Library
- Alejandro Aguirre, Gilles Barthe, Justin Hsu, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. 2021. A Pre-Expectation Calculus for Probabilistic Sensitivity. In POPL. https://doi.org/10.1145/3434333
Google Scholar
Digital Library
- Martin Avanzini, Gilles Barthe, and Ugo Dal Lago. 2021. On Continuation-Passing Transformations and Expected Cost Analysis. In ICFP. https://doi.org/10.1145/3473592
Google Scholar
Digital Library
- Martin Avanzini, Georg Moser, and Michael Schaper. 2020. A Modular Cost Analysis for Probabilistic Programs. In OOPSLA. https://doi.org/10.1145/3428240
Google Scholar
Digital Library
- Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2018. Proving Expected Sensitivity of Probabilistic Programs. In POPL. https://doi.org/10.1145/3158145
Google Scholar
Digital Library
- Gilles Barthe, Cédric Fournet, Benjamin Grégoire, Pierre-Yves Strub, Nikhil Swamy, and Santiago Zanella Béguelin. 2014. Probabilistic Relational Verification for Cryptographic Implementations. In POPL. https://doi.org/10.1145/2535838.2535847
Google Scholar
Digital Library
- Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, and Pierre-Yves Strub. 2015. Higher-Order Approximate Relational Refinement Types for Mechanism Design and Differential Privacy. In POPL. https://doi.org/10.1145/2676726.2677000
Google Scholar
Digital Library
- Gilles Barthe, Benjamin Grégoire, and Santiago Zanella Béguelin. 2009. Formal Certification of Code-Based Cryptographic Proofs. In POPL. https://doi.org/10.1145/1480881.1480894
Google Scholar
Digital Library
- Gilles Barthe, Benjamin Grégoire, Sylvain Heraud, and Santiago Zanella Béguelin. 2011. Computer-Aided Security Proofs for the Working Cryptographer. In CRYPTO. https://doi.org/10.1007/978-3-642-22792-9_5
Google Scholar
Cross Ref
- Gilles Barthe and Justin Hsu. 2020. Probabilistic Couplings from Program Logics. Cambridge University Press. https://doi.org/10.1017/9781108770750.006
Google Scholar
Cross Ref
- Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic Relational Reasoning for Differential Privacy. In POPL. https://doi.org/10.1145/2103656.2103670
Google Scholar
Digital Library
- David A. Basin, Andreas Lochbihler, and S. Reza Sefidgar. 2017. CryptHOL: Game-based Proofs in Higher-order Logic. In Journal of Cryptology. https://doi.org/10.1007/s00145-019-09341-z
Google Scholar
Cross Ref
- Olivier Bousquet and André Elisseeff. 2002. Stability and Generalization. In Journal of Machine Learning Research. https://doi.org/10.1162/153244302760200704
Google Scholar
Digital Library
- Ezgi Çiçek, Weihao Qu, Gilles Barthe, Marco Gaboardi, and Deepak Garg. 2019. Bidirectional Type Checking for Relational Properties. In PLDI. https://doi.org/10.1145/3314221.3314603
Google Scholar
Digital Library
- Yuxin Deng. 2015. Semantics of Probabilistic Processes: An Operational Approach. Springer. https://doi.org/10.1007/978-3-662-45198-4
Google Scholar
Cross Ref
- Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, and Benjamin C. Pierce. 2013. Linear Dependent Types for Differential Privacy. In POPL. https://doi.org/10.1145/2429069.2429113
Google Scholar
Digital Library
- Niklas Grimm, Kenji Maillard, Cédric Fournet, Catalin Hritcu, Matteo Maffei, Jonathan Protzenko, Tahina Ramananandro, Aseem Rastogi, Nikhil Swamy, and Santiago Zanella Béguelin. 2018. A Monadic Framework for Relational Verification: Applied to Information Security, Program Equivalence, and Optimizations. In CPP. https://doi.org/10.1145/3167090
Google Scholar
Digital Library
- Jad Hamza, Nicolas Voirol, and Viktor Kunčak. 2019. System FR: Formalized Foundations for the Stainless Verifier. In OOPSLA. https://doi.org/10.1145/3360592
Google Scholar
Digital Library
- Martin A. T. Handley, Niki Vazou, and Graham Hutton. 2019. Liquidate Your Assets: Reasoning about Resource Usage in Liquid Haskell. In POPL. https://doi.org/10.1145/3371092
Google Scholar
Digital Library
- Moritz Hardt, Ben Recht, and Yoram Singer. 2016. Train Faster, Generalize Better: Stability of Stochastic Gradient Descent. In ICML. https://dl.acm.org/doi/10.5555/3045390.3045520
Google Scholar
- Johannes Hölzl. 2016. Formalising Semantics for Expected Running Time of Probabilistic Programs. In ITP. https://doi.org/10.1007/978-3-319-43144-4_30
Google Scholar
Cross Ref
- Zixin Huang, Zhenbang Wang, and Sasa Misailovic. 2018. PSense: Automatic Sensitivity Analysis for Probabilistic Programs. In ATVA. https://doi.org/10.1007/978-3-030-01090-4_23
Google Scholar
Cross Ref
- Joe Hurd. 2003. Verification of the Miller-Rabin probabilistic primality test. In The Journal of Logic and Algebraic Programming. https://doi.org/10.1016/S1567-8326(02)00065-6
Google Scholar
Cross Ref
- june wunder and Arthur Azevedo de Amorim and Patrick Baillot and Marco Gaboardi. 2022. Bunched Fuzz: Sensitivity for Vector Metrics. In CoRR. https://doi.org/10.48550/arXiv.2202.01901
Google Scholar
- Dexter Kozen. 1985. A Probabilistic PDL. In Journal of Computer and System Sciences. https://doi.org/10.1016/0022-0000(85)90012-1
Google Scholar
Cross Ref
- Nico Lehmann, Rose Kunkel, Jordan Brown, Jean Yang, Niki Vazou, Nadia Polikarpova, Deian Stefan, and Ranjit Jhala. 2021. STORM: Refinement Types for Secure Web Applications. In OSDI. https://www.usenix.org/conference/osdi21/presentation/lehmann
Google Scholar
- Torgny Lindvall. 2002. Lectures on the Coupling Method. https://doi.org/10.1137/1035121
Google Scholar
Cross Ref
- Elisabet Lobo-Vesga, Alejandro Russo, and Marco Gaboardi. 2021. A Programming Language for Data Privacy with Accuracy Estimations. In Transactions on Programming Languages and Systems. https://doi.org/10.1145/3452096
Google Scholar
Digital Library
- Kenji Maillard, Catalin Hritcu, Exequiel Rivas, and Antoine Van Muylder. 2020. The Next 700 Relational Program Logics. In POPL. https://doi.org/10.1145/3371072
Google Scholar
Digital Library
- Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. In TOPLAS. https://doi.org/10.1145/229542.229547
Google Scholar
Digital Library
- Aleksandar Nanevski, Anindya Banerjee, and Deepak Garg. 2011. Verification of Information Flow and Access Control Policies with Dependent Types. In S&P. https://doi.org/10.1109/SP.2011.12
Google Scholar
Digital Library
- Joseph P. Near, David Darais, Chike Abuah, Tim Stevens, Pranav Gaddamadugu, Lun Wang, Neel Somani, Mu Zhang, Nikhil Sharma, Alex Shan, and Dawn Song. 2019. Duet: An Expressive Higher-Order Language and Linear Type System for Statically Enforcing Differential Privacy. https://doi.org/10.1145/3360598
Google Scholar
Digital Library
- Van Chan Ngo, Quentin Carbonneaux, and Jan Hoffmann. 2018. Bounded Expectations: Resource Analysis for Probabilistic Programs. In PLDI. https://doi.org/10.1145/3192366.3192394
Google Scholar
Digital Library
- Adam Petcher and Greg Morrisett. 2015. The Foundational Cryptography Framework. In POST. https://doi.org/10.1007/978-3-662-46666-7_4
Google Scholar
Digital Library
- Norman Ramsey and Avi Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In POPL. https://doi.org/10.1145/503272.503288
Google Scholar
Digital Library
- Jason Reed and Benjamin C. Pierce. 2010. Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy. In ICFP. https://doi.org/10.1145/1863543.1863568
Google Scholar
Digital Library
- Adam Scibior, Zoubin Ghahramani, and Andrew D. Gordon. 2015. Practical Probabilistic Programming with Monads. In Haskell. https://doi.org/10.1145/2887747.2804317
Google Scholar
Digital Library
- Nikhil Swamy, Catalin Hritcu, Chantal Keller, Aseem Rastogi, Antoine Delignat-Lavaud, Simon Forest, Karthikeyan Bhargavan, Cédric Fournet, Pierre-Yves Strub, Markulf Kohlweiss, Jean-Karim Zinzindohoué, and Santiago Zanella-Béguelin. 2016. Dependent Types and Multi-Monadic Effects in F*. In POPL. https://doi.org/10.1145/2914770.2837655
Google Scholar
Digital Library
- Joseph Tassarotti and Robert Harper. 2018. Verified Tail Bounds for Randomized Programs. In ITP. https://doi.org/10.1007/978-3-319-94821-8_33
Google Scholar
Cross Ref
- Hermann Thorisson. 2000. Coupling, Stationarity, and Regeneration. Springer. https://notendur.hi.is/hermann/iid/csr/
Google Scholar
- Elizaveta Vasilenko and Niki Vazou. 2022. Safe Couplings: Coupled Refinement Types. Zenodo. https://doi.org/10.5281/zenodo.6710298
Google Scholar
Digital Library
- Niki Vazou, Joachim Breitner, Rose Kunkel, David Van Horn, and Graham Hutton. 2018. Theorem Proving for All: Equational Reasoning in Liquid Haskell (Functional Pearl). In Haskell. https://doi.org/10.1145/3242744.3242756
Google Scholar
Digital Library
- Niki Vazou, Eric L. Seidel, and Ranjit Jhala. 2014. LiquidHaskell: Experience with Refinement Types in the Real World. In Haskell. https://doi.org/10.1145/2775050.2633366
Google Scholar
Digital Library
- Niki Vazou, Eric L. Seidel, Ranjit Jhala, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2014. Refinement Types for Haskell. In ICFP. https://doi.org/10.1145/2692915.2628161
Google Scholar
Digital Library
- Niki Vazou, Anish Tondwalkar, Vikraman Choudhury, Ryan G. Scott, Ryan R. Newton, Philip Wadler, and Ranjit Jhala. 2017. Refinement Reflection: Complete Verification with SMT. In POPL. https://doi.org/10.1145/3158141
Google Scholar
Digital Library
- Cédric Villani. 2009. Optimal Transport, old and new. Springer. https://link.springer.com/book/10.1007/978-3-540-71050-9
Google Scholar
- Peixin Wang, Hongfei Fu, Krishnendu Chatterjee, Yuxin Deng, and Ming Xu. 2020. Proving Expected Sensitivity of Probabilistic Programs with Randomized Variable-Dependent Termination Time. In POPL. https://doi.org/10.1145/3371093
Google Scholar
Digital Library
- Daniel Winograd-Cort, Andreas Haeberlen, Aaron Roth, and Benjamin C. Pierce. 2017. A Framework for Adaptive Differential Privacy. In ICFP. https://doi.org/10.1145/3110254
Google Scholar
Digital Library
- Hengchu Zhang, Edo Roth, Andreas Haeberlen, Benjamin C. Pierce, and Aaron Roth. 2019. Fuzzi: A Three-Level Logic for Differential Privacy. In ICFP. https://doi.org/10.1145/3341697
Google Scholar
Digital Library
Index Terms
Safe couplings: coupled refinement types
Recommendations
Refinement types for Haskell
PLPV '14: Proceedings of the ACM SIGPLAN 2014 Workshop on Programming Languages meets Program VerificationWe present LiquidHaskell (http://goto.ucsd.edu/liquid), an automatic verifier for Haskell. LiquidHaskell uses Refinement types, a restricted form of dependent types where relationships between values are encoded by decorating types with logical ...
Gradual refinement types
POPL '17Refinement types are an effective language-based verification technique. However, as any expressive typing discipline, its strength is its weakness, imposing sometimes undesired rigidity. Guided by abstract interpretation, we extend the gradual typing ...
Gradual refinement types
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesRefinement types are an effective language-based verification technique. However, as any expressive typing discipline, its strength is its weakness, imposing sometimes undesired rigidity. Guided by abstract interpretation, we extend the gradual typing ...






Comments