skip to main content

Safe couplings: coupled refinement types

Published:31 August 2022Publication History
Skip Abstract Section

Abstract

We enhance refinement types with mechanisms to reason about relational properties of probabilistic computations. Our mechanisms, which are inspired from probabilistic couplings, are applicable to a rich set of probabilistic properties, including expected sensitivity, which ensures that the distance between outputs of two probabilistic computations can be controlled from the distance between their inputs. We implement our mechanisms in the type system of Liquid Haskell and we use them to formally verify Haskell implementations of two classic machine learning algorithms: Temporal Difference (TD) reinforcement learning and stochastic gradient descent (SGD). We formalize a fragment of our system for discrete distributions and we prove soundness with respect to a set-theoretical semantics.

References

  1. Chike Abuah, David Darais, and Joseph P. Near. 2021. Solo: Enforcing Differential Privacy Without Fancy Types. In CoRR. arxiv:2105.01632 Google ScholarGoogle Scholar
  2. Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, Shin-ya Katsumata, and Tetsuya Sato. 2021. Higher-Order Probabilistic Adversarial Computations: Categorical Semantics and Program Logics. In ICFP. https://doi.org/10.1145/3473598 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Pierre-Yves Strub. 2017. A Relational Logic for Higher-Order Programs. In ICFP. https://doi.org/10.1145/3110265 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Alejandro Aguirre, Gilles Barthe, Justin Hsu, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. 2021. A Pre-Expectation Calculus for Probabilistic Sensitivity. In POPL. https://doi.org/10.1145/3434333 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Martin Avanzini, Gilles Barthe, and Ugo Dal Lago. 2021. On Continuation-Passing Transformations and Expected Cost Analysis. In ICFP. https://doi.org/10.1145/3473592 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Martin Avanzini, Georg Moser, and Michael Schaper. 2020. A Modular Cost Analysis for Probabilistic Programs. In OOPSLA. https://doi.org/10.1145/3428240 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2018. Proving Expected Sensitivity of Probabilistic Programs. In POPL. https://doi.org/10.1145/3158145 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gilles Barthe, Cédric Fournet, Benjamin Grégoire, Pierre-Yves Strub, Nikhil Swamy, and Santiago Zanella Béguelin. 2014. Probabilistic Relational Verification for Cryptographic Implementations. In POPL. https://doi.org/10.1145/2535838.2535847 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, and Pierre-Yves Strub. 2015. Higher-Order Approximate Relational Refinement Types for Mechanism Design and Differential Privacy. In POPL. https://doi.org/10.1145/2676726.2677000 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gilles Barthe, Benjamin Grégoire, and Santiago Zanella Béguelin. 2009. Formal Certification of Code-Based Cryptographic Proofs. In POPL. https://doi.org/10.1145/1480881.1480894 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Gilles Barthe, Benjamin Grégoire, Sylvain Heraud, and Santiago Zanella Béguelin. 2011. Computer-Aided Security Proofs for the Working Cryptographer. In CRYPTO. https://doi.org/10.1007/978-3-642-22792-9_5 Google ScholarGoogle ScholarCross RefCross Ref
  12. Gilles Barthe and Justin Hsu. 2020. Probabilistic Couplings from Program Logics. Cambridge University Press. https://doi.org/10.1017/9781108770750.006 Google ScholarGoogle ScholarCross RefCross Ref
  13. Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic Relational Reasoning for Differential Privacy. In POPL. https://doi.org/10.1145/2103656.2103670 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. David A. Basin, Andreas Lochbihler, and S. Reza Sefidgar. 2017. CryptHOL: Game-based Proofs in Higher-order Logic. In Journal of Cryptology. https://doi.org/10.1007/s00145-019-09341-z Google ScholarGoogle ScholarCross RefCross Ref
  15. Olivier Bousquet and André Elisseeff. 2002. Stability and Generalization. In Journal of Machine Learning Research. https://doi.org/10.1162/153244302760200704 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Ezgi Çiçek, Weihao Qu, Gilles Barthe, Marco Gaboardi, and Deepak Garg. 2019. Bidirectional Type Checking for Relational Properties. In PLDI. https://doi.org/10.1145/3314221.3314603 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Yuxin Deng. 2015. Semantics of Probabilistic Processes: An Operational Approach. Springer. https://doi.org/10.1007/978-3-662-45198-4 Google ScholarGoogle ScholarCross RefCross Ref
  18. Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, and Benjamin C. Pierce. 2013. Linear Dependent Types for Differential Privacy. In POPL. https://doi.org/10.1145/2429069.2429113 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Niklas Grimm, Kenji Maillard, Cédric Fournet, Catalin Hritcu, Matteo Maffei, Jonathan Protzenko, Tahina Ramananandro, Aseem Rastogi, Nikhil Swamy, and Santiago Zanella Béguelin. 2018. A Monadic Framework for Relational Verification: Applied to Information Security, Program Equivalence, and Optimizations. In CPP. https://doi.org/10.1145/3167090 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Jad Hamza, Nicolas Voirol, and Viktor Kunčak. 2019. System FR: Formalized Foundations for the Stainless Verifier. In OOPSLA. https://doi.org/10.1145/3360592 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Martin A. T. Handley, Niki Vazou, and Graham Hutton. 2019. Liquidate Your Assets: Reasoning about Resource Usage in Liquid Haskell. In POPL. https://doi.org/10.1145/3371092 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Moritz Hardt, Ben Recht, and Yoram Singer. 2016. Train Faster, Generalize Better: Stability of Stochastic Gradient Descent. In ICML. https://dl.acm.org/doi/10.5555/3045390.3045520 Google ScholarGoogle Scholar
  23. Johannes Hölzl. 2016. Formalising Semantics for Expected Running Time of Probabilistic Programs. In ITP. https://doi.org/10.1007/978-3-319-43144-4_30 Google ScholarGoogle ScholarCross RefCross Ref
  24. Zixin Huang, Zhenbang Wang, and Sasa Misailovic. 2018. PSense: Automatic Sensitivity Analysis for Probabilistic Programs. In ATVA. https://doi.org/10.1007/978-3-030-01090-4_23 Google ScholarGoogle ScholarCross RefCross Ref
  25. Joe Hurd. 2003. Verification of the Miller-Rabin probabilistic primality test. In The Journal of Logic and Algebraic Programming. https://doi.org/10.1016/S1567-8326(02)00065-6 Google ScholarGoogle ScholarCross RefCross Ref
  26. june wunder and Arthur Azevedo de Amorim and Patrick Baillot and Marco Gaboardi. 2022. Bunched Fuzz: Sensitivity for Vector Metrics. In CoRR. https://doi.org/10.48550/arXiv.2202.01901 Google ScholarGoogle Scholar
  27. Dexter Kozen. 1985. A Probabilistic PDL. In Journal of Computer and System Sciences. https://doi.org/10.1016/0022-0000(85)90012-1 Google ScholarGoogle ScholarCross RefCross Ref
  28. Nico Lehmann, Rose Kunkel, Jordan Brown, Jean Yang, Niki Vazou, Nadia Polikarpova, Deian Stefan, and Ranjit Jhala. 2021. STORM: Refinement Types for Secure Web Applications. In OSDI. https://www.usenix.org/conference/osdi21/presentation/lehmann Google ScholarGoogle Scholar
  29. Torgny Lindvall. 2002. Lectures on the Coupling Method. https://doi.org/10.1137/1035121 Google ScholarGoogle ScholarCross RefCross Ref
  30. Elisabet Lobo-Vesga, Alejandro Russo, and Marco Gaboardi. 2021. A Programming Language for Data Privacy with Accuracy Estimations. In Transactions on Programming Languages and Systems. https://doi.org/10.1145/3452096 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Kenji Maillard, Catalin Hritcu, Exequiel Rivas, and Antoine Van Muylder. 2020. The Next 700 Relational Program Logics. In POPL. https://doi.org/10.1145/3371072 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. In TOPLAS. https://doi.org/10.1145/229542.229547 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Aleksandar Nanevski, Anindya Banerjee, and Deepak Garg. 2011. Verification of Information Flow and Access Control Policies with Dependent Types. In S&P. https://doi.org/10.1109/SP.2011.12 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Joseph P. Near, David Darais, Chike Abuah, Tim Stevens, Pranav Gaddamadugu, Lun Wang, Neel Somani, Mu Zhang, Nikhil Sharma, Alex Shan, and Dawn Song. 2019. Duet: An Expressive Higher-Order Language and Linear Type System for Statically Enforcing Differential Privacy. https://doi.org/10.1145/3360598 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Van Chan Ngo, Quentin Carbonneaux, and Jan Hoffmann. 2018. Bounded Expectations: Resource Analysis for Probabilistic Programs. In PLDI. https://doi.org/10.1145/3192366.3192394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Adam Petcher and Greg Morrisett. 2015. The Foundational Cryptography Framework. In POST. https://doi.org/10.1007/978-3-662-46666-7_4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Norman Ramsey and Avi Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In POPL. https://doi.org/10.1145/503272.503288 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jason Reed and Benjamin C. Pierce. 2010. Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy. In ICFP. https://doi.org/10.1145/1863543.1863568 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Adam Scibior, Zoubin Ghahramani, and Andrew D. Gordon. 2015. Practical Probabilistic Programming with Monads. In Haskell. https://doi.org/10.1145/2887747.2804317 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Nikhil Swamy, Catalin Hritcu, Chantal Keller, Aseem Rastogi, Antoine Delignat-Lavaud, Simon Forest, Karthikeyan Bhargavan, Cédric Fournet, Pierre-Yves Strub, Markulf Kohlweiss, Jean-Karim Zinzindohoué, and Santiago Zanella-Béguelin. 2016. Dependent Types and Multi-Monadic Effects in F*. In POPL. https://doi.org/10.1145/2914770.2837655 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Joseph Tassarotti and Robert Harper. 2018. Verified Tail Bounds for Randomized Programs. In ITP. https://doi.org/10.1007/978-3-319-94821-8_33 Google ScholarGoogle ScholarCross RefCross Ref
  42. Hermann Thorisson. 2000. Coupling, Stationarity, and Regeneration. Springer. https://notendur.hi.is/hermann/iid/csr/ Google ScholarGoogle Scholar
  43. Elizaveta Vasilenko and Niki Vazou. 2022. Safe Couplings: Coupled Refinement Types. Zenodo. https://doi.org/10.5281/zenodo.6710298 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Niki Vazou, Joachim Breitner, Rose Kunkel, David Van Horn, and Graham Hutton. 2018. Theorem Proving for All: Equational Reasoning in Liquid Haskell (Functional Pearl). In Haskell. https://doi.org/10.1145/3242744.3242756 Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Niki Vazou, Eric L. Seidel, and Ranjit Jhala. 2014. LiquidHaskell: Experience with Refinement Types in the Real World. In Haskell. https://doi.org/10.1145/2775050.2633366 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Niki Vazou, Eric L. Seidel, Ranjit Jhala, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2014. Refinement Types for Haskell. In ICFP. https://doi.org/10.1145/2692915.2628161 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Niki Vazou, Anish Tondwalkar, Vikraman Choudhury, Ryan G. Scott, Ryan R. Newton, Philip Wadler, and Ranjit Jhala. 2017. Refinement Reflection: Complete Verification with SMT. In POPL. https://doi.org/10.1145/3158141 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Cédric Villani. 2009. Optimal Transport, old and new. Springer. https://link.springer.com/book/10.1007/978-3-540-71050-9 Google ScholarGoogle Scholar
  49. Peixin Wang, Hongfei Fu, Krishnendu Chatterjee, Yuxin Deng, and Ming Xu. 2020. Proving Expected Sensitivity of Probabilistic Programs with Randomized Variable-Dependent Termination Time. In POPL. https://doi.org/10.1145/3371093 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Daniel Winograd-Cort, Andreas Haeberlen, Aaron Roth, and Benjamin C. Pierce. 2017. A Framework for Adaptive Differential Privacy. In ICFP. https://doi.org/10.1145/3110254 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Hengchu Zhang, Edo Roth, Andreas Haeberlen, Benjamin C. Pierce, and Aaron Roth. 2019. Fuzzi: A Three-Level Logic for Differential Privacy. In ICFP. https://doi.org/10.1145/3341697 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Safe couplings: coupled refinement types

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!