skip to main content
research-article
Open Access

Higher-order probabilistic adversarial computations: categorical semantics and program logics

Published:19 August 2021Publication History
Skip Abstract Section

Abstract

Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. These computations arise routinely in several domains, including security, privacy and machine learning.

In this paper, we develop program logics for reasoning about adversarial computations in a higher-order setting. Our logics are built on top of a simply typed λ-calculus extended with a graded monad for probabilities and state. The grading is used to model and restrict the memory footprint and the cost (in terms of oracle calls) of computations. Under this view, an adversary is a higher-order expression that expects as arguments the code of its oracles. We develop unary program logics for reasoning about error probabilities and expected values, and a relational logic for reasoning about coupling-based properties. All logics feature rules for adversarial computations, and yield guarantees that are valid for all adversaries that satisfy a fixed resource policy. We prove the soundness of the logics in the category of quasi-Borel spaces, using a general notion of graded predicate liftings, and we use logical relations over graded predicate liftings to establish the soundness of proof rules for adversaries. We illustrate the working of our logics with simple but illustrative examples.

Skip Supplemental Material Section

Supplemental Material

3473598.mp4

Presentation Videos

Auxiliary Presentation Video

This is a presentation of our paper at ICFP 21, titled Higher-Order Probabilistic Adversarial Computations: Categorical Semantics and Program Logics. Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. We develop program logics for reasoning about adversarial computations in a higher-order setting: unary program logics for reasoning about error probabilities and expected values, and a relational logic for reasoning about coupling-based properties. All logics feature rules for adversarial computations, and yield guarantees that are valid for all adversaries that satisfy a fixed resource policy. We prove the soundness of the logics in the category of quasi-Borel spaces, using a general notion of graded predicate liftings, and we use logical relations over graded predicate liftings to establish the soundness of proof rules for adversaries.

References

  1. Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Pierre-Yves Strub. 2017. A relational logic for higher-order programs. PACMPL, 1, ICFP (2017), 21:1–21:29. https://doi.org/10.1145/3110265 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Alejandro Aguirre, Gilles Barthe, Justin Hsu, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. 2021. A Pre-Expectation Calculus for Probabilistic Sensitivity. Proc. ACM Program. Lang., 5, POPL (2021), Article 52, Jan., 28 pages. https://doi.org/10.1145/3434333 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Alejandro Aguirre and Shin-ya Katsumata. 2020. Weakest Preconditions in Fibrations. In 36th Mathematical Foundations of Programming Semantics Conference, MFPS 2020 (Electronic Notes in Theoretical Computer Science, Vol. 352). 5–27. issn:1571-0661 https://doi.org/10.1016/j.entcs.2020.09.002 Google ScholarGoogle ScholarCross RefCross Ref
  4. Robert J Aumann. 1961. Borel structures for function spaces. Illinois Journal of Mathematics, 5, 4 (1961), 614–630. https://doi.org/10.1215/ijm/1255631584 Google ScholarGoogle ScholarCross RefCross Ref
  5. Manuel Barbosa, Gilles Barthe, Benjamin Grégoire, Adrien Koutsos, and Pierre-Yves Strub. 2021. Mechanized Proofs of Adversarial Complexity and Application to Universal Composability. Cryptology ePrint Archive, Report 2021/156. https://eprint.iacr.org/2021/156Google ScholarGoogle Scholar
  6. Gilles Barthe, Marion Daubignard, Bruce M. Kapron, and Yassine Lakhnech. 2010. Computational indistinguishability logic. In Proceedings of the 17th ACM Conference on Computer and Communications Security, CCS 2010, Chicago, Illinois, USA, October 4-8, 2010, Ehab Al-Shaer, Angelos D. Keromytis, and Vitaly Shmatikov (Eds.). ACM, 375–386. https://doi.org/10.1145/1866307.1866350 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2018. Proving expected sensitivity of probabilistic programs. Proc. ACM Program. Lang., 2, POPL (2018), 57:1–57:29. https://doi.org/10.1145/3158145 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016. Advanced Probabilistic Couplings for Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, Edgar R. Weippl, Stefan Katzenbeisser, Christopher Kruegel, Andrew C. Myers, and Shai Halevi (Eds.). ACM, 55–67. https://doi.org/10.1145/2976749.2978391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Gilles Barthe, Cédric Fournet, Benjamin Grégoire, Pierre-Yves Strub, Nikhil Swamy, and Santiago Zanella Béguelin. 2014. Probabilistic relational verification for cryptographic implementations. In POPL 2014, Suresh Jagannathan and Peter Sewell (Eds.). https://doi.org/10.1145/2578855.2535847 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gilles Barthe, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016. A Program Logic for Union Bounds. In 43rd International Colloquium on Automata, Languages, and Programming, ICALP 2016, July 11-15, 2016, Rome, Italy, Ioannis Chatzigiannakis, Michael Mitzenmacher, Yuval Rabani, and Davide Sangiorgi (Eds.) (LIPIcs, Vol. 55). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 107:1–107:15. https://doi.org/10.4230/LIPIcs.ICALP.2016.107 Google ScholarGoogle ScholarCross RefCross Ref
  11. Gilles Barthe, Benjamin Grégoire, and Santiago Zanella Béguelin. 2009. Formal certification of code-based cryptographic proofs. In POPL 2009, Savannah, GA, USA, January 21-23, 2009. https://doi.org/10.1145/1480881.1480894 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gilles Barthe, Boris Köpf, Laurent Mauborgne, and Martín Ochoa. 2014. Leakage Resilience against Concurrent Cache Attacks. In Proc. 3rd Conference on Principles of Security and Trust (POST ’14). Springer. https://doi.org/10.1007/978-3-642-54792-8_8 Google ScholarGoogle ScholarCross RefCross Ref
  13. Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic relational reasoning for differential privacy. In POPL 2012, Philadelphia, Pennsylvania, USA, January 22-28, 2012. https://doi.org/10.1145/2103656.2103670 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Nick Benton, Martin Hofmann, and Vivek Nigam. 2014. Abstract effects and proof-relevant logical relations. In The 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, San Diego, CA, USA, January 20-21, 2014, Suresh Jagannathan and Peter Sewell (Eds.). ACM, 619–632. https://doi.org/10.1145/2535838.2535869 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ales Bizjak and Lars Birkedal. 2015. Step-Indexed Logical Relations for Probability. In FoSSaCS 2015, London, UK, April 11-18, 2015. Proceedings. https://doi.org/10.1007/978-3-662-46678-0_18 Google ScholarGoogle ScholarCross RefCross Ref
  16. Burton H. Bloom. 1970. Space/Time Trade-offs in Hash Coding with Allowable Errors. Commun. ACM, 13, 7 (1970), 422–426. https://doi.org/10.1145/362686.362692 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Chris Brzuska, Antoine Delignat-Lavaud, Cédric Fournet, Konrad Kohbrok, and Markulf Kohlweiss. 2018. State Separation for Code-Based Game-Playing Proofs. In Advances in Cryptology - ASIACRYPT 2018 - 24th International Conference on the Theory and Application of Cryptology and Information Security, Brisbane, QLD, Australia, December 2-6, 2018, Proceedings, Part III, Thomas Peyrin and Steven D. Galbraith (Eds.) (Lecture Notes in Computer Science, Vol. 11274). Springer, 222–249. https://doi.org/10.1007/978-3-030-03332-3_9 Google ScholarGoogle ScholarCross RefCross Ref
  18. David Clayton, Christopher Patton, and Thomas Shrimpton. 2019. Probabilistic Data Structures in Adversarial Environments. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019, Lorenzo Cavallaro, Johannes Kinder, XiaoFeng Wang, and Jonathan Katz (Eds.). ACM, 1317–1334. https://doi.org/10.1145/3319535.3354235 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Raphaëlle Crubillé and Ugo Dal Lago. 2015. Metric Reasoning about λ -Terms: The Affine Case. In 30th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2015, Kyoto, Japan, July 6-10, 2015. IEEE Computer Society, 633–644. https://doi.org/10.1109/LICS.2015.64 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Vincent Danos and Thomas Ehrhard. 2011. Probabilistic coherence spaces as a model of higher-order probabilistic computation. Inf. Comput., 209, 6 (2011), 966–991. https://doi.org/10.1016/j.ic.2011.02.001 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Dominique Devriese, Lars Birkedal, and Frank Piessens. 2016. Reasoning about Object Capabilities with Logical Relations and Effect Parametricity. In IEEE European Symposium on Security and Privacy (EuroS&P). 147–162. https://doi.org/10.1109/EuroSP.2016.22 Google ScholarGoogle ScholarCross RefCross Ref
  22. Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science, 9, 3–4 (2014), 211–407. http://dx.doi.org/10.1561/0400000042 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Deepak Garg, Jason Franklin, Dilsun Kirli Kaynar, and Anupam Datta. 2010. Compositional System Security with Interface-Confined Adversaries. Electr. Notes Theor. Comput. Sci., 265 (2010), 49–71. https://doi.org/10.1016/j.entcs.2010.08.005 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Thomas Gerbet, Amrit Kumar, and Cédric Lauradoux. 2015. The Power of Evil Choices in Bloom Filters. In 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2015, Rio de Janeiro, Brazil, June 22-25, 2015. IEEE Computer Society, 101–112. https://doi.org/10.1109/DSN.2015.21 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. 2017. A convenient category for higher-order probability theory. In 32nd Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2017, Reykjavik, Iceland, June 20-23, 2017. IEEE Computer Society, 1–12. https://doi.org/10.1109/LICS.2017.8005137 Google ScholarGoogle Scholar
  26. R. Impagliazzo and S. Rudich. 1989. Limits on the provable consequences of one-way permutations. In 21st Annual ACM Symposium on Theory of Computing, 1989. ACM, New York. 44–61. https://doi.org/10.1145/73007.73012 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kenneth E. Iverson. 1962. A Programming Language. John Wiley & Sons, Inc., USA. isbn:0471430145Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Limin Jia, Shayak Sen, Deepak Garg, and Anupam Datta. 2015. A Logic of Programs with Interface-Confined Code. In IEEE 28th Computer Security Foundations Symposium (CSF). 512–525. https://doi.org/10.1109/CSF.2015.38 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ralf Jung, David Swasey, Filip Sieczkowski, Kasper Svendsen, Aaron Turon, Lars Birkedal, and Derek Dreyer. 2015. Iris: Monoids and Invariants as an Orthogonal Basis for Concurrent Reasoning. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, Sriram K. Rajamani and David Walker (Eds.). ACM, 637–650. https://doi.org/10.1145/2676726.2676980 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Benjamin Lucien Kaminski, Joost-Pieter Katoen, Christoph Matheja, and Federico Olmedo. 2016. Weakest Precondition Reasoning for Expected Run-Times of Probabilistic Programs. 9632, 364–389. https://doi.org/10.1007/978-3-662-49498-1_15 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Shin-ya Katsumata. 2014. Parametric Effect Monads and Semantics of Effect Systems. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14). Association for Computing Machinery, New York, NY, USA. 633–645. isbn:9781450325448 https://doi.org/10.1145/2535838.2535846 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Shin-ya Katsumata, Tetsuya Sato, and Tarmo Uustalu. 2018. Codensity Lifting of Monads and its Dual. Log. Methods Comput. Sci., 14, 4 (2018), https://doi.org/10.23638/LMCS-14(4:6)2018 Google ScholarGoogle ScholarCross RefCross Ref
  33. Dexter Kozen. 1985. A Probabilistic PDL. 30, 2 (1985), 162–178. https://doi.org/10.1145/800061.808758 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Saunders MacLane. 1971. Categories for the Working Mathematician. Springer-Verlag, New York. Graduate Texts in Mathematics, Vol. 5.Google ScholarGoogle Scholar
  35. Kenji Maillard, Catalin Hritcu, Exequiel Rivas, and Antoine Van Muylder. 2020. The next 700 relational program logics. Proc. ACM Program. Lang., 4, POPL (2020), 4:1–4:33. https://doi.org/10.1145/3371072 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Cristina Matache and Sam Staton. 2019. A Sound and Complete Logic for Algebraic Effects. In Foundations of Software Science and Computation Structures - 22nd International Conference, FOSSACS 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings, Mikolaj Bojanczyk and Alex Simpson (Eds.) (Lecture Notes in Computer Science, Vol. 11425). Springer, 382–399. https://doi.org/10.1007/978-3-030-17127-8_22 Google ScholarGoogle ScholarCross RefCross Ref
  37. Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. 18, 3 (1996), 325–353. https://doi.org/10.1145/229542.229547 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Aleksandar Nanevski, Anindya Banerjee, and Deepak Garg. 2013. Dependent Type Theory for Verification of Information Flow and Access Control Policies. ACM Trans. Program. Lang. Syst., 35, 2 (2013), 6:1–6:41. https://doi.org/10.1145/2491522.2491523 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Aleksandar Nanevski, J. Gregory Morrisett, and Lars Birkedal. 2008. Hoare type theory, polymorphism and separation. J. Funct. Program., 18, 5-6 (2008), 865–911. https://doi.org/10.1017/S0956796808006953 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Moni Naor and Eylon Yogev. 2019. Bloom Filters in Adversarial Environments. ACM Trans. Algorithms, 15, 3 (2019), 35:1–35:30. https://doi.org/10.1145/3306193 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Andrew Pitts and Ian Stark. 1998. Operational Reasoning for Functions with Local State. In Higher Order Operational Techniques in Semantics, Andrew Gordon and Andrew Pitts (Eds.). Publications of the Newton Institute, Cambridge University Press, 227–273. http://www.inf.ed.ac.uk/~stark/operfl.htmlGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  42. Gordon Plotkin. 1973. Lambda-definability and logical relations.Google ScholarGoogle Scholar
  43. Thomas Ristenpart, Hovav Shacham, and Thomas Shrimpton. 2011. Careful with Composition: Limitations of the Indifferentiability Framework. In Advances in Cryptology - EUROCRYPT 2011 - 30th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tallinn, Estonia, May 15-19, 2011. Proceedings, Kenneth G. Paterson (Ed.) (Lecture Notes in Computer Science, Vol. 6632). Springer, 487–506. https://doi.org/10.1007/978-3-642-20465-4_27 Google ScholarGoogle ScholarCross RefCross Ref
  44. Tetsuya Sato. 2016. Approximate Relational Hoare Logic for Continuous Random Samplings. In The Thirty-second Conference on the Mathematical Foundations of Programming Semantics, MFPS 2016, Carnegie Mellon University, Pittsburgh, PA, USA, May 23-26, 2016, Lars Birkedal (Ed.) (Electronic Notes in Theoretical Computer Science, Vol. 325). Elsevier, 277–298. https://doi.org/10.1016/j.entcs.2016.09.043 Google ScholarGoogle ScholarCross RefCross Ref
  45. Tetsuya Sato, Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Justin Hsu. 2019. Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization. PACMPL, 3, POPL (2019), 38:1–38:30. https://doi.org/10.1145/3290351 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Adam Ścibior, Ohad Kammar, Matthijs Vákár, Sam Staton, Hongseok Yang, Yufei Cai, Klaus Ostermann, Sean K. Moss, Chris Heunen, and Zoubin Ghahramani. 2017. Denotational Validation of Higher-order Bayesian Inference. Proc. ACM Program. Lang., 2, POPL (2017), Article 60, Dec., 29 pages. issn:2475-1421 https://doi.org/10.1145/3158148 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Joseph Tassarotti and Robert Harper. 2019. A Separation Logic for Concurrent Randomized Programs. Proc. ACM Program. Lang., 3, POPL (2019), Article 64, Jan., 30 pages. https://doi.org/10.1145/3290377 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Matthijs Vákár, Ohad Kammar, and Sam Staton. 2019. A domain theory for statistical probabilistic programming. PACMPL, 3, POPL (2019), 36:1–36:29. https://doi.org/10.1145/3290349 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Higher-order probabilistic adversarial computations: categorical semantics and program logics

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Proceedings of the ACM on Programming Languages
      Proceedings of the ACM on Programming Languages  Volume 5, Issue ICFP
      August 2021
      1011 pages
      EISSN:2475-1421
      DOI:10.1145/3482883
      Issue’s Table of Contents

      Copyright © 2021 Owner/Author

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 19 August 2021
      Published in pacmpl Volume 5, Issue ICFP

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!