skip to main content
research-article
Open Access

Synthesizing coupling proofs of differential privacy

Published:27 December 2017Publication History
Skip Abstract Section

Abstract

Differential privacy has emerged as a promising probabilistic formulation of privacy, generating intense interest within academia and industry. We present a push-button, automated technique for verifying ε-differential privacy of sophisticated randomized algorithms. We make several conceptual, algorithmic, and practical contributions: (i) Inspired by the recent advances on approximate couplings and randomness alignment, we present a new proof technique called coupling strategies, which casts differential privacy proofs as a winning strategy in a game where we have finite privacy resources to expend. (ii) To discover a winning strategy, we present a constraint-based formulation of the problem as a set of Horn modulo couplings (HMC) constraints, a novel combination of first-order Horn clauses and probabilistic constraints. (iii) We present a technique for solving HMC constraints by transforming probabilistic constraints into logical constraints with uninterpreted functions. (iv) Finally, we implement our technique in the FairSquare verifier and provide the first automated privacy proofs for a number of challenging algorithms from the differential privacy literature, including Report Noisy Max, the Exponential Mechanism, and the Sparse Vector Mechanism.

Skip Supplemental Material Section

Supplemental Material

synthesizingcouplingproofsofdifferentialprivacy.webm

References

  1. Aws Albarghouthi, Loris D’Antoni, Samuel Drews, and Aditya V. Nori. 2017. FairSquare: Probabilistic Verification of Program Fairness. Proc. ACM Program. Lang. 1, OOPSLA, Article 80 (Oct. 2017), 30 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Arthur Azevedo de Amorim, Marco Gaboardi, Emilio Jesús Gallego Arias, and Justin Hsu. 2014. Really natural linear indexed type-checking. In Symposium on Implementation and Application of Functional Programming Languages (IFL), Boston, Massachusetts . ACM Press, 5:1–5:12. http://arxiv.org/abs/1503.04522Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Arthur Azevedo de Amorim, Marco Gaboardi, Justin Hsu, Shin-ya Katsumata, and Ikram Cherigui. 2017. A semantic account of metric preservation. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Paris, France . 545–556. http://arxiv.org/abs/1702.00374 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2017a. Proving uniformity and independence by self-composition and coupling. In International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR), Maun, Botswana (EPiC Series in Computing) , Vol. 46. 385–403. https://arxiv.org/abs/1701.06477Google ScholarGoogle Scholar
  5. Gilles Barthe, Thomas Espitau, Justin Hsu, Tetsuya Sato, and Pierre-Yves Strub. 2017b. ⋆-Liftings for differential privacy. In International Colloquium on Automata, Languages and Programming (ICALP), Warsaw, Poland . https://arxiv.org/abs/1705. 00133Google ScholarGoogle Scholar
  6. Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016a. Advanced probabilistic couplings for differential privacy. In ACM SIGSAC Conference on Computer and Communications Security (CCS), Vienna, Austria . https://arxiv.org/abs/1606.07143 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, César Kunz, and Pierre-Yves Strub. 2014. Proving differential privacy in Hoare logic. In IEEE Computer Security Foundations Symposium (CSF), Vienna, Austria. 411–424. http://arxiv.org/abs/1407.2988 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, and Pierre-Yves Strub. 2015. Higher-order approximate relational refinement types for mechanism design and differential privacy. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Mumbai, India . 55–68. http://arxiv.org/abs/1407.6845 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Gilles Barthe, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016b. Proving differential privacy via probabilistic couplings. In IEEE Symposium on Logic in Computer Science (LICS), New York, New York. 749–758. http://arxiv.org/abs/1601.05047 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gilles Barthe, Marco Gaboardi, Justin Hsu, and Benjamin C. Pierce. 2016c. Programming language techniques for differential privacy. ACM SIGLOG News 3, 1 (Jan. 2016), 34–53. http://siglog.hosting.acm.org/wp-content/uploads/2016/01/siglog_ news_7.pdfGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  11. Gilles Barthe, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2017c. Coupling proofs are probabilistic product programs. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Paris, France. 161–174. http://arxiv.org/abs/1607.03455 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella-Béguelin. 2013. Probabilistic Relational Reasoning for Differential Privacy. ACM Transactions on Programming Languages and Systems 35, 3 (2013), 9. http://software.imdea. org/~bkoepf/papers/toplas13.pdfGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  13. Gilles Barthe and Federico Olmedo. 2013. Beyond Differential Privacy: Composition Theorems and Relational Logic for f -divergences between Probabilistic Programs. In International Colloquium on Automata, Languages and Programming (ICALP), Riga, Latvia (Lecture Notes in Computer Science) , Vol. 7966. Springer-Verlag, 49–60. http://certicrypt.gforge.inria. fr/2013.ICALP.pdf Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Tewodros Beyene, Swarat Chaudhuri, Corneliu Popeea, and Andrey Rybalchenko. 2014. A Constraint-based Approach to Solving Games on Infinite Graphs. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), San Diego, California . New York, NY, USA, 221–233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Tewodros A Beyene, Corneliu Popeea, and Andrey Rybalchenko. 2013. Solving existentially quantified Horn clauses. In International Conference on Computer Aided Verification (CAV), Saint Petersburg, Russia . Springer-Verlag, 869–882. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Dirk Beyer, Alessandro Cimatti, Alberto Griggio, M. Erkan Keremoglu, and Roberto Sebastiani. 2009. Software model checking via large-block encoding. In Formal Methods in Computer-Aided Design (FMCAD), Austin, Texas. IEEE, 25–32. Google ScholarGoogle ScholarCross RefCross Ref
  17. Nikolaj Bjørner, Arie Gurfinkel, Kenneth L. McMillan, and Andrey Rybalchenko. 2015. Horn clause solvers for program verification. In Fields of Logic and Computation II. Springer-Verlag, 24–51. Google ScholarGoogle Scholar
  18. James Bornholt, Emina Torlak, Dan Grossman, and Luis Ceze. 2016. Optimizing synthesis with metasketches. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Saint Petersburg, Florida . 775–788. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. T.-H. Hubert Chan, Elaine Shi, and Dawn Song. 2011. Private and continual release of statistics. ACM Transactions on Information and System Security 14, 3 (2011), 26. http://eprint.iacr.org/2010/076.pdfGoogle ScholarGoogle Scholar
  20. Alessandro Cimatti, Alberto Griggio, Bastiaan Joost Schaafsma, and Roberto Sebastiani. 2013. The MathSAT5 SMT Solver., Vol. 7795. Springer-Verlag, 93–107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Michael R. Clarkson and Fred B. Schneider. 2010. Hyperproperties. Journal of Computer Security 18, 6 (2010), 1157–1210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. 2006. Calibrating Noise to Sensitivity in Private Data Analysis. In IACR Theory of Cryptography Conference (TCC), New York, New York (Lecture Notes in Computer Science), Vol. 3876. Springer-Verlag, 265–284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Cynthia Dwork, Moni Naor, Toniann Pitassi, and Guy N. Rothblum. 2010. Differential privacy under continual observation. In ACM SIGACT Symposium on Theory of Computing (STOC), Cambridge, Massachusetts. 715–724. http://www.mit.edu/ ~rothblum/papers/continalobs.pdf Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Cynthia Dwork, Moni Naor, Omer Reingold, Guy N. Rothblum, and Salil Vadhan. 2009. On the complexity of differentially private data release: Efficient algorithms and hardness results. In ACM SIGACT Symposium on Theory of Computing (STOC), Bethesda, Maryland . 381–390. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Vol. 9. Now Publishers, Inc. 211–407 pages.Google ScholarGoogle Scholar
  26. Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, and Benjamin C. Pierce. 2013. Linear dependent types for differential privacy. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Rome, Italy. 357–370. http://dl.acm.org/citation.cfm?id=2429113 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Susanne Graf and Hassen Saïdi. 1997. Construction of abstract state graphs with PVS. In International Conference on Computer Aided Verification (CAV), Haifa, Israel . Springer-Verlag, 72–83. Google ScholarGoogle ScholarCross RefCross Ref
  28. Sergey Grebenshchikov, Ashutosh Gupta, Nuno P. Lopes, Corneliu Popeea, and Andrey Rybalchenko. 2012a. HSF (C): A software verifier based on Horn clauses. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Tallinn, Estonia . Springer-Verlag, 549–551. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sergey Grebenshchikov, Nuno P. Lopes, Corneliu Popeea, and Andrey Rybalchenko. 2012b. Synthesizing Software Verifiers from Proof Rules. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Beijing, China . 405–416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sumit Gulwani, Susmit Jha, Ashish Tiwari, and Ramarathnam Venkatesan. 2011. Synthesis of loop-free programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), San Jose, California . Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Arie Gurfinkel, Temesghen Kahsai, and Jorge A. Navas. 2015. SeaHorn: A Framework for Verifying C Programs (Competition Contribution). In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), London, England . 447–450. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Hossein Hojjat, Filip Konečn`y, Florent Garnier, Radu Iosif, Viktor Kuncak, and Philipp Rümmer. 2012. A verification toolkit for numerical transition systems. In International Symposium on Formal Methods (FM), Paris, France. Springer-Verlag, 247–251. Google ScholarGoogle ScholarCross RefCross Ref
  33. Justin Hsu. 2017. Probabilistic Couplings for Probabilistic Reasoning. Ph.D. Dissertation. University of Pennsylvania. arXiv: cs.LO/1710.09951 https://arxiv.org/abs/1710.09951Google ScholarGoogle Scholar
  34. Boris Köpf and Andrey Rybalchenko. 2010. Approximation and randomization for quantitative information-flow analysis. In IEEE Computer Security Foundations Symposium (CSF), Edinburgh, Scotland. 3–14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Shuvendu K. Lahiri, Robert Nieuwenhuis, and Albert Oliveras. 2006. SMT techniques for fast predicate abstraction. In International Conference on Computer Aided Verification (CAV), Seattle, Washington . Springer-Verlag, 424–437. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Torgny Lindvall. 2002. Lectures on the coupling method. Courier Corporation.Google ScholarGoogle Scholar
  37. Min Lyu, Dong Su, and Ninghui Li. 2017. Understanding the Sparse Vector Technique for Differential Privacy. In International Conference on Very Large Data Bases (VLDB), Munich, Germany , Vol. 10. 637–648. http://arxiv.org/abs/1603.01699Google ScholarGoogle Scholar
  38. Kenneth L. McMillan and Andrey Rybalchenko. 2013. Solving constrained Horn clauses using interpolation. Technical Report MSR-TR-2013-6. Microsoft Research.Google ScholarGoogle Scholar
  39. Frank McSherry and Kunal Talwar. 2007. Mechanism Design via Differential Privacy. In IEEE Symposium on Foundations of Computer Science (FOCS), Providence, Rhode Island . 94–103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. George C. Necula. 2000. Translation validation for an optimizing compiler. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Vancouver, British Columbia , Vol. 35. 83–94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Federico Olmedo. 2014. Approximate relational reasoning for probabilistic programs. Ph.D. Dissertation. Universidad Politécnica de Madrid. http://oa.upm.es/23088/1/FEDERICO_OLMEDO.pdfGoogle ScholarGoogle Scholar
  42. Amir Pnueli, Michael Siegel, and Eli Singerman. 1998. Translation validation. International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Lisbon, Portugal (1998), 151–166.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Jason Reed and Benjamin C. Pierce. 2010. Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy. In ACM SIGPLAN International Conference on Functional Programming (ICFP), Baltimore, Maryland. http://dl.acm.org/ citation.cfm?id=1863568Google ScholarGoogle Scholar
  44. Tetsuya Sato. 2016. Approximate Relational Hoare Logic for Continuous Random Samplings. In Conference on the Mathematical Foundations of Programming Semantics (MFPS), Pittsburgh, Pennsylvania (Electronic Notes in Theoretical Computer Science) , Vol. 325. Elsevier, 277–298. https://arxiv.org/abs/1603.01445Google ScholarGoogle Scholar
  45. Armando Solar-Lezama, Liviu Tancau, Rastislav Bodík, Sanjit A. Seshia, and Vijay A. Saraswat. 2006. Combinatorial sketching for finite programs. In Asian Symposium on Programming Languages and Systems (APLAS), San Jose, California. 404–415. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Marcelo Sousa and Isil Dillig. 2016. Cartesian Hoare Logic for Verifying k -safety Properties. ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Santa Barbara, California , 57–69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Saurabh Srivastava, Sumit Gulwani, and Jeffrey S. Foster. 2010. From Program Verification to Program Synthesis. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Madrid, Spain . New York, NY, USA, 313–326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Tachio Terauchi and Alexander Aiken. 2005. Secure information flow as a safety problem. In International Symposium on Static Analysis (SAS), London, England (Lecture Notes in Computer Science) , Vol. 3672. Springer-Verlag, 352–367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Daniel Winograd-Cort, Andreas Haeberlen, Aaron Roth, and Benjamin C. Pierce. 2017. A framework for adaptive differential privacy. In ACM SIGPLAN International Conference on Functional Programming (ICFP), Oxford, England. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Danfeng Zhang and Daniel Kifer. 2017. LightDP: Towards Automating Differential Privacy Proofs. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Paris, France . 888–901. https://arxiv.org/abs/1607.08228Google ScholarGoogle Scholar

Index Terms

  1. Synthesizing coupling proofs of differential privacy

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Programming Languages
        Proceedings of the ACM on Programming Languages  Volume 2, Issue POPL
        January 2018
        1961 pages
        EISSN:2475-1421
        DOI:10.1145/3177123
        Issue’s Table of Contents

        Copyright © 2017 Owner/Author

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 27 December 2017
        Published in pacmpl Volume 2, Issue POPL

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!