skip to main content
research-article

Probabilistic relational reasoning for differential privacy

Published:25 January 2012Publication History
Skip Abstract Section

Abstract

Differential privacy is a notion of confidentiality that protects the privacy of individuals while allowing useful computations on their private data. Deriving differential privacy guarantees for real programs is a difficult and error-prone task that calls for principled approaches and tool support. Approaches based on linear types and static analysis have recently emerged; however, an increasing number of programs achieve privacy using techniques that cannot be analyzed by these approaches. Examples include programs that aim for weaker, approximate differential privacy guarantees, programs that use the Exponential mechanism, and randomized programs that achieve differential privacy without using any standard mechanism. Providing support for reasoning about the privacy of such programs has been an open problem.

We report on CertiPriv, a machine-checked framework for reasoning about differential privacy built on top of the Coq proof assistant. The central component of CertiPriv is a quantitative extension of a probabilistic relational Hoare logic that enables one to derive differential privacy guarantees for programs from first principles. We demonstrate the expressiveness of CertiPriv using a number of examples whose formal analysis is out of the reach of previous techniques. In particular, we provide the first machine-checked proofs of correctness of the Laplacian and Exponential mechanisms and of the privacy of randomized and streaming algorithms from the recent literature.

Skip Supplemental Material Section

Supplemental Material

popl_2a_2.mp4

References

  1. P. Audebaud and C. Paulin-Mohring. Proofs of randomized algorithms in Coq. Sci. Comput. Program., 74 (8): 568--589, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. G. Barthe and B. Köpf. Information-theoretic bounds for differentially private mechanisms. In 24rd IEEE Computer Security Foundations Symposium, CSF 2011, pages 191--204, Los Alamitos, 2011. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Barthe, B. Grégoire, and S. Zanella Béguelin. Formal certification of code-based cryptographic proofs. In 36th ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2009, pages 90--101, New York, 2009. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. G. Barthe, B. Grégoire, S. Heraud, and S. Zanella Béguelin. Computer-aided security proofs for the working cryptographer. In Advances in Cryptology -- CRYPTO 2011, volume 6841 of Lecture Notes in Computer Science, pages 71--90, Heidelberg, 2011. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. Beimel, K. Nissim, and E. Omri. Distributed private data analysis: Simultaneously solving how and what. In Advances in Cryptology -- CRYPTO 2008, volume 5157 of Lecture Notes in Computer Science, pages 451--468, Heidelberg, 2008. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. Bellare and P. Rogaway. The security of triple encryption and a framework for code-based game-playing proofs. In Advances in Cryptology -- EUROCRYPT 2006, volume 4004 of Lecture Notes in Computer Science, pages 409--426, Heidelberg, 2006. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. N. Benton. Simple relational correctness proofs for static analyses and program transformations. In 31st ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2004, pages 14--25, New York, 2004. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. T.-H. H. Chan, E. Shi, and D. Song. Private and continual release of statistics. In 37th International colloquium on Automata, Languages and Programming, ICALP 2010, volume 6199 of Lecture Notes in Computer Science, pages 405--417, Heidelberg, 2010. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. S. Chaudhuri, S. Gulwani, R. Lublinerman, and S. Navidpour. Proving programs robust. In 8th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE '11. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. D. Clark, S. Hunt, and P. Malacaria. A static analysis for quantifying information flow in a simple imperative language. Journal of Computer Security, 15 (3): 321--371, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. R. Clarkson and F. B. Schneider. Hyperproperties. Journal of Computer Security, 18 (6): 1157--1210, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Desharnais, F. Laviolette, and M. Tracol. Approximate analysis of probabilistic processes: Logic, simulation and games. In 5th International Conference on Quantitative Evaluation of Systems, QEST 2008, pages 264--273. IEEE Computer Society, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C. Dwork. Differential privacy. In 33rd International Colloquium on Automata, Languages and Programming, ICALP 2006, volume 4052 of Lecture Notes in Computer Science, pages 1--12, Heidelberg, 2006. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Dwork. Differential privacy: A survey of results. In Theory and Applications of Models of Computation, volume 4978 of Lecture Notes in Computer Science, pages 1--19, Heidelberg, 2008. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. C. Dwork. A firm foundation for private data analysis. Commun. ACM, 54 (1): 86--95, January 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in Cryptology -- EUROCRYPT 2006, volume 4004 of Lecture Notes in Computer Science, pages 486--503, Heidelberg, 2006. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In 3rd Theory of Cryptography Conference, TCC 2006, volume 3876 of Lecture Notes in Computer Science, pages 265--284, Heidelberg, 2006. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Gupta, K. Ligett, F. McSherry, A. Roth, and K. Talwar. Differentially private combinatorial optimization. In 21st Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, pages 1106--1125. SIAM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Hurd, A. McIver, and C. Morgan. Probabilistic guarded commands mechanized in HOL. Theor. Comput. Sci., 346 (1): 96--112, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. B. Jonsson, W. Yi, and K. G. Larsen. Probabilistic extensions of process algebras. In J. Bergstra, A. Ponse, and S. Smolka, editors, Handbook of Process Algebra, pages 685--710. Elsevier, Amsterdam, 2001.Google ScholarGoogle ScholarCross RefCross Ref
  21. S. P. Kasiviswanathan and A. Smith. A note on differential privacy: Defining resistance to arbitrary side information. Cryptology ePrint Archive, Report 2008/144, 2008.Google ScholarGoogle Scholar
  22. D. Kifer and A. Machanavajjhala. No free lunch in data privacy. In ph2011 International conference on Management of Data, SIGMOD '11, pages 193--204. ACM Press, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. F. McSherry and K. Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2007, pages 94--103, Washington, 2007. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. F. D. McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In 35th SIGMOD international conference on Management of Data, SIGMOD 2009, pages 19--30, New York, 2009. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. I. Mironov, O. Pandey, O. Reingold, and S. Vadhan. Computational differential privacy. In Advances in Cryptology -- CRYPTO 2009, volume 5677 of Lecture Notes in Computer Science, pages 126--142, Heidelberg, 2009. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. D. Pierro, C. Hankin, and H. Wiklicky. Approximate non-interference. Journal of Computer Security, 12 (1): 37--82, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. L. Pitt. A simple probabilistic approximation algorithm for vertex cover. Technical Report TR-404, Yale University, 1985.Google ScholarGoogle Scholar
  28. N. Ramsey and A. Pfeffer. Stochastic lambda calculus and monads of probability distributions. In 29th ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2002, pages 154--165, New York, 2002. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. J. Reed and B. C. Pierce. Distance makes the types grow stronger: a calculus for differential privacy. In 15th ACM SIGPLAN international conference on Functional programming, ICFP 2010, pages 157--168, New York, 2010. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. I. Roy, S. T. V. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: security and privacy for MapReduce. In 7th USENIX conference on Networked Systems Design and Implementation, NSDI 2010, pages 297--312, Berkeley, 2010. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. A. Sabelfeld and D. Sands. Probabilistic noninterference for multi-threaded programs. In 13th IEEE workshop on Computer Security Foundations, CSFW 2000, pages 200--215, Los Alamitos, 2000. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. R. Segala and A. Turrini. Approximated computationally bounded simulation relations for probabilistic automata. In 20th IEEE Computer Security Foundations symposium, CSF 2007, pages 140--156, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. T. Terauchi and A. Aiken. Secure information flow as a safety problem. In 12th International Symposium on Static Analysis, SAS 2005, volume 3672 of Lecture Notes in Computer Science, pages 352--367, Heidelberg, 2005. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. The Coq development team. The Coq Proof Assistant Reference Manual Version 8.3. Online -- http://coq.inria.fr, 2010.Google ScholarGoogle Scholar
  35. M. C. Tschantz, D. Kaynar, and A. Datta. Formal verification of differential privacy for interactive systems. Electronic Notes in Theoretical Computer Science, 276: 61--79, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Probabilistic relational reasoning for differential privacy

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGPLAN Notices
            ACM SIGPLAN Notices  Volume 47, Issue 1
            POPL '12
            January 2012
            569 pages
            ISSN:0362-1340
            EISSN:1558-1160
            DOI:10.1145/2103621
            Issue’s Table of Contents
            • cover image ACM Conferences
              POPL '12: Proceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages
              January 2012
              602 pages
              ISBN:9781450310833
              DOI:10.1145/2103656

            Copyright © 2012 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 25 January 2012

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!