Abstract

Differential privacy is a notion of confidentiality that protects the privacy of individuals while allowing useful computations on their private data. Deriving differential privacy guarantees for real programs is a difficult and error-prone task that calls for principled approaches and tool support. Approaches based on linear types and static analysis have recently emerged; however, an increasing number of programs achieve privacy using techniques that cannot be analyzed by these approaches. Examples include programs that aim for weaker, approximate differential privacy guarantees, programs that use the Exponential mechanism, and randomized programs that achieve differential privacy without using any standard mechanism. Providing support for reasoning about the privacy of such programs has been an open problem.
We report on CertiPriv, a machine-checked framework for reasoning about differential privacy built on top of the Coq proof assistant. The central component of CertiPriv is a quantitative extension of a probabilistic relational Hoare logic that enables one to derive differential privacy guarantees for programs from first principles. We demonstrate the expressiveness of CertiPriv using a number of examples whose formal analysis is out of the reach of previous techniques. In particular, we provide the first machine-checked proofs of correctness of the Laplacian and Exponential mechanisms and of the privacy of randomized and streaming algorithms from the recent literature.
Supplemental Material
- P. Audebaud and C. Paulin-Mohring. Proofs of randomized algorithms in Coq. Sci. Comput. Program., 74 (8): 568--589, 2009. Google Scholar
Digital Library
- G. Barthe and B. Köpf. Information-theoretic bounds for differentially private mechanisms. In 24rd IEEE Computer Security Foundations Symposium, CSF 2011, pages 191--204, Los Alamitos, 2011. IEEE Computer Society. Google Scholar
Digital Library
- G. Barthe, B. Grégoire, and S. Zanella Béguelin. Formal certification of code-based cryptographic proofs. In 36th ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2009, pages 90--101, New York, 2009. ACM. Google Scholar
Digital Library
- G. Barthe, B. Grégoire, S. Heraud, and S. Zanella Béguelin. Computer-aided security proofs for the working cryptographer. In Advances in Cryptology -- CRYPTO 2011, volume 6841 of Lecture Notes in Computer Science, pages 71--90, Heidelberg, 2011. Springer. Google Scholar
Digital Library
- A. Beimel, K. Nissim, and E. Omri. Distributed private data analysis: Simultaneously solving how and what. In Advances in Cryptology -- CRYPTO 2008, volume 5157 of Lecture Notes in Computer Science, pages 451--468, Heidelberg, 2008. Springer. Google Scholar
Digital Library
- M. Bellare and P. Rogaway. The security of triple encryption and a framework for code-based game-playing proofs. In Advances in Cryptology -- EUROCRYPT 2006, volume 4004 of Lecture Notes in Computer Science, pages 409--426, Heidelberg, 2006. Springer. Google Scholar
Digital Library
- N. Benton. Simple relational correctness proofs for static analyses and program transformations. In 31st ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2004, pages 14--25, New York, 2004. ACM. Google Scholar
Digital Library
- T.-H. H. Chan, E. Shi, and D. Song. Private and continual release of statistics. In 37th International colloquium on Automata, Languages and Programming, ICALP 2010, volume 6199 of Lecture Notes in Computer Science, pages 405--417, Heidelberg, 2010. Springer. Google Scholar
Digital Library
- S. Chaudhuri, S. Gulwani, R. Lublinerman, and S. Navidpour. Proving programs robust. In 8th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE '11. ACM, 2011. Google Scholar
Digital Library
- D. Clark, S. Hunt, and P. Malacaria. A static analysis for quantifying information flow in a simple imperative language. Journal of Computer Security, 15 (3): 321--371, 2007. Google Scholar
Digital Library
- M. R. Clarkson and F. B. Schneider. Hyperproperties. Journal of Computer Security, 18 (6): 1157--1210, 2010. Google Scholar
Digital Library
- J. Desharnais, F. Laviolette, and M. Tracol. Approximate analysis of probabilistic processes: Logic, simulation and games. In 5th International Conference on Quantitative Evaluation of Systems, QEST 2008, pages 264--273. IEEE Computer Society, 2008. Google Scholar
Digital Library
- C. Dwork. Differential privacy. In 33rd International Colloquium on Automata, Languages and Programming, ICALP 2006, volume 4052 of Lecture Notes in Computer Science, pages 1--12, Heidelberg, 2006. Springer. Google Scholar
Digital Library
- C. Dwork. Differential privacy: A survey of results. In Theory and Applications of Models of Computation, volume 4978 of Lecture Notes in Computer Science, pages 1--19, Heidelberg, 2008. Springer. Google Scholar
Digital Library
- C. Dwork. A firm foundation for private data analysis. Commun. ACM, 54 (1): 86--95, January 2011. Google Scholar
Digital Library
- C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in Cryptology -- EUROCRYPT 2006, volume 4004 of Lecture Notes in Computer Science, pages 486--503, Heidelberg, 2006. Springer. Google Scholar
Digital Library
- C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In 3rd Theory of Cryptography Conference, TCC 2006, volume 3876 of Lecture Notes in Computer Science, pages 265--284, Heidelberg, 2006. Springer. Google Scholar
Digital Library
- A. Gupta, K. Ligett, F. McSherry, A. Roth, and K. Talwar. Differentially private combinatorial optimization. In 21st Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, pages 1106--1125. SIAM, 2010. Google Scholar
Digital Library
- J. Hurd, A. McIver, and C. Morgan. Probabilistic guarded commands mechanized in HOL. Theor. Comput. Sci., 346 (1): 96--112, 2005. Google Scholar
Digital Library
- B. Jonsson, W. Yi, and K. G. Larsen. Probabilistic extensions of process algebras. In J. Bergstra, A. Ponse, and S. Smolka, editors, Handbook of Process Algebra, pages 685--710. Elsevier, Amsterdam, 2001.Google Scholar
Cross Ref
- S. P. Kasiviswanathan and A. Smith. A note on differential privacy: Defining resistance to arbitrary side information. Cryptology ePrint Archive, Report 2008/144, 2008.Google Scholar
- D. Kifer and A. Machanavajjhala. No free lunch in data privacy. In ph2011 International conference on Management of Data, SIGMOD '11, pages 193--204. ACM Press, 2011. Google Scholar
Digital Library
- F. McSherry and K. Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2007, pages 94--103, Washington, 2007. IEEE Computer Society. Google Scholar
Digital Library
- F. D. McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In 35th SIGMOD international conference on Management of Data, SIGMOD 2009, pages 19--30, New York, 2009. ACM. Google Scholar
Digital Library
- I. Mironov, O. Pandey, O. Reingold, and S. Vadhan. Computational differential privacy. In Advances in Cryptology -- CRYPTO 2009, volume 5677 of Lecture Notes in Computer Science, pages 126--142, Heidelberg, 2009. Springer. Google Scholar
Digital Library
- A. D. Pierro, C. Hankin, and H. Wiklicky. Approximate non-interference. Journal of Computer Security, 12 (1): 37--82, 2004. Google Scholar
Digital Library
- L. Pitt. A simple probabilistic approximation algorithm for vertex cover. Technical Report TR-404, Yale University, 1985.Google Scholar
- N. Ramsey and A. Pfeffer. Stochastic lambda calculus and monads of probability distributions. In 29th ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, POPL 2002, pages 154--165, New York, 2002. ACM. Google Scholar
Digital Library
- J. Reed and B. C. Pierce. Distance makes the types grow stronger: a calculus for differential privacy. In 15th ACM SIGPLAN international conference on Functional programming, ICFP 2010, pages 157--168, New York, 2010. ACM. Google Scholar
Digital Library
- I. Roy, S. T. V. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: security and privacy for MapReduce. In 7th USENIX conference on Networked Systems Design and Implementation, NSDI 2010, pages 297--312, Berkeley, 2010. USENIX Association. Google Scholar
Digital Library
- A. Sabelfeld and D. Sands. Probabilistic noninterference for multi-threaded programs. In 13th IEEE workshop on Computer Security Foundations, CSFW 2000, pages 200--215, Los Alamitos, 2000. IEEE Computer Society. Google Scholar
Digital Library
- R. Segala and A. Turrini. Approximated computationally bounded simulation relations for probabilistic automata. In 20th IEEE Computer Security Foundations symposium, CSF 2007, pages 140--156, 2007. Google Scholar
Digital Library
- T. Terauchi and A. Aiken. Secure information flow as a safety problem. In 12th International Symposium on Static Analysis, SAS 2005, volume 3672 of Lecture Notes in Computer Science, pages 352--367, Heidelberg, 2005. Springer. Google Scholar
Digital Library
- The Coq development team. The Coq Proof Assistant Reference Manual Version 8.3. Online -- http://coq.inria.fr, 2010.Google Scholar
- M. C. Tschantz, D. Kaynar, and A. Datta. Formal verification of differential privacy for interactive systems. Electronic Notes in Theoretical Computer Science, 276: 61--79, 2011. Google Scholar
Digital Library
Index Terms
Probabilistic relational reasoning for differential privacy
Recommendations
Probabilistic Relational Reasoning for Differential Privacy
Differential privacy is a notion of confidentiality that allows useful computations on sensible data while protecting the privacy of individuals. Proving differential privacy is a difficult and error-prone task that calls for principled approaches and ...
Probabilistic relational reasoning for differential privacy
POPL '12: Proceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languagesDifferential privacy is a notion of confidentiality that protects the privacy of individuals while allowing useful computations on their private data. Deriving differential privacy guarantees for real programs is a difficult and error-prone task that ...
Proving Differential Privacy in Hoare Logic
CSF '14: Proceedings of the 2014 IEEE 27th Computer Security Foundations SymposiumDifferential privacy is a rigorous, worst-case notion of privacy-preserving computation. Informally, a probabilistic program is differentially private if the participation of a single individual in the input database has a limited effect on the program'...







Comments