skip to main content
research-article

LightDP: towards automating differential privacy proofs

Published:01 January 2017Publication History
Skip Abstract Section

Abstract

The growing popularity and adoption of differential privacy in academic and industrial settings has resulted in the development of increasingly sophisticated algorithms for releasing information while preserving privacy. Accompanying this phenomenon is the natural rise in the development and publication of incorrect algorithms, thus demonstrating the necessity of formal verification tools. However, existing formal methods for differential privacy face a dilemma: methods based on customized logics can verify sophisticated algorithms but come with a steep learning curve and significant annotation burden on the programmers, while existing programming platforms lack expressive power for some sophisticated algorithms.

In this paper, we present LightDP, a simple imperative language that strikes a better balance between expressive power and usability. The core of LightDP is a novel relational type system that separates relational reasoning from privacy budget calculations. With dependent types, the type system is powerful enough to verify sophisticated algorithms where the composition theorem falls short. In addition, the inference engine of LightDP infers most of the proof details, and even searches for the proof with minimal privacy cost when multiple proofs exist. We show that LightDP verifies sophisticated algorithms with little manual effort.

References

  1. A. Aiken and E. L. Wimmers. Type inclusion constraints and type inference. In FPLCA, pages 31–41, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. G. Barthe and F. Olmedo. Beyond differential privacy: Composition theorems and relational logic for f-divergences between probabilistic programs. In ICALP, pages 49–60, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Barthe, B. Köpf, F. Olmedo, and S. Zanella Béguelin. Probabilistic relational reasoning for differential privacy. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 97–110, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. G. Barthe, G. Danezis, B. Grégoire, C. Kunz, and S. Zanella-Béguelin. Verified computational differential privacy with applications to smart metering. In 2013 IEEE 26th Computer Security Foundations Symposium, pages 287–301, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, C. Kunz, and P. Y. Strub. Proving differential privacy in hoare logic. In 2014 IEEE 27th Computer Security Foundations Symposium, pages 411–424, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, A. Roth, and P. Strub. Higher-order approximate relational refinement types for mechanism design and differential privacy. In POPL, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. N. Bjørner, A.-D. Phan, and L. Fleckenstein. νZ — An Optimizing SMT Solver, pages 194–199. 2015.Google ScholarGoogle Scholar
  8. H. Chan, E. Shi, and D. Song. Private and continual release of statistics. ACM Transactions on Information and System Security, 14(3), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Y. Chen and A. Machanavajjhala. On the privacy properties of variants on the sparse vector technique. http://arxiv.org/abs/1508.07306, 2015.Google ScholarGoogle Scholar
  10. L. D’Antoni, M. Gaboardi, E. J. Gallego Arias, A. Haeberlen, and B. Pierce. Sensitivity analysis using type-based constraints. In Proceedings of the 1st Annual Workshop on Functional Programming Concepts in Domainspecific Languages, pages 43–50, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4):211– 407, 2014. ISSN 1551-305X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486–503, 2006a. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006b. H. Ebadi, D. Sands, and G. Schneider. Differential privacy: Now it’s getting personal. In POPL, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. U. Erlingsson, V. Pihur, and A. Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS ’14, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. Gaboardi, A. Haeberlen, J. Hsu, A. Narayan, and B. C. Pierce. Linear dependent types for differential privacy. In Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’13, pages 357–370, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Greenberg. Apple’s ‘differential privacy’ is about collecting your data – but not Your data. Wired, https://www.wired.com/2016/ 06/apples-differential-privacy-collecting-data/, 2016.Google ScholarGoogle Scholar
  17. C. Haack and J. B. Wells. Type error slicing in implicitly typed higher-order languages. Science of Computer Programming, 50(1–3):189–224, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. D. Kifer and A. Machanavajjhala. Pufferfish: A framework for mathematical privacy definitions. ACM Trans. Database Syst., 39(1):3:1–3:36, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. Kozen. Semantics of probabilistic programs. Journal of Computer and System Sciences, 22(3):328 – 350, 1981.Google ScholarGoogle ScholarCross RefCross Ref
  20. K. R. M. Leino. Dafny: An automatic program verifier for functional correctness. In Proceedings of the 16th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning, pages 348–370, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Lyu, D. Su, and N. Li. Understanding the sparse vector technique for differential privacy. https://arxiv.org/abs/1603.01699, 2016.Google ScholarGoogle Scholar
  22. A. Machanavajjhala, D. Kifer, J. Abowd, J. Gehrke, and L. Vilhuber. Privacy: From theory to practice on the map. In Proceedings of the IEEE International Conference on Data Engineering (ICDE), pages 277–286, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. P. Martin-Löf. Intuitionistic type theory. Naples: Bibliopolis, 76, 1984.Google ScholarGoogle Scholar
  24. F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, pages 94–103, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. F. D. McSherry. Privacy integrated queries: An extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pages 19– 30, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. P. Mohan, A. Thakurta, E. Shi, D. Song, and D. Culler. Gupt: Privacy preserving data analysis made easy. In Proceedings of the ACM SIGMOD International Conference on Management of Data, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. Reed and B. C. Pierce. Distance makes the types grow stronger: A calculus for differential privacy. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming, ICFP ’10, pages 157–168, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. A. Roth. The sparse vector technique. http://www.cis.upenn. edu/˜aaroth/courses/slides/Lecture11.pdf, 2011.Google ScholarGoogle Scholar
  29. I. Roy, S. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: Security and privacy for MapReduce. In NSDI, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. C. Tschantz, D. Kaynar, and A. Datta. Formal verification of differential privacy for interactive systems (extended abstract). Electron. Notes Theor. Comput. Sci., 276:61–79, Sept. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Wand. A simple algorithm and proof for type inference. Fundamenta Informaticae, 10:115–122, 1987.Google ScholarGoogle ScholarCross RefCross Ref
  32. L. Xu, K. Chatzikokolakis, and H. Lin. Metrics for Differential Privacy in Concurrent Systems, pages 199–215. 2014.Google ScholarGoogle Scholar
  33. D. Zhang and D. Kifer. LightDP: Towards automating differential privacy proofs. CoRR, abs/1607.08228, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Zhang, X. Xiao, and X. Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In SIGMOD, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. LightDP: towards automating differential privacy proofs

                  Recommendations

                  Comments

                  Login options

                  Check if you have access through your login credentials or your institution to get full access on this article.

                  Sign in

                  Full Access

                  • Published in

                    cover image ACM SIGPLAN Notices
                    ACM SIGPLAN Notices  Volume 52, Issue 1
                    POPL '17
                    January 2017
                    901 pages
                    ISSN:0362-1340
                    EISSN:1558-1160
                    DOI:10.1145/3093333
                    Issue’s Table of Contents
                    • cover image ACM Conferences
                      POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages
                      January 2017
                      901 pages
                      ISBN:9781450346603
                      DOI:10.1145/3009837

                    Copyright © 2017 ACM

                    Publisher

                    Association for Computing Machinery

                    New York, NY, United States

                    Publication History

                    • Published: 1 January 2017

                    Check for updates

                    Qualifiers

                    • research-article

                  PDF Format

                  View or Download as a PDF file.

                  PDF

                  eReader

                  View online with eReader.

                  eReader
                  About Cookies On This Site

                  We use cookies to ensure that we give you the best experience on our website.

                  Learn more

                  Got it!