Abstract
The growing popularity and adoption of differential privacy in academic and industrial settings has resulted in the development of increasingly sophisticated algorithms for releasing information while preserving privacy. Accompanying this phenomenon is the natural rise in the development and publication of incorrect algorithms, thus demonstrating the necessity of formal verification tools. However, existing formal methods for differential privacy face a dilemma: methods based on customized logics can verify sophisticated algorithms but come with a steep learning curve and significant annotation burden on the programmers, while existing programming platforms lack expressive power for some sophisticated algorithms.
In this paper, we present LightDP, a simple imperative language that strikes a better balance between expressive power and usability. The core of LightDP is a novel relational type system that separates relational reasoning from privacy budget calculations. With dependent types, the type system is powerful enough to verify sophisticated algorithms where the composition theorem falls short. In addition, the inference engine of LightDP infers most of the proof details, and even searches for the proof with minimal privacy cost when multiple proofs exist. We show that LightDP verifies sophisticated algorithms with little manual effort.
- A. Aiken and E. L. Wimmers. Type inclusion constraints and type inference. In FPLCA, pages 31–41, 1993. Google Scholar
Digital Library
- G. Barthe and F. Olmedo. Beyond differential privacy: Composition theorems and relational logic for f-divergences between probabilistic programs. In ICALP, pages 49–60, 2013. Google Scholar
Digital Library
- G. Barthe, B. Köpf, F. Olmedo, and S. Zanella Béguelin. Probabilistic relational reasoning for differential privacy. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 97–110, 2012. Google Scholar
Digital Library
- G. Barthe, G. Danezis, B. Grégoire, C. Kunz, and S. Zanella-Béguelin. Verified computational differential privacy with applications to smart metering. In 2013 IEEE 26th Computer Security Foundations Symposium, pages 287–301, 2013. Google Scholar
Digital Library
- G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, C. Kunz, and P. Y. Strub. Proving differential privacy in hoare logic. In 2014 IEEE 27th Computer Security Foundations Symposium, pages 411–424, 2014. Google Scholar
Digital Library
- G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, A. Roth, and P. Strub. Higher-order approximate relational refinement types for mechanism design and differential privacy. In POPL, 2015. Google Scholar
Digital Library
- N. Bjørner, A.-D. Phan, and L. Fleckenstein. νZ — An Optimizing SMT Solver, pages 194–199. 2015.Google Scholar
- H. Chan, E. Shi, and D. Song. Private and continual release of statistics. ACM Transactions on Information and System Security, 14(3), 2011. Google Scholar
Digital Library
- Y. Chen and A. Machanavajjhala. On the privacy properties of variants on the sparse vector technique. http://arxiv.org/abs/1508.07306, 2015.Google Scholar
- L. D’Antoni, M. Gaboardi, E. J. Gallego Arias, A. Haeberlen, and B. Pierce. Sensitivity analysis using type-based constraints. In Proceedings of the 1st Annual Workshop on Functional Programming Concepts in Domainspecific Languages, pages 43–50, 2013. Google Scholar
Digital Library
- L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2008. Google Scholar
Digital Library
- C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4):211– 407, 2014. ISSN 1551-305X. Google Scholar
Digital Library
- C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486–503, 2006a. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006b. H. Ebadi, D. Sands, and G. Schneider. Differential privacy: Now it’s getting personal. In POPL, 2015. Google Scholar
Digital Library
- U. Erlingsson, V. Pihur, and A. Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS ’14, 2014. Google Scholar
Digital Library
- M. Gaboardi, A. Haeberlen, J. Hsu, A. Narayan, and B. C. Pierce. Linear dependent types for differential privacy. In Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’13, pages 357–370, 2013. Google Scholar
Digital Library
- A. Greenberg. Apple’s ‘differential privacy’ is about collecting your data – but not Your data. Wired, https://www.wired.com/2016/ 06/apples-differential-privacy-collecting-data/, 2016.Google Scholar
- C. Haack and J. B. Wells. Type error slicing in implicitly typed higher-order languages. Science of Computer Programming, 50(1–3):189–224, 2004. Google Scholar
Digital Library
- D. Kifer and A. Machanavajjhala. Pufferfish: A framework for mathematical privacy definitions. ACM Trans. Database Syst., 39(1):3:1–3:36, 2014. Google Scholar
Digital Library
- D. Kozen. Semantics of probabilistic programs. Journal of Computer and System Sciences, 22(3):328 – 350, 1981.Google Scholar
Cross Ref
- K. R. M. Leino. Dafny: An automatic program verifier for functional correctness. In Proceedings of the 16th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning, pages 348–370, 2010. Google Scholar
Digital Library
- M. Lyu, D. Su, and N. Li. Understanding the sparse vector technique for differential privacy. https://arxiv.org/abs/1603.01699, 2016.Google Scholar
- A. Machanavajjhala, D. Kifer, J. Abowd, J. Gehrke, and L. Vilhuber. Privacy: From theory to practice on the map. In Proceedings of the IEEE International Conference on Data Engineering (ICDE), pages 277–286, 2008. Google Scholar
Digital Library
- P. Martin-Löf. Intuitionistic type theory. Naples: Bibliopolis, 76, 1984.Google Scholar
- F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, pages 94–103, 2007. Google Scholar
Digital Library
- F. D. McSherry. Privacy integrated queries: An extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pages 19– 30, 2009. Google Scholar
Digital Library
- P. Mohan, A. Thakurta, E. Shi, D. Song, and D. Culler. Gupt: Privacy preserving data analysis made easy. In Proceedings of the ACM SIGMOD International Conference on Management of Data, 2012. Google Scholar
Digital Library
- J. Reed and B. C. Pierce. Distance makes the types grow stronger: A calculus for differential privacy. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming, ICFP ’10, pages 157–168, 2010. Google Scholar
Digital Library
- A. Roth. The sparse vector technique. http://www.cis.upenn. edu/˜aaroth/courses/slides/Lecture11.pdf, 2011.Google Scholar
- I. Roy, S. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: Security and privacy for MapReduce. In NSDI, 2010. Google Scholar
Digital Library
- M. C. Tschantz, D. Kaynar, and A. Datta. Formal verification of differential privacy for interactive systems (extended abstract). Electron. Notes Theor. Comput. Sci., 276:61–79, Sept. 2011. Google Scholar
Digital Library
- M. Wand. A simple algorithm and proof for type inference. Fundamenta Informaticae, 10:115–122, 1987.Google Scholar
Cross Ref
- L. Xu, K. Chatzikokolakis, and H. Lin. Metrics for Differential Privacy in Concurrent Systems, pages 199–215. 2014.Google Scholar
- D. Zhang and D. Kifer. LightDP: Towards automating differential privacy proofs. CoRR, abs/1607.08228, 2016. Google Scholar
Digital Library
- J. Zhang, X. Xiao, and X. Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In SIGMOD, 2016. Google Scholar
Digital Library
Index Terms
LightDP: towards automating differential privacy proofs
Recommendations
LightDP: towards automating differential privacy proofs
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesThe growing popularity and adoption of differential privacy in academic and industrial settings has resulted in the development of increasingly sophisticated algorithms for releasing information while preserving privacy. Accompanying this phenomenon is ...
Linear dependent types for differential privacy
POPL '13: Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languagesDifferential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the ...
Linear dependent types for differential privacy
POPL '13Differential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the ...







Comments