skip to main content
research-article
Open Access
Artifacts Available
Artifacts Evaluated & Functional

Duet: an expressive higher-order language and linear type system for statically enforcing differential privacy

Published:10 October 2019Publication History
Skip Abstract Section

Abstract

During the past decade, differential privacy has become the gold standard for protecting the privacy of individuals. However, verifying that a particular program provides differential privacy often remains a manual task to be completed by an expert in the field. Language-based techniques have been proposed for fully automating proofs of differential privacy via type system design, however these results have lagged behind advances in differentially-private algorithms, leaving a noticeable gap in programs which can be automatically verified while also providing state-of-the-art bounds on privacy.

We propose Duet, an expressive higher-order language, linear type system and tool for automatically verifying differential privacy of general-purpose higher-order programs. In addition to general purpose programming, Duet supports encoding machine learning algorithms such as stochastic gradient descent, as well as common auxiliary data analysis tasks such as clipping, normalization and hyperparameter tuning - each of which are particularly challenging to encode in a statically verified differential privacy framework.

We present a core design of the Duet language and linear type system, and complete key proofs about privacy for well-typed programs. We then show how to extend Duet to support realistic machine learning applications and recent variants of differential privacy which result in improved accuracy for many practical differentially private algorithms. Finally, we implement several differentially private machine learning algorithms in Duet which have never before been automatically verified by a language-based tool, and we present experimental results which demonstrate the benefits of Duet's language design in terms of accuracy of trained machine learning models.

Skip Supplemental Material Section

Supplemental Material

a172-near

Presentation at OOPSLA '19

References

  1. 2016. Apple previews iOS 10, the biggest iOS release ever. http://www.apple.com/newsroom/2016/06/apple-previews-ios10-biggest-ios-release-ever.html .Google ScholarGoogle Scholar
  2. 2019. scikit-learn: Standardization, or mean removal and variance scaling. https://scikit-learn.org/stable/modules/ preprocessing.html#preprocessing-scalerGoogle ScholarGoogle Scholar
  3. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 308–318.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Arthur Azevedo de Amorim, Marco Gaboardi, Justin Hsu, Shin-ya Katsumata, and Ikram Cherigui. 2017. A semantic account of metric preservation. In POPL, Vol. 52. ACM, 545–556.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Andrew Barber. 1996. Dual Intuitionistic Linear Logic. Technical Report ECS-LFCS-96-347. University of Edinburgh.Google ScholarGoogle Scholar
  6. Gilles Barthe, Gian Pietro Farina, Marco Gaboardi, Emilio Jesus Gallego Arias, Andy Gordon, Justin Hsu, and Pierre-Yves Strub. 2016a. Differentially Private Bayesian Programming. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, New York, NY, USA, 68–79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016b. Advanced probabilistic couplings for differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 55–67.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, and Pierre-Yves Strub. 2015. HigherOrder Approximate Relational Refinement Types for Mechanism Design and Differential Privacy. In POPL. ACM, 55–68.Google ScholarGoogle Scholar
  9. Gilles Barthe, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016c. Proving differential privacy via probabilistic couplings. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science. ACM, 749–758.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gilles Barthe, Marco Gaboardi, Justin Hsu, and Benjamin Pierce. 2016d. Programming language techniques for differential privacy. ACM SIGLOG News 3, 1 (2016), 34–53.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella-Béguelin. 2013. Probabilistic relational reasoning for differential privacy. ACM Transactions on Programming Languages and Systems (TOPLAS) 35, 3 (2013), 9.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gilles Barthe and Federico Olmedo. 2013. Beyond differential privacy: Composition theorems and relational logic for fdivergences between probabilistic programs. In International Colloquium on Automata, Languages, and Programming. Springer, 49–60.Google ScholarGoogle Scholar
  13. Raef Bassily, Adam Smith, and Abhradeep Thakurta. 2014a. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Foundations of Computer Science (FOCS), 2014 IEEE 55th Annual Symposium on. IEEE, 464–473.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Raef Bassily, Adam Smith, and Abhradeep Thakurta. 2014b. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Foundations of Computer Science (FOCS), 2014 IEEE 55th Annual Symposium on. IEEE, 464–473.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mark Bun, Cynthia Dwork, Guy N Rothblum, and Thomas Steinke. 2018. Composable and versatile privacy via truncated CDP. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing. ACM, 74–86.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mark Bun and Thomas Steinke. 2016. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography Conference. Springer, 635–658.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. 2011. Differentially private empirical risk minimization. Journal of Machine Learning Research 12, Mar (2011), 1069–1109.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kamalika Chaudhuri and Staal A Vinterbo. 2013. A stability-based validation procedure for differentially private machine learning. In Advances in Neural Information Processing Systems. 2652–2660.Google ScholarGoogle Scholar
  19. Ezgi Çiçek, Weihao Qu, Gilles Barthe, Marco Gaboardi, and Deepak Garg. 2018. Bidirectional Type Checking for Relational Properties. CoRR abs/1812.05067 (2018). arXiv: 1812.05067 http://arxiv.org/abs/1812.05067Google ScholarGoogle Scholar
  20. Arthur Azevedo De Amorim, Marco Gaboardi, Emilio Jesús Gallego Arias, and Justin Hsu. 2014. Really Natural Linear Indexed Type Checking. In Proceedings of the 26nd 2014 International Symposium on Implementation and Application of Functional Languages. ACM, 5.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Arthur Azevedo de Amorim, Marco Gaboardi, Justin Hsu, and Shin-ya Katsumata. 2018. Metric Semantics for Probabilistic Relational Reasoning. CoRR abs/1807.05091 (2018). arXiv: 1807.05091 http://arxiv.org/abs/1807.05091Google ScholarGoogle Scholar
  22. Cynthia Dwork. 2006. Differential Privacy. In Automata, Languages and Programming, Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener (Eds.). Lecture Notes in Computer Science, Vol. 4052. Springer Berlin Heidelberg, 1–12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Cynthia Dwork and Jing Lei. 2009. Differential privacy and robust statistics. In Proceedings of the forty-first annual ACM symposium on Theory of computing. ACM, 371–380.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography Conference. Springer, 265–284.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science 9, 3–4 (2014), 211–407.Google ScholarGoogle Scholar
  26. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS ’15). ACM, New York, NY, USA, 1322–1333. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Arik Friedman, Shlomo Berkovsky, and Mohamed Ali Kaafar. 2016. A differential privacy framework for matrix factorization recommender systems. User Modeling and User-Adapted Interaction 26, 5 (2016), 425–458.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, and Benjamin C Pierce. 2013. Linear dependent types for differential privacy. In POPL, Vol. 48. ACM, 357–370.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jean-Yves Girard. 1987. Linear Logic. Theor. Comput. Sci. 50, 1 (Jan. 1987), 1–102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Samuel Haney, Ashwin Machanavajjhala, John M Abowd, Matthew Graham, Mark Kutzbach, and Lars Vilhuber. 2017. Utility cost of formal privacy for releasing national employer-employee statistics. In Proceedings of the 2017 ACM International Conference on Management of Data. ACM, 1339–1354.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Noah Johnson, Joseph P Near, and Dawn Song. 2018. Towards practical differential privacy for SQL queries. Proceedings of the VLDB Endowment 11, 5 (2018), 526–539.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Noah M. Johnson, Joseph P. Near, and Dawn Xiaodong Song. 2017. Towards Practical Differential Privacy for SQL Queries. CoRR abs/1706.09479 (2017). http://arxiv.org/abs/1706.09479Google ScholarGoogle Scholar
  33. Ashwin Machanavajjhala, Daniel Kifer, John Abowd, Johannes Gehrke, and Lars Vilhuber. 2008. Privacy: Theory meets practice on the map. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering. IEEE Computer Society, 277–286.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Frank McSherry and Kunal Talwar. 2007. Mechanism design via differential privacy. In Foundations of Computer Science, 2007. FOCS’07. 48th Annual IEEE Symposium on. IEEE, 94–103.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Frank D McSherry. 2009a. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. ACM, 19–30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Frank D McSherry. 2009b. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. ACM, 19–30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Ilya Mironov. 2017. Renyi differential privacy. In Computer Security Foundations Symposium (CSF), 2017 IEEE 30th. IEEE, 263–275.Google ScholarGoogle ScholarCross RefCross Ref
  38. Prashanth Mohan, Abhradeep Thakurta, Elaine Shi, Dawn Song, and David Culler. 2012. GUPT: privacy preserving data analysis made easy. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. ACM, 349–360.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Aleksandar Nanevski, Frank Pfenning, and Brigitte Pientka. 2008. Contextual Modal Type Theory. ACM Trans. Comput. Logic 9, 3, Article 23 (June 2008), 49 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Arjun Narayan and Andreas Haeberlen. 2012. DJoin: differentially private join queries over distributed databases. In Presented as part of the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 12). 149–162.Google ScholarGoogle Scholar
  41. Joseph P. Near, David Darais, Chike Abuah, Tim Stevens, Pranav Gaddamadugu, Lun Wang, Neel Somani, Mu Zhang, Nikhil Sharma, Alex Shan, and Dawn Song. 2019. Duet: An Expressive Higher-order Language and Linear Type System for Statically Enforcing Differential Privacy. CoRR abs/1909.02481 (2019). https://arxiv.org/abs/1909.02481Google ScholarGoogle Scholar
  42. Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. 2016. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755 (2016).Google ScholarGoogle Scholar
  43. Davide Proserpio, Sharon Goldberg, and Frank McSherry. 2014. Calibrating data to sensitivity in private data analysis: A platform for differentially-private analysis of weighted datasets. PVLDB 7, 8 (2014), 637–648.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Jason Reed and Benjamin C Pierce. 2010. Distance makes the types grow stronger: a calculus for differential privacy. ICFP 45, 9 (2010), 157–168.Google ScholarGoogle Scholar
  45. Indrajit Roy, Srinath TV Setty, Ann Kilzer, Vitaly Shmatikov, and Emmett Witchel. 2010. Airavat: Security and Privacy for MapReduce.. In NSDI, Vol. 10. 297–312.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Tetsuya Sato. 2016. Approximate relational Hoare logic for continuous random samplings. Electronic Notes in Theoretical Computer Science 325 (2016), 277–298.Google ScholarGoogle ScholarCross RefCross Ref
  47. Tetsuya Sato, Gilles Barthe, Marco Gaboardi, Justin Hsu, and Shin-ya Katsumata. 2019. Approximate span liftings: Compositional semantics for relaxations of differential privacy. In 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS). IEEE, 1–14.Google ScholarGoogle ScholarCross RefCross Ref
  48. R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP). 3–18. Google ScholarGoogle ScholarCross RefCross Ref
  49. Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. 2013. Stochastic gradient descent with differentially private updates. In Global Conference on Signal and Information Processing (GlobalSIP), 2013 IEEE. IEEE, 245–248.Google ScholarGoogle ScholarCross RefCross Ref
  50. Kunal Talwar, Abhradeep Guha Thakurta, and Li Zhang. 2015. Nearly optimal private lasso. In Advances in Neural Information Processing Systems. 3025–3033.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Yu-Xiang Wang, Borja Balle, and Shiva Kasiviswanathan. 2018. Subsampled Rényi Differential Privacy and Analytical Moments Accountant. CoRR abs/1808.00087 (2018). arXiv: 1808.00087 http://arxiv.org/abs/1808.00087Google ScholarGoogle Scholar
  52. X. Wu, M. Fredrikson, S. Jha, and J. F. Naughton. 2016. A Methodology for Formalizing Model-Inversion Attacks. In 2016 IEEE 29th Computer Security Foundations Symposium (CSF). 355–370. Google ScholarGoogle ScholarCross RefCross Ref
  53. Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey Naughton. 2017. Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD ’17). ACM, New York, NY, USA, 1307–1322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Danfeng Zhang and Daniel Kifer. 2017. LightDP: Towards automating differential privacy proofs. In POPL, Vol. 52. ACM, 888–901.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Hengchu Zhang, Edo Roth, Andreas Haeberlen, Benjamin C. Pierce, and Aaron Roth. 2019. Fuzzi: A Three-Level Logic for Differential Privacy. Accepted for publication in PACMPL / ICFP 2019.Google ScholarGoogle Scholar

Index Terms

  1. Duet: an expressive higher-order language and linear type system for statically enforcing differential privacy

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!