Tight on Budget?: Tight Bounds for r-Fold Approximate Differential Privacy

ABSTRACT

Many applications, such as anonymous communication systems, privacy-enhancing database queries, or privacy-enhancing machine-learning methods, require robust guarantees under thousands and sometimes millions of observations. The notion of r-fold approximate differential privacy (ADP) offers a well-established framework with a precise characterization of the degree of privacy after r observations of an attacker. However, existing bounds for r-fold ADP are loose and, if used for estimating the required degree of noise for an application, can lead to over-cautious choices for perturbation randomness and thus to suboptimal utility or overly high costs. We present a numerical and widely applicable method for capturing the privacy loss of differentially private mechanisms under composition, which we call privacy buckets. With privacy buckets we compute provable upper and lower bounds for ADP for a given number of observations. We compare our bounds with state-of-the-art bounds for r-fold ADP, including Kairouz, Oh, and Viswanath's composition theorem (KOV), concentrated differential privacy and the moments accountant. While KOV proved optimal bounds for heterogeneous adaptive k-fold composition, we show that for concrete sequences of mechanisms tighter bounds can be derived by taking the mechanisms' structure into account. We compare previous bounds for the Laplace mechanism, the Gauss mechanism, for a timing leakage reduction mechanism, and for the stochastic gradient descent and we significantly improve over their results (except that we match the KOV bound for the Laplace mechanism, for which it seems tight). Our lower bounds almost meet our upper bounds, showing that no significantly tighter bounds are possible.

References

1. Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, 308--318.
2. Mark Bun and Thomas Steinke. 2016. Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds Theory of Cryptography (TCC). Springer, 635--658.
3. Cynthia Dwork. 2006. Differential Privacy. In Automata, Languages and Programming, 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10--14, 2006, Proceedings, Part II. Springer, 1--12.
4. Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. 2006. Our Data, Ourselves: Privacy Via Distributed Noise Generation Advances in Cryptology - EUROCRYPT 2006. Springer, 486--503.
5. Cynthia Dwork, Moni Naor, Toniann Pitassi, and Guy N. Rothblum. 2010 a. Differential Privacy Under Continual Observation Proceedings of the 42th Annual ACM Symposium on Theory of Computing (STOC). ACM, 715--724.
6. Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Foundations and Trends® in Theoretical Computer Science, Vol. 9, 3--4 (2014), 211--407.
7. Cynthia Dwork and Guy N. Rothblum. 2016. Concentrated Differential Privacy. CoRR Vol. abs/1603.01887 (2016). showeprint{arXiv}1603.01887
8. Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. 2010 b. Boosting and differential privacy. In 2010 51st Annual IEEE Symposium on Foundations of Computer Science. IEEE, 51--60.
9. Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. 2014. Analyze Gauss: Optimal Bounds for Privacy-preserving Principal Component Analysis. In Proceedings of the 46th Annual ACM Symposium on Theory of Computing (STOC). ACM, 11--20.
10. Q. Geng and P. Viswanath. 2014. The optimal mechanism in differential privacy. 2014 IEEE International Symposium on Information Theory (ISIT). IEEE, 2371--2375.
11. Michaela Götz, Ashwin Machanavajjhala, Guozhang Wang, Xiaokui Xiao, and Johannes Gehrke. 2009. Privacy in Search Logs. CoRR Vol. abs/0904.0682 (2009). showeprint{arxiv}0904.0682
12. M. Hardt and G. N. Rothblum. 2010. A Multiplicative Weights Mechanism for Privacy-Preserving Data Analysis 2010 51st Annual IEEE Symposium on Foundations of Computer Science. Springer, 61--70.
13. T-H. Hubert Chan, Elaine Shi, and Dawn Song. 2010. Private and Continual Release of Statistics. In Automata, Languages and Programming. ICALP 2010. Springer, 405--417.
14. Peter Kairouz, Sewoong Oh, and Pramod Viswanath. 2017. The composition theorem for differential privacy. IEEE Transactions on Information Theory Vol. 63, 6 (2017), 4037--4049.
15. K. Kalantari, L. Sankar, and A. D. Sarwate. 2016. Optimal differential privacy mechanisms under Hamming distortion for structured source classes. In 2016 IEEE International Symposium on Information Theory (ISIT). IEEE, 2069--2073.
16. A. Machanavajjhala, D. Kifer, J. Abowd, J. Gehrke, and L. Vilhuber. 2008. Privacy: Theory meets Practice on the Map. In 2008 IEEE 24th International Conference on Data Engineering. IEEE, 277--286.
17. Sebastian Meiser. 2018. Approximate and Probabilistic Differential Privacy Definitions. (2018).
18. Sebastian Meiser and Esfandiar Mohammadi. {n. d.}. Implementation of privacy buckets (improved implementation by David Sommer), including FAQ. https://github.com/dabingo/privacybuckets. (. {n. d.}).
19. Sebastian Meiser and Esfandiar Mohammadi. 2017. Tight on Budget? Tight Bounds for r-Fold Approximate Differential Privacy (Technical Report). Cryptology ePrint Archive, Report 2017/1034. (2017).
20. Ilya Mironov. 2017. Renyi Differential Privacy. In Proceedings of the 30th IEEE Computer Security Foundations Symposium (CSF). IEEE, 263--275.
21. Jack Murtagh and Salil Vadhan. 2016. The Complexity of Computing the Optimal Composition of Differential Privacy Proceedings of the 13th International Conference on Theory of Cryptography (TCC). Springer, 157--175.
22. Jack Murtagh and Salil P. Vadhan. 2015. The Complexity of Computing the Optimal Composition of Differential Privacy. CoRR Vol. abs/1507.03113 (2015). {arXiv}1507.03113
23. Ryan M Rogers, Aaron Roth, Jonathan Ullman, and Salil Vadhan. 2016. Privacy Odometers and Filters: Pay-as-you-Go Composition Advances in Neural Information Processing Systems 29. Curran Associates, Inc., 1921--1929.
24. David Sommer, Aritra Dhar, Luka Malitsa, Esfandiar Mohammadi, Daniel Ronzani, and Srdjan Capkun. 2017. Anonymous Communication for Messengers via "Forced" Participation. Technical report, available under https://eprint.iacr.org/2017/191. (2017).
25. J. Tang, A. Korolova, X. Bai, X. Wang, and X. Wang. 2017. Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12. ArXiv e-prints (2017). {arxiv}1709.02753
26. Jelle van den Hooff, David Lazar, Matei Zaharia, and Nickolai Zeldovich. 2015. Vuvuzela: Scalable Private Messaging Resistant to Traffic Analysis Proceedings of the 25th ACM Symposium on Operating Systems Principles (SOSP 2015). ACM, 137--152.

Index Terms

• Security and privacy

  • Human and societal aspects of security and privacy

    • Usability in security and privacy