Abstract
Differential privacy is a mathematical framework for developing statistical computations with provable guarantees of privacy and accuracy. In contrast to the privacy component of differential privacy, which has a clear mathematical and intuitive meaning, the accuracy component of differential privacy does not have a generally accepted definition; accuracy claims of differential privacy algorithms vary from algorithm to algorithm and are not instantiations of a general definition. We identify program discontinuity as a common theme in existing ad hoc definitions and introduce an alternative notion of accuracy parametrized by, what we call, — the of an input x w.r.t. a deterministic computation f and a distance d, is the minimal distance d(x,y) over all y such that f(y)≠ f(x). We show that our notion of accuracy subsumes the definition used in theoretical computer science, and captures known accuracy claims for differential privacy algorithms. In fact, our general notion of accuracy helps us prove better claims in some cases. Next, we study the decidability of accuracy. We first show that accuracy is in general undecidable. Then, we define a non-trivial class of probabilistic computations for which accuracy is decidable (unconditionally, or assuming Schanuel’s conjecture). We implement our decision procedure and experimentally evaluate the effectiveness of our approach for generating proofs or counterexamples of accuracy for common algorithms from the literature.
- Erika Ábrahám and Borzoo Bonakdarpour. 2018. HyperPCTL: A Temporal Logic for Probabilistic Hyperproperties. In Quantitative Evaluation of Systems-15th International Conference, QEST 2018, Beijing, China, September 4-7, 2018, Proceedings (Lecture Notes in Computer Science), Annabelle McIver and András Horváth (Eds.), Vol. 11024. Springer, 20-35. https://doi.org/10.1007/978-3-319-99154-2_2 Google Scholar
Cross Ref
- Aws Albarghouthi and Justin Hsu. 2018. Synthesizing coupling proofs of diferential privacy. PACMPL 2, POPL ( 2018 ), 58 : 1-58 : 30. https://doi.org/10.1145/3158146 Google Scholar
Digital Library
- Gilles Barthe, Rohit Chadha, Vishal Jagannath, A. Prasad Sistla, and Mahesh Viswanathan. 2020a. Deciding Diefrential Privacy for Programs with Finite Inputs and Outputs. In LICS ' 20 : 35th Annual ACM /IEEE Symposium on Logic in Computer Science, Saarbrücken, Germany, July 8-11, 2020, Holger Hermanns, Lijun Zhang, Naoki Kobayashi, and Dale Miller (Eds.). ACM, 141-154. https://doi.org/10.1145/3373718.3394796 Google Scholar
Digital Library
- Gilles Barthe, Rohit Chadha, Vishal Jagannath, A. Prasad Sistla, and Mahesh Viswanathan. 2020b. Deciding Diferential Privacy for Programs with Finite Inputs and Outputs. CORR abs/ 1910.04137 ( 2020 ). arXiv: 1910.04137 http://arxiv.org/ abs/ 1910.04137Google Scholar
- Gilles Barthe, Rohit Chadha, Paul Krogmeier, A. Prasad Sistla, and Mahesh Viswanathan. 2020c. Deciding Accuracy of Diferential Privacy Schemes. CoRR abs/ 2011.06404 ( 2020 ). arXiv: 2011.06404 https://arxiv.org/abs/ 2011.06404Google Scholar
- Gilles Barthe, Thomas Espitau, Luis María Ferrer Fioriti, and Justin Hsu. 2016a. Synthesizing Probabilistic Invariants via Doob's Decomposition. In Computer Aided Verification-28th International Conference, CAV 2016, Toronto, ON, Canada, July 17-23, 2016, Proceedings, Part I (Lecture Notes in Computer Science), Swarat Chaudhuri and Azadeh Farzan (Eds.), Vol. 9779. Springer, 43-61. https://doi.org/10.1007/978-3-319-41528-4_3 Google Scholar
Cross Ref
- Gilles Barthe, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016b. A program logic for union bounds. In International Colloquium on Automata, Languages and Programming (ICALP), Rome, Italy. arXiv:Yes http://arxiv.org/abs/1602.05681Google Scholar
- Gilles Barthe, Marco Gaboardi, Justin Hsu, and Benjamin C. Pierce. 2016c. Programming language techniques for diferential privacy. SIGLOG News 3, 1 ( 2016 ), 34-53. https://dl.acm.org/citation.cfm?id= 2893591Google Scholar
- Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella-Béguelin. 2013. Probabilistic Relational Reasoning for Diferential Privacy. ACM Transactions on Programming Languages and Systems 35, 3 ( 2013 ), 9. http://software.imdea. org/~bkoepf/papers/toplas13.pdfGoogle Scholar
Digital Library
- Raghav Bhaskar, Srivatsan Laxman, Adam D. Smith, and Abhradeep Thakurta. 2010. Discovering frequent patterns in sensitive data. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, July 25-28, 2010, Bharat Rao, Balaji Krishnapuram, Andrew Tomkins, and Qiang Yang (Eds.). ACM, 503-512. https://doi.org/10.1145/1835804.1835869 Google Scholar
Digital Library
- Benjamin Bichsel, Timon Gehr, Dana Drachsler-Cohen, Petar Tsankov, and Martin T. Vechev. 2018. DP-Finder: Finding Diferential Privacy Violations by Sampling and Optimization. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15-19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 508-524. https://doi.org/10.1145/3243734.3243863 Google Scholar
Digital Library
- Avrim Blum, Katrina Ligett, and Aaron Roth. 2013. A learning theory approach to noninteractive database privacy. J. ACM 60, 2 ( 2013 ), 12 : 1-12 : 25. https://doi.org/10.1145/2450142.2450148 Google Scholar
Digital Library
- Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In Computer Aided Verification-25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. Proceedings (Lecture Notes in Computer Science), Natasha Sharygina and Helmut Veith (Eds.), Vol. 8044. Springer, 511-526. https: //doi.org/10.1007/978-3-642-39799-8_34 Google Scholar
Cross Ref
- T.-H. Hubert Chan, Elaine Shi, and Dawn Song. 2011. Private and continual release of statistics. ACM Transactions on Information and System Security 14, 3 ( 2011 ), 26. http://eprint.iacr.org/ 2010 /076.pdfGoogle Scholar
- Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic analysis of qualitative and quantitative termination problems for afine probabilistic programs. In Proceedings of the 43rd Annual ACM SIGPLANSIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20-22, 2016, Rastislav Bodík and Rupak Majumdar (Eds.). ACM, 327-342. https://doi.org/10.1145/2837614.2837639 Google Scholar
Digital Library
- Michael R. Clarkson and Fred B. Schneider. 2010. Hyperproperties. J. Comput. Secur. 18, 6 ( 2010 ), 1157-1210. https: //doi.org/10.3233/JCS-2009-0393 Google Scholar
Cross Ref
- Patrick Cousot and Michael Monerau. 2012. Probabilistic Abstract Interpretation. In Programming Languages and Systems-21st European Symposium on Programming, ESOP 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, March 24-April 1, 2012. Proceedings (Lecture Notes in Computer Science), Helmut Seidl (Ed.), Vol. 7211. Springer, 169-193. https://doi.org/10.1007/978-3-642-28869-2_9 Google Scholar
Digital Library
- Rayna Dimitrova, Bernd Finkbeiner, and Hazem Torfah. 2020. Probabilistic Hyperproperties of Markov Decision Processes. In Automated Technology for Verification and Analysis, Dang Van Hung and Oleg Sokolsky (Eds.). Springer International Publishing, Cham, 484-500.Google Scholar
- Zeyu Ding, Yuxin Wang, Guanhong Wang, Danfeng Zhang, and Daniel Kifer. 2018. Detecting Violations of Diferential Privacy. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15-19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 475-489. https://doi.org/10.1145/3243734.3243818 Google Scholar
Digital Library
- Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In IACR Theory of Cryptography Conference (TCC), New York, New York. 265-284. http://dx.doi.org/10.1007/11681878_14 Google Scholar
Digital Library
- Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Diferential Privacy. Foundations and Trends in Theoretical Computer Science 9, 3-4 ( 2014 ), 211-407. http://dx.doi.org/10.1561/0400000042 Google Scholar
Digital Library
- Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, and Benjamin C Pierce. 2013. Linear dependent types for diferential privacy. In ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), Rome, Italy. 357-370. http://dl.acm.org/citation.cfm?id= 2429113Google Scholar
Digital Library
- Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, and Kunal Talwar. 2010. Diferentially private combinatorial optimization. In ACM-SIAM Symposium on Discrete Algorithms (SODA), Austin, Texas. 1106-1125. http://arxiv.org/pdf/ 0903.4510v2Google Scholar
Cross Ref
- Matthias Heizmann, Jochen Hoenicke, and Andreas Podelski. 2009. Refinement of Trace Abstraction. In Static Analysis, 16th International Symposium, SAS 2009, Los Angeles, CA, USA, August 9-11, 2009. Proceedings (Lecture Notes in Computer Science), Jens Palsberg and Zhendong Su (Eds.), Vol. 5673. Springer, 69-85. https://doi.org/10.1007/978-3-642-03237-0_7 Google Scholar
Digital Library
- Benjamin Lucien Kaminski. 2019. Advanced weakest precondition calculi for probabilistic programs. Ph.D. Dissertation. RWTH Aachen University, Germany. http://publications.rwth-aachen.de/record/755408Google Scholar
- Joost-Pieter Katoen. 2016. The Probabilistic Model Checking Landscape. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, LICS ' 16, New York, NY, USA, July 5-8, 2016, Martin Grohe, Eric Koskinen, and Natarajan Shankar (Eds.). ACM, 31-45. https://doi.org/10.1145/2933575.2934574 Google Scholar
Digital Library
- Dexter Kozen. 1985. A Probabilistic PDL. J. Comput. System Sci. 30, 2 ( 1985 ), 162-178.Google Scholar
Cross Ref
- Satoshi Kura, Natsuki Urabe, and Ichiro Hasuo. 2019. Tail Probabilities for Randomized Program Runtimes via Martingales for Higher Moments. In Tools and Algorithms for the Construction and Analysis of Systems-25th International Conference, TACAS 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings, Part II (Lecture Notes in Computer Science), Tomás Vojnar and Lijun Zhang (Eds.), Vol. 11428. Springer, 135-153. https://doi.org/10.1007/978-3-030-17465-1_8 Google Scholar
Cross Ref
- Marta Kwiatkowska, Gethin Norman, and David Parker. 2010. Advances and challenges of probabilistic model checking. In 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 1691-1698.Google Scholar
Cross Ref
- Serge Lang. 1966. Introduction to Transcendental Numbers. Addison-Wesley.Google Scholar
- Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, and Steven Z. Wu. 2017. Accuracy First: Selecting a Diferential Privacy Level for Accuracy Constrained ERM. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2566-2576.Google Scholar
- Angus MacIntyre and Alex J. Wilkie. 1996. On the decidability of the real exponential field. In Kreiseliana. About and Around Georg Kreisel, Piergiorgio Odifreddi (Ed.). A.K. Peters, 441-467.Google Scholar
- Scott McCallum and Volker Weispfenning. 2012. Deciding polynomial-transcendental problems. Journal of Symbolic Computation 47, 1 ( 2012 ), 16-31.Google Scholar
Digital Library
- Frank McSherry and Kunal Talwar. 2007. Mechanism Design via Diferential Privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2007 ), October 20-23, 2007, Providence, RI, USA, Proceedings. IEEE Computer Society, 94-103. https://doi.org/10.1109/FOCS. 2007.41 Google Scholar
Digital Library
- David Monniaux. 2000. Abstract Interpretation of Probabilistic Semantics. In Static Analysis, 7th International Symposium, SAS 2000, Santa Barbara, CA, USA, June 29-July 1, 2000, Proceedings (Lecture Notes in Computer Science), Jens Palsberg (Ed.), Vol. 1824. Springer, 322-339. https://doi.org/10.1007/978-3-540-45099-3_17 Google Scholar
Cross Ref
- Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. ACM Transactions on Programming Languages and Systems 18, 3 ( 1996 ), 325-353.Google Scholar
Digital Library
- Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press.Google Scholar
Digital Library
- Jason Reed and Benjamin C. Pierce. 2010. Distance Makes the Types Grow Stronger: A Calculus for Diferential Privacy. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming (ICFP '10). Association for Computing Machinery, New York, NY, USA, 157-168. https://doi.org/10.1145/1863543.1863568 Google Scholar
Digital Library
- Sriram Sankaranarayanan. 2020. Quantitative Analysis of Programs with Probabilities and Concentration of Measure Inequalities. In Foundations of Probabilistic Programming, Gilles Barthe, Joost-Pieter Katoen, and Alexandra Silva (Eds.). Cambridge University Press, TBA.Google Scholar
- Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '13, Seattle, WA, USA, June 16-19, 2013, Hans-Juergen Boehm and Cormac Flanagan (Eds.). ACM, 447-458. https://doi.org/10.1145/2491956.2462179 Google Scholar
Digital Library
- Calvin Smith, Justin Hsu, and Aws Albarghouthi. 2019. Trace abstraction modulo probability. PACMPL 3, POPL ( 2019 ), 39 : 1-39 : 31. https://dl.acm.org/citation.cfm?id= 3290352Google Scholar
- A. Tarski. 1951. A decision method for Elementary Algebra and Geometry. University of California Press.Google Scholar
- Elisabet Lobo Vesga, Alejandro Russo, and Marco Gaboardi. 2019. A Programming Framework for Diferential Privacy with Accuracy Concentration Bounds. CoRR abs/ 1909.07918 ( 2019 ). arXiv: 1909.07918 http://arxiv.org/abs/ 1909.07918Google Scholar
- Di Wang, Jan Hofmann, and Thomas W. Reps. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018, Philadelphia, PA, USA, June 18-22, 2018, Jefrey S. Foster and Dan Grossman (Eds.). ACM, 513-528. https: //doi.org/10.1145/3192366.3192408 Google Scholar
Digital Library
- Di Wang, Jan Hofmann, and Thomas W. Reps. 2020. Tail Bound Analysis for Probabilistic Programs via Central Moments. CoRR abs/ 2001.10150 ( 2020 ). arXiv: 2001.10150 https://arxiv.org/abs/ 2001.10150Google Scholar
- Yu Wang, Siddhartha Nalluri, Borzoo Bonakdarpour, and Miroslav Pajic. 2019. Statistical Model Checking for Probabilistic Hyperproperties. CoRR abs/ 1902.04111 ( 2019 ). arXiv: 1902.04111 http://arxiv.org/abs/ 1902.04111Google Scholar
- Danfeng Zhang and Daniel Kifer. 2017. LightDP: towards automating diferential privacy proofs. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18-20, 2017, Giuseppe Castagna and Andrew D. Gordon (Eds.). ACM, 888-901. http://dl.acm.org/citation.cfm?id= 3009884Google Scholar
Digital Library
Index Terms
Deciding accuracy of differential privacy schemes
Recommendations
A Novel Differential Privacy Approach that Enhances Classification Accuracy
C3S2E '16: Proceedings of the Ninth International C* Conference on Computer Science & Software EngineeringIn the recent past, there has been a tremendous increase of large repositories of data, examples being in healthcare data, consumer data from retailers, and airline passenger data. These data are continually being shared with interested parties, either ...
Differential Privacy as a Mutual Information Constraint
CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications SecurityDifferential privacy is a precise mathematical constraint meant to ensure privacy of individual pieces of information in a database even while queries are being answered about the aggregate. Intuitively, one must come to terms with what differential ...
Sensitive Disclosures under Differential Privacy Guarantees
BIGDATACONGRESS '15: Proceedings of the 2015 IEEE International Congress on Big DataNon-independent reasoning (NIR) refers to learning the information of one record from other records, under the assumption that these records share the same underlying distribution. Accurate NIR could disclose private information of an individual. An ...






Comments