skip to main content

Perfectly parallel fairness certification of neural networks

Published:13 November 2020Publication History
Skip Abstract Section

Abstract

Recently, there is growing concern that machine-learned software, which currently assists or even automates decision making, reproduces, and in the worst case reinforces, bias present in the training data. The development of tools and techniques for certifying fairness of this software or describing its biases is, therefore, critical. In this paper, we propose a perfectly parallel static analysis for certifying fairness of feed-forward neural networks used for classification of tabular data. When certification succeeds, our approach provides definite guarantees, otherwise, it describes and quantifies the biased input space regions. We design the analysis to be sound, in practice also exact, and configurable in terms of scalability and precision, thereby enabling pay-as-you-go certification. We implement our approach in an open-source tool called Libra and demonstrate its effectiveness on neural networks trained on popular datasets.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

This is the talk video presenting the published paper.

References

  1. Aws Albarghouthi, Loris D'Antoni, and Samuel Drews. 2017a. Repairing Decision-Making Programs Under Uncertainty. In CAV. 181-200. https://doi.org/10.1007/978-3-319-63387-9_9 Google ScholarGoogle ScholarCross RefCross Ref
  2. Aws Albarghouthi, Loris D'Antoni, Samuel Drews, and Aditya V. Nori. 2017b. FairSquare: Probabilistic Verification of Program Fairness. PACMPL 1, OOPSLA ( 2017 ), 80 : 1-80 : 30. https://doi.org/10.1145/3133904 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Aws Albarghouthi and Samuel Vinitsky. 2019. Fairness-Aware Programming. In FAT*. 211-219. https://doi.org/10.1145/ 3287560.3287588 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Anish Athalye, Nicholas Carlini, and David A. Wagner. 2018. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In ICML (PMLR), Vol. 80. PMLR, 274-283.Google ScholarGoogle Scholar
  5. Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. California Law Review 104, 3 ( 2016 ), 671-732.Google ScholarGoogle Scholar
  6. Clark W. Barrett and Cesare Tinelli. 2018. Satisfiability Modulo Theories. In Handbook of Model Checking. Springer, 305-343.Google ScholarGoogle Scholar
  7. Alexander I. Barvinok. 1994. A Polynomial Time Algorithm for Counting Integral Points in Polyhedra When the Dimension is Fixed. Mathematics of Operations Research 19, 4 ( 1994 ), 769-779. https://doi.org/10.1287/moor.19.4. 769 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Osbert Bastani, Xin Zhang, and Armando Solar-Lezama. 2019. Probabilistic Verification of Fairness Properties via Concentration. PACMPL 3, OOPSLA ( 2019 ), 118 : 1-118 : 27.Google ScholarGoogle Scholar
  9. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In FAT (PMLR), Vol. 81. PMLR, 77-91.Google ScholarGoogle Scholar
  10. Nicholas Carlini and David A. Wagner. 2016. Defensive Distillation is Not Robust to Adversarial Examples. CoRR abs/1607.04311 ( 2016 ).Google ScholarGoogle Scholar
  11. Nicholas Carlini and David A. Wagner. 2017a. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. In [email protected]. ACM, 3-14.Google ScholarGoogle Scholar
  12. Nicholas Carlini and David A. Wagner. 2017b. Towards Evaluating the Robustness of Neural Networks. In S &P. IEEE Computer Society, 39-57.Google ScholarGoogle Scholar
  13. Swarat Chaudhuri, Sumit Gulwani, and Roberto Lublinerman. 2012. Continuity and Robustness of Programs. Commun. ACM 55, 8 ( 2012 ), 107-115. https://doi.org/10.1145/2240236.2240262 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Liqian Chen, Antoine Miné, and Patrick Cousot. 2008. A Sound Floating-Point Polyhedra Abstract Domain. In APLAS. 3-18. https://doi.org/10.1007/978-3-540-89330-1_2 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. In AAAI. 7801-7808. https://doi.org/10.1609/aaai.v33i01. 33017801 Google ScholarGoogle ScholarCross RefCross Ref
  16. Lori A. Clarke. 1976. A System to Generate Test Data and Symbolically Execute Programs. TSE 2 ( 1976 ), 215-222. Issue 3.Google ScholarGoogle Scholar
  17. Cory Cornelius. 2019. The Eficacy of SHIELD Under Diferent Threat Models. CoRR abs/ 1902.00541 ( 2019 ).Google ScholarGoogle Scholar
  18. Patrick Cousot. 2002. Constructive Design of a Hierarchy of Semantics of a Transition System by Abstract Interpretation. Theoretical Computer Science 277, 1-2 ( 2002 ), 47-103. https://doi.org/10.1016/S0304-3975 ( 00 ) 00313-3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Patrick Cousot and Radhia Cousot. 1976. Static Determination of Dynamic Properties of Programs. In Second International Symposium on Programming. 106-130.Google ScholarGoogle Scholar
  20. Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In POPL. 238-252. https://doi.org/10.1145/512950.512973 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Patrick Cousot and Radhia Cousot. 1979. Systematic Design of Program Analysis Frameworks. In POPL. 269-282. https: //doi.org/10.1145/567752.567778 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Patrick Cousot and Nicolas Halbwachs. 1978. Automatic Discovery of Linear Restraints Among Variables of a Program. In POPL. 84-96. https://doi.org/10.1145/512760.512770 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Anupam Datta, Matthew Fredrikson, Gihyuk Ko, Piotr Mardziel, and Shayak Sen. 2017. Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programs. In CCS. 1193-1210. https://doi.org/10.1145/3133956.3134097 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Logan Engstrom, Andrew Ilyas, and Anish Athalye. 2018. Evaluating and Understanding the Robustness of Adversarial Logit Pairing. CoRR abs/ 1807.10272 ( 2018 ).Google ScholarGoogle Scholar
  25. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In KDD. ACM, 259-268.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness Testing: Testing Software for Discrimination. In FSE. 498-510. https://doi.org/10.1145/3106237.3106277 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev. 2018. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In S & P. 3-18. https: //doi.org/10.1109/SP. 2018.00058 Google ScholarGoogle ScholarCross RefCross Ref
  28. Khalil Ghorbal, Eric Goubault, and Sylvie Putot. 2009. The Zonotope Abstract Domain Taylor1+. In CAV. 627-633. https: //doi.org/10.1007/978-3-642-02658-4_47 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed Automated Random Testing. In PLDI. ACM, 213-223.Google ScholarGoogle Scholar
  30. Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2016. Deep Learning. MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In ICLR. http://arxiv.org/abs/1412.6572Google ScholarGoogle Scholar
  32. Divya Gopinath, Kaiyuan Wang, Mengshi Zhang, Corina S. Pasareanu, and Sarfraz Khurshid. 2018. Symbolic Execution for Deep Neural Networks. CoRR abs/ 1807.10439 ( 2018 ).Google ScholarGoogle Scholar
  33. Eric Goubault and Sylvie Putot. 2013. Robustness Analysis of Finite Precision Implementations. In APLAS. 50-57. https: //doi.org/10.1007/978-3-319-03542-0_4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Nina Grgić-Hlača, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. 2016. The Case for Process Fairness in Learning: Feature Selection for Fair Decision Making. In NIPS 2016 ML and the Law.Google ScholarGoogle Scholar
  35. Cheng Guo and Felix Berkhahn. 2016. Entity Embeddings of Categorical Variables. CoRR abs/1604.06737 ( 2016 ).Google ScholarGoogle Scholar
  36. Boris Hanin and David Rolnick. 2019. Deep ReLU Networks Have Surprisingly Few Activation Patterns. In NIPS. Curran Associates, Inc., 359-368. http://papers.nips.cc/paper/8328-deep-relu-networks-have-surprisingly-few-activationpatterns.pdfGoogle ScholarGoogle Scholar
  37. Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NIPS. 3315-3323.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvári. 2015. Learning with a Strong Adversary. CoRR abs/1511.03034 ( 2015 ). http://arxiv.org/abs/1511.03034Google ScholarGoogle Scholar
  39. Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety Verification of Deep Neural Networks. In CAV. 3-29. https://doi.org/10.1007/978-3-319-63387-9_1 Google ScholarGoogle ScholarCross RefCross Ref
  40. Bertrand Jeannet and Antoine Miné. 2009. APRON: A Library of Numerical Abstract Domains for Static Analysis. In CAV. 661-667. https://doi.org/10.1007/978-3-642-02658-4_52 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Guy Katz, Clark W. Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Eficient SMT Solver for Verifying Deep Neural Networks. In CAV. 97-117. https://doi.org/10.1007/978-3-319-63387-9_5 Google ScholarGoogle ScholarCross RefCross Ref
  42. Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. In CHI. ACM, 3819-3828.Google ScholarGoogle Scholar
  43. Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding Discrimination Through Causal Reasoning. In NIPS. 656-666.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. James C. King. 1976. Symbolic Execution and Program Testing. CACM 19 ( 1976 ), 385-394. Issue 7.Google ScholarGoogle Scholar
  45. Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In NIPS. 4069-4079.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Jef Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.Google ScholarGoogle Scholar
  47. Jianlin Li, Jiangchao Liu, Pengfei Yang, Liqian Chen, Xiaowei Huang, and Lijun Zhang. 2019. Analyzing Deep Neural Networks with Symbolic Propagation: Towards Higher Precision and Faster Verification. In SAS. 296-319. https: //doi.org/10.1007/978-3-030-32304-2_15 Google ScholarGoogle ScholarCross RefCross Ref
  48. Kristian Lum and William Isaac. 2016. To Predict and Serve? Significance 13 ( 2016 ), 14-19. Issue 5.Google ScholarGoogle Scholar
  49. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR. OpenReview.net.Google ScholarGoogle Scholar
  50. Rupak Majumdar and Indranil Saha. 2009. Symbolic Robustness Analysis. In RTSS. 355-363. https://doi.org/10.1109/RTSS. 2009.17 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Padala Manisha and Sujit Gujar. 2020. FNNC: Achieving Fairness Through Neural Networks. In IJCAI. 2277-2283.Google ScholarGoogle Scholar
  52. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A Survey on Bias and Fairness in Machine Learning. CoRR abs/ 1908.09635 ( 2019 ).Google ScholarGoogle Scholar
  53. Antoine Miné. 2004. Relational Abstract Domains for the Detection of Floating-Point Run-Time Errors. In ESOP. 3-17. https://doi.org/10.1007/978-3-540-24725-8_2 Google ScholarGoogle ScholarCross RefCross Ref
  54. Antoine Miné. 2006a. Symbolic Methods to Enhance the Precision of Numerical Abstract Domains. In VMCAI. 348-363. https://doi.org/10.1007/11609773_23 Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Antoine Miné. 2006b. The Octagon Abstract Domain. Higher-Order and Symbolic Computation 19, 1 ( 2006 ), 31-100. https://doi.org/10.1007/s10990-006-8609-1 Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Matthew Mirman, Timon Gehr, and Martin T. Vechev. 2018. Diferentiable Abstract Interpretation for Provably Robust Neural Networks. In ICML. 3575-3583.Google ScholarGoogle Scholar
  57. Matthew Mirman, Gagandeep Singh, and Martin T. Vechev. 2019. A Provable Defense for Deep Residual Networks. CoRR abs/ 1903.12519 ( 2019 ).Google ScholarGoogle Scholar
  58. Razieh Nabi and Ilya Shpitser. 2018. Fair Inference on Outcomes. In AAAI. AAAI Press.Google ScholarGoogle Scholar
  59. Vinod Nair and Geofrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML. 807-814.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Anh Mai Nguyen, Jason Yosinski, and Jef Clune. 2015. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. In CVPR. 427-436. https://doi.org/10.1109/CVPR. 2015.7298640 Google ScholarGoogle ScholarCross RefCross Ref
  61. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations. Science 366 ( 2019 ), 447-453. Issue 6464.Google ScholarGoogle Scholar
  62. Augustus Odena, Catherine Olsson, David Andersen, and Ian J. Goodfellow. 2019. TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing. In ICML (PMLR), Vol. 97. PMLR, 4901-4911.Google ScholarGoogle Scholar
  63. Judea Pearl. 2009. Causality: Models, Reasoning and Inference. Cambridge University Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. In SOSP. 1-18. https://doi.org/10.1145/3132747.3132785 Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Luca Pulina and Armando Tacchella. 2010. An Abstraction-Refinement Approach to Verification of Artificial Neural Networks. In CAV. 243-257. https://doi.org/10.1007/978-3-642-14295-6_24 Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Anian Ruoss, Mislav Balunovic, Marc Fischer, and Martin T. Vechev. 2020. Learning Certified Individually Fair Representations. CoRR abs/ 2002.10312 ( 2020 ).Google ScholarGoogle Scholar
  67. Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A Concolic Unit Testing Engine for C. In ESEC/FSE. ACM, 263-272.Google ScholarGoogle Scholar
  68. Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. 2019. An Abstract Domain for Certifying Neural Networks. PACMPL 3, POPL ( 2019 ), 41 : 1-41 : 30. https://doi.org/10.1145/3290354 Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, and Daniel Kroening. 2018. Concolic Testing for Deep Neural Networks. In ASE. ACM, 109-119.Google ScholarGoogle Scholar
  70. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing Properties of Neural Networks. In ICLR. http://arxiv.org/abs/1312.6199Google ScholarGoogle Scholar
  71. Pedro Tabacof and Eduardo Valle. 2016. Exploring the Space of Adversarial Images. In IJCNN. 426-433. https://doi.org/10. 1109/IJCNN. 2016.7727230 Google ScholarGoogle ScholarCross RefCross Ref
  72. Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars. In ICSE. ACM, 303-314.Google ScholarGoogle Scholar
  73. Florian Tramèr, Vaggelis Atlidakis, Roxana Geambasu, Daniel J. Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2017. FairTest: Discovering Unwarranted Associations in Data-Driven Applications. In EuroS&P. IEEE, 401-416.Google ScholarGoogle Scholar
  74. Hoang-Dung Tran, Stanley Bak, Weiming Xiang, and Taylor T. Johnson. 2020. Verification of Deep Convolutional Neural Networks Using ImageStars. In CAV. 18-42.Google ScholarGoogle Scholar
  75. Sakshi Udeshi, Pryanshu Arora, and Sudipta Chattopadhyay. 2018. Automated Directed Fairness Testing. In ASE. ACM, 98-108.Google ScholarGoogle Scholar
  76. Caterina Urban and Peter Müller. 2018. An Abstract Interpretation Framework for Input Data Usage. In ESOP. 683-710. https://doi.org/10.1007/978-3-319-89884-1_24 Google ScholarGoogle ScholarCross RefCross Ref
  77. Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Formal Security Analysis of Neural Networks Using Symbolic Intervals. In Security. USENIX, 1599-1614.Google ScholarGoogle Scholar
  78. Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. 2020. Training Individually Fair ML Models with Sensitive Subspace Robustness. In ICLR.Google ScholarGoogle Scholar
  79. Fuyuan Zhang, Sankalan Pal Chowdhury, and Maria Christakis. 2020. DeepSearch: A Simple and Efective Blackbox Attack for Deep Neural Networks. In ESEC/FSE. ACM. To appear.Google ScholarGoogle Scholar
  80. Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid. 2018. DeepRoad: GAN-Based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems. In ASE. ACM, 132-142.Google ScholarGoogle Scholar

Index Terms

  1. Perfectly parallel fairness certification of neural networks

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!