Abstract
Recently, there is growing concern that machine-learned software, which currently assists or even automates decision making, reproduces, and in the worst case reinforces, bias present in the training data. The development of tools and techniques for certifying fairness of this software or describing its biases is, therefore, critical. In this paper, we propose a perfectly parallel static analysis for certifying fairness of feed-forward neural networks used for classification of tabular data. When certification succeeds, our approach provides definite guarantees, otherwise, it describes and quantifies the biased input space regions. We design the analysis to be sound, in practice also exact, and configurable in terms of scalability and precision, thereby enabling pay-as-you-go certification. We implement our approach in an open-source tool called Libra and demonstrate its effectiveness on neural networks trained on popular datasets.
Supplemental Material
- Aws Albarghouthi, Loris D'Antoni, and Samuel Drews. 2017a. Repairing Decision-Making Programs Under Uncertainty. In CAV. 181-200. https://doi.org/10.1007/978-3-319-63387-9_9 Google Scholar
Cross Ref
- Aws Albarghouthi, Loris D'Antoni, Samuel Drews, and Aditya V. Nori. 2017b. FairSquare: Probabilistic Verification of Program Fairness. PACMPL 1, OOPSLA ( 2017 ), 80 : 1-80 : 30. https://doi.org/10.1145/3133904 Google Scholar
Digital Library
- Aws Albarghouthi and Samuel Vinitsky. 2019. Fairness-Aware Programming. In FAT*. 211-219. https://doi.org/10.1145/ 3287560.3287588 Google Scholar
Digital Library
- Anish Athalye, Nicholas Carlini, and David A. Wagner. 2018. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In ICML (PMLR), Vol. 80. PMLR, 274-283.Google Scholar
- Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. California Law Review 104, 3 ( 2016 ), 671-732.Google Scholar
- Clark W. Barrett and Cesare Tinelli. 2018. Satisfiability Modulo Theories. In Handbook of Model Checking. Springer, 305-343.Google Scholar
- Alexander I. Barvinok. 1994. A Polynomial Time Algorithm for Counting Integral Points in Polyhedra When the Dimension is Fixed. Mathematics of Operations Research 19, 4 ( 1994 ), 769-779. https://doi.org/10.1287/moor.19.4. 769 Google Scholar
Digital Library
- Osbert Bastani, Xin Zhang, and Armando Solar-Lezama. 2019. Probabilistic Verification of Fairness Properties via Concentration. PACMPL 3, OOPSLA ( 2019 ), 118 : 1-118 : 27.Google Scholar
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In FAT (PMLR), Vol. 81. PMLR, 77-91.Google Scholar
- Nicholas Carlini and David A. Wagner. 2016. Defensive Distillation is Not Robust to Adversarial Examples. CoRR abs/1607.04311 ( 2016 ).Google Scholar
- Nicholas Carlini and David A. Wagner. 2017a. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. In [email protected]. ACM, 3-14.Google Scholar
- Nicholas Carlini and David A. Wagner. 2017b. Towards Evaluating the Robustness of Neural Networks. In S &P. IEEE Computer Society, 39-57.Google Scholar
- Swarat Chaudhuri, Sumit Gulwani, and Roberto Lublinerman. 2012. Continuity and Robustness of Programs. Commun. ACM 55, 8 ( 2012 ), 107-115. https://doi.org/10.1145/2240236.2240262 Google Scholar
Digital Library
- Liqian Chen, Antoine Miné, and Patrick Cousot. 2008. A Sound Floating-Point Polyhedra Abstract Domain. In APLAS. 3-18. https://doi.org/10.1007/978-3-540-89330-1_2 Google Scholar
Digital Library
- Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. In AAAI. 7801-7808. https://doi.org/10.1609/aaai.v33i01. 33017801 Google Scholar
Cross Ref
- Lori A. Clarke. 1976. A System to Generate Test Data and Symbolically Execute Programs. TSE 2 ( 1976 ), 215-222. Issue 3.Google Scholar
- Cory Cornelius. 2019. The Eficacy of SHIELD Under Diferent Threat Models. CoRR abs/ 1902.00541 ( 2019 ).Google Scholar
- Patrick Cousot. 2002. Constructive Design of a Hierarchy of Semantics of a Transition System by Abstract Interpretation. Theoretical Computer Science 277, 1-2 ( 2002 ), 47-103. https://doi.org/10.1016/S0304-3975 ( 00 ) 00313-3 Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 1976. Static Determination of Dynamic Properties of Programs. In Second International Symposium on Programming. 106-130.Google Scholar
- Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In POPL. 238-252. https://doi.org/10.1145/512950.512973 Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 1979. Systematic Design of Program Analysis Frameworks. In POPL. 269-282. https: //doi.org/10.1145/567752.567778 Google Scholar
Digital Library
- Patrick Cousot and Nicolas Halbwachs. 1978. Automatic Discovery of Linear Restraints Among Variables of a Program. In POPL. 84-96. https://doi.org/10.1145/512760.512770 Google Scholar
Digital Library
- Anupam Datta, Matthew Fredrikson, Gihyuk Ko, Piotr Mardziel, and Shayak Sen. 2017. Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programs. In CCS. 1193-1210. https://doi.org/10.1145/3133956.3134097 Google Scholar
Digital Library
- Logan Engstrom, Andrew Ilyas, and Anish Athalye. 2018. Evaluating and Understanding the Robustness of Adversarial Logit Pairing. CoRR abs/ 1807.10272 ( 2018 ).Google Scholar
- Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In KDD. ACM, 259-268.Google Scholar
Digital Library
- Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness Testing: Testing Software for Discrimination. In FSE. 498-510. https://doi.org/10.1145/3106237.3106277 Google Scholar
Digital Library
- Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev. 2018. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In S & P. 3-18. https: //doi.org/10.1109/SP. 2018.00058 Google Scholar
Cross Ref
- Khalil Ghorbal, Eric Goubault, and Sylvie Putot. 2009. The Zonotope Abstract Domain Taylor1+. In CAV. 627-633. https: //doi.org/10.1007/978-3-642-02658-4_47 Google Scholar
Digital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed Automated Random Testing. In PLDI. ACM, 213-223.Google Scholar
- Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2016. Deep Learning. MIT Press.Google Scholar
Digital Library
- Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In ICLR. http://arxiv.org/abs/1412.6572Google Scholar
- Divya Gopinath, Kaiyuan Wang, Mengshi Zhang, Corina S. Pasareanu, and Sarfraz Khurshid. 2018. Symbolic Execution for Deep Neural Networks. CoRR abs/ 1807.10439 ( 2018 ).Google Scholar
- Eric Goubault and Sylvie Putot. 2013. Robustness Analysis of Finite Precision Implementations. In APLAS. 50-57. https: //doi.org/10.1007/978-3-319-03542-0_4 Google Scholar
Digital Library
- Nina Grgić-Hlača, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. 2016. The Case for Process Fairness in Learning: Feature Selection for Fair Decision Making. In NIPS 2016 ML and the Law.Google Scholar
- Cheng Guo and Felix Berkhahn. 2016. Entity Embeddings of Categorical Variables. CoRR abs/1604.06737 ( 2016 ).Google Scholar
- Boris Hanin and David Rolnick. 2019. Deep ReLU Networks Have Surprisingly Few Activation Patterns. In NIPS. Curran Associates, Inc., 359-368. http://papers.nips.cc/paper/8328-deep-relu-networks-have-surprisingly-few-activationpatterns.pdfGoogle Scholar
- Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NIPS. 3315-3323.Google Scholar
Digital Library
- Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvári. 2015. Learning with a Strong Adversary. CoRR abs/1511.03034 ( 2015 ). http://arxiv.org/abs/1511.03034Google Scholar
- Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety Verification of Deep Neural Networks. In CAV. 3-29. https://doi.org/10.1007/978-3-319-63387-9_1 Google Scholar
Cross Ref
- Bertrand Jeannet and Antoine Miné. 2009. APRON: A Library of Numerical Abstract Domains for Static Analysis. In CAV. 661-667. https://doi.org/10.1007/978-3-642-02658-4_52 Google Scholar
Digital Library
- Guy Katz, Clark W. Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Eficient SMT Solver for Verifying Deep Neural Networks. In CAV. 97-117. https://doi.org/10.1007/978-3-319-63387-9_5 Google Scholar
Cross Ref
- Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. In CHI. ACM, 3819-3828.Google Scholar
- Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding Discrimination Through Causal Reasoning. In NIPS. 656-666.Google Scholar
Digital Library
- James C. King. 1976. Symbolic Execution and Program Testing. CACM 19 ( 1976 ), 385-394. Issue 7.Google Scholar
- Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In NIPS. 4069-4079.Google Scholar
Digital Library
- Jef Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.Google Scholar
- Jianlin Li, Jiangchao Liu, Pengfei Yang, Liqian Chen, Xiaowei Huang, and Lijun Zhang. 2019. Analyzing Deep Neural Networks with Symbolic Propagation: Towards Higher Precision and Faster Verification. In SAS. 296-319. https: //doi.org/10.1007/978-3-030-32304-2_15 Google Scholar
Cross Ref
- Kristian Lum and William Isaac. 2016. To Predict and Serve? Significance 13 ( 2016 ), 14-19. Issue 5.Google Scholar
- Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR. OpenReview.net.Google Scholar
- Rupak Majumdar and Indranil Saha. 2009. Symbolic Robustness Analysis. In RTSS. 355-363. https://doi.org/10.1109/RTSS. 2009.17 Google Scholar
Digital Library
- Padala Manisha and Sujit Gujar. 2020. FNNC: Achieving Fairness Through Neural Networks. In IJCAI. 2277-2283.Google Scholar
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A Survey on Bias and Fairness in Machine Learning. CoRR abs/ 1908.09635 ( 2019 ).Google Scholar
- Antoine Miné. 2004. Relational Abstract Domains for the Detection of Floating-Point Run-Time Errors. In ESOP. 3-17. https://doi.org/10.1007/978-3-540-24725-8_2 Google Scholar
Cross Ref
- Antoine Miné. 2006a. Symbolic Methods to Enhance the Precision of Numerical Abstract Domains. In VMCAI. 348-363. https://doi.org/10.1007/11609773_23 Google Scholar
Digital Library
- Antoine Miné. 2006b. The Octagon Abstract Domain. Higher-Order and Symbolic Computation 19, 1 ( 2006 ), 31-100. https://doi.org/10.1007/s10990-006-8609-1 Google Scholar
Digital Library
- Matthew Mirman, Timon Gehr, and Martin T. Vechev. 2018. Diferentiable Abstract Interpretation for Provably Robust Neural Networks. In ICML. 3575-3583.Google Scholar
- Matthew Mirman, Gagandeep Singh, and Martin T. Vechev. 2019. A Provable Defense for Deep Residual Networks. CoRR abs/ 1903.12519 ( 2019 ).Google Scholar
- Razieh Nabi and Ilya Shpitser. 2018. Fair Inference on Outcomes. In AAAI. AAAI Press.Google Scholar
- Vinod Nair and Geofrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML. 807-814.Google Scholar
Digital Library
- Anh Mai Nguyen, Jason Yosinski, and Jef Clune. 2015. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. In CVPR. 427-436. https://doi.org/10.1109/CVPR. 2015.7298640 Google Scholar
Cross Ref
- Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations. Science 366 ( 2019 ), 447-453. Issue 6464.Google Scholar
- Augustus Odena, Catherine Olsson, David Andersen, and Ian J. Goodfellow. 2019. TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing. In ICML (PMLR), Vol. 97. PMLR, 4901-4911.Google Scholar
- Judea Pearl. 2009. Causality: Models, Reasoning and Inference. Cambridge University Press.Google Scholar
Digital Library
- Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. In SOSP. 1-18. https://doi.org/10.1145/3132747.3132785 Google Scholar
Digital Library
- Luca Pulina and Armando Tacchella. 2010. An Abstraction-Refinement Approach to Verification of Artificial Neural Networks. In CAV. 243-257. https://doi.org/10.1007/978-3-642-14295-6_24 Google Scholar
Digital Library
- Anian Ruoss, Mislav Balunovic, Marc Fischer, and Martin T. Vechev. 2020. Learning Certified Individually Fair Representations. CoRR abs/ 2002.10312 ( 2020 ).Google Scholar
- Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A Concolic Unit Testing Engine for C. In ESEC/FSE. ACM, 263-272.Google Scholar
- Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. 2019. An Abstract Domain for Certifying Neural Networks. PACMPL 3, POPL ( 2019 ), 41 : 1-41 : 30. https://doi.org/10.1145/3290354 Google Scholar
Digital Library
- Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, and Daniel Kroening. 2018. Concolic Testing for Deep Neural Networks. In ASE. ACM, 109-119.Google Scholar
- Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing Properties of Neural Networks. In ICLR. http://arxiv.org/abs/1312.6199Google Scholar
- Pedro Tabacof and Eduardo Valle. 2016. Exploring the Space of Adversarial Images. In IJCNN. 426-433. https://doi.org/10. 1109/IJCNN. 2016.7727230 Google Scholar
Cross Ref
- Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars. In ICSE. ACM, 303-314.Google Scholar
- Florian Tramèr, Vaggelis Atlidakis, Roxana Geambasu, Daniel J. Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2017. FairTest: Discovering Unwarranted Associations in Data-Driven Applications. In EuroS&P. IEEE, 401-416.Google Scholar
- Hoang-Dung Tran, Stanley Bak, Weiming Xiang, and Taylor T. Johnson. 2020. Verification of Deep Convolutional Neural Networks Using ImageStars. In CAV. 18-42.Google Scholar
- Sakshi Udeshi, Pryanshu Arora, and Sudipta Chattopadhyay. 2018. Automated Directed Fairness Testing. In ASE. ACM, 98-108.Google Scholar
- Caterina Urban and Peter Müller. 2018. An Abstract Interpretation Framework for Input Data Usage. In ESOP. 683-710. https://doi.org/10.1007/978-3-319-89884-1_24 Google Scholar
Cross Ref
- Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Formal Security Analysis of Neural Networks Using Symbolic Intervals. In Security. USENIX, 1599-1614.Google Scholar
- Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. 2020. Training Individually Fair ML Models with Sensitive Subspace Robustness. In ICLR.Google Scholar
- Fuyuan Zhang, Sankalan Pal Chowdhury, and Maria Christakis. 2020. DeepSearch: A Simple and Efective Blackbox Attack for Deep Neural Networks. In ESEC/FSE. ACM. To appear.Google Scholar
- Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid. 2018. DeepRoad: GAN-Based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems. In ASE. ACM, 132-142.Google Scholar
Index Terms
Perfectly parallel fairness certification of neural networks
Recommendations
Airtime Fairness for IEEE 802.11 Multirate Networks
Under a multi rate network scenario, the IEEE 802.11 DCF MAC fails to provide air-time fairness for all competing stations since the protocol is designed for ensuring max-min throughput fairness and the maximum achievable throughput by any station gets ...
Neural Network Trainer through Computer Networks
AINA '10: Proceedings of the 2010 24th IEEE International Conference on Advanced Information Networking and ApplicationsThis paper introduces a neural network training tool through computer networks. The following algorithms, such as neuron by neuron (NBN) [1][2], error back propagation (EBP), Levenberg Marquardt (LM) and its improved versions are implemented in two ...






Comments