ABSTRACT
As research on algorithms and their impact proliferates, so do calls for scrutiny/accountability of algorithms. A systematic review of the work that has been done in the field of 'algorithmic accountability' has so far been lacking. This contribution puts forth such a systematic review, following the PRISMA statement. 242 English articles from the period 2008 up to and including 2018 were collected and extracted from Web of Science and SCOPUS, using a recursive query design coupled with computational methods. The 242 articles were prioritized and ordered using affinity mapping, resulting in 93 'core articles' which are presented in this contribution. The recursive search strategy made it possible to look beyond the term 'algorithmic accountability'. That is, the query also included terms closely connected to the theme (e.g. ethics and AI, regulation of algorithms). This approach allows for a perspective not just from critical algorithm studies, but an interdisciplinary overview drawing on material from data studies to law, and from computer science to governance studies. To structure the material, Bovens's widely accepted definition of accountability serves as a focal point. The material is analyzed on the five points Bovens identified as integral to accountability: its arguments on (1) the actor, (2) the forum, (3) the relationship between the two, (3) the content and criteria of the account, and finally (5) the consequences which may result from the account. The review makes three contributions. First, an integration of accountability theory in the algorithmic accountability discussion. Second, a cross-sectoral overview of the that same discussion viewed in light of accountability theory which pays extra attention to accountability risks in algorithmic systems. Lastly, it provides a definition of algorithmic accountability based on accountability theory and algorithmic accountability literature.
References
- ACM. 2018. ACM Code of Ethics and Professional Conduct. https://www.acm.org/code-of-ethicsGoogle Scholar
- Amina Adadi and Mohammed Berrada. 2018. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138--52160. Google Scholar
Cross Ref
- Optimity advisors and Future of privacy forum. 2018. algo:aware: Raising awareness on algorithms. Technical Report. Washington.Google Scholar
- AI Now. 2018. Algorithmic Accountability Policy Toolkit. Technical Report. AI Now / New York University, New York.Google Scholar
- AI Now. 2018. Litigating algorithms: challenging government use of algorithmic decision systems. Technical Report. AI Now/New York University, New York.Google Scholar
- Madeleine Akrich and Bruno Latour. 1992. A summary of a convenient vocabulary for the semiotics of human and nonhuman assemblies. In Shaping technology / building society: studies in sociotechnical change, Wiebe E. Bijker and John Law (Eds.). MIT Press, Cambridge/London, Chapter 9, 259--264.Google Scholar
- Mike Ananny. 2015. Toward an Ethics of Algorithms. Science, Technology, & Human Values 41, 1 (2015), 93--117. Google Scholar
Cross Ref
- Mike Ananny. 2016. Toward an Ethics of Algorithms: Convening, Observation, Probability, and Timeliness. Science Technology and Human Values 41, 1 (2016), 93--117. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media and Society 20, 3 (2018), 973--989. Google Scholar
Cross Ref
- Leighton Andrews. 2018. Public administration, public leadership and the construction of public value in the age of the algorithm and 'big data'. Public Administration (2018). Google Scholar
Cross Ref
- Theo Araujo, Claes De Vreese, Natali Helberger, Sanne Kruikemeier, Julia Van Weert, Bol, Nadine, Daniel Oberski, Mykola Pechenizkiy, Gabi Schaap, and Linnet Taylor. 2018. Automated Decision-Making Fairness in an AI-driven world: Public perceptions, hopes, and concerns. Technical Report. University of Amsterdam, Amsterdam.Google Scholar
- Thomas Arnold and Matthias Scheutz. 2018. The "big red button" is too late: an alternative model for the ethical evaluation of AI systems. Ethics and Information Technology 20, 1 (2018), 59--69. Google Scholar
Digital Library
- Karen Barad. 2007. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke University Press, Durham/London.Google Scholar
Cross Ref
- Mathieu Bastian, Sebastien Heymann, and Mathieu Jacomy. 2009. Gephi: an open source software for exploring and manipulating networks. In Proceedings of the Third International ICWSM Conference. http://www.aaai.org/ocs/index.php/ICWSM/09/paper/download/154/1009Google Scholar
- Seth D. Baum. 2017. Social choice ethics in artificial intelligence. AI and Society (2017), 1--12. Google Scholar
Cross Ref
- Beall's List. 2018. Beall's list of Predatory Journals and Publishers. https://beallslist.weebly.com/Google Scholar
- Lyria Bennett Moses and Janet Chan. 2018. Algorithmic prediction in policing: assumptions, evaluation, and accountability. Policing and Society 28, 7 (2018), 806--822. Google Scholar
Cross Ref
- Bij Voorbaat Verdacht. 2018. Wat is SyRI? https://bijvoorbaatverdacht.nl/watis-syri/Google Scholar
- Bij Voorbaat Verdacht. 2019. Missie. https://bijvoorbaatverdacht.nl/missie/Google Scholar
- Reuben Binns. 2017. Algorithmic Accountability and Public Reason. Philosophy & Technology (2017). arXiv:arXiv:1702.08608 Google Scholar
Cross Ref
- Reuben Binns. 2018. What can political philosophy teach us about algorithmic fairness? IEEE Security & Privacy, 2018 16, 3 (2018), 73--80.Google Scholar
Cross Ref
- Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions. In CHI '18. arXiv:1801.10408 Google Scholar
Digital Library
- Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008, 10 (2008), 1--12. arXiv:0803.0476 Google Scholar
Cross Ref
- Mark Bovens. 2007. Analysing and Assessing Accountability: A Conceptual Framework. European Law Journal 13, 4 (2007), 447--468. arXiv:1468-0386 Google Scholar
Cross Ref
- Mark Bovens. 2010. Two concepts of accountability: Accountability as a virtue and as a Mechanism. West European Politics 33, 5 (2010), 946--967. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Mark Bovens, Thomas Schillemans, and Paul T. Hart. 2008. Does public accountability work? An assessment tool. Public Administration 86, 1 (2008), 225--242. Google Scholar
Cross Ref
- Mark Bovens and Stavros Zouridis. 2002. From to System Level -Level Bureaucracies: How Information and Communication Technology Is Transforming Administrative Discretion and Constitutional Control. Public Administration Review 62, 2 (2002), 174--184.Google Scholar
Cross Ref
- Engin Bozdag. 2013. Bias in algorithmic filtering and personalization. Ethics and Information Technology 15, 3 (2013), 209--227. Google Scholar
Digital Library
- Louis Brandeis. 1914. What publicity can do. In Other people's money and how the bankers use it. Frederick A. Srokes, New York. http://louisville.edu/law/library/special-collections/the-louis-d.-brandeis-collection/other-peoples-money-chapter-vGoogle Scholar
- Gijs Jan Brandsma and Thomas Schillemans. 2013. The accountability cube: Measuring accountability. Journal of Public Administration Research and Theory 23, 4 (2013), 953--975. Google Scholar
Cross Ref
- Kiel Brennan-Marquez. 2016. Plausible Cause: Explanatory Standards in the Age of Powerful Machines. Ssrn 1 (2016). Google Scholar
Cross Ref
- Dennis Broeders, Erik Schrijvers, Bart van der Sloot, Rosamunde van Brakel, Josta de Hoog, and Ernst Hirsch Ballin. 2017. Big Data and security policies: Towards a framework for regulating the phases of analytics and use of Big Data. Computer Law and Security Review 33, 3 (2017), 309--323. Google Scholar
Cross Ref
- Joanna Bryson and Alan Winfield. 2017. Standardizing Ethical Design for Artificial Intelligence and Autonomous Systems. Computer 50, 5 (2017), 116--119. Google Scholar
Digital Library
- Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are explanations always important?: a study of deployed, low-cost intelligent interactive systems. Proceedings of the ACM Conference on Intelligent User Interfaces (2012), 169--178. Google Scholar
Digital Library
- Jenna Burrell. 2015. How the Machine 'Thinks:' Understanding Opacity in Machine Learning Algorithms. Ssrn June (2015), 1--12. arXiv:arXiv:1307.4531v1 Google Scholar
Cross Ref
- Robyn Caplan and danah Boyd. 2018. Isomorphism through algorithms: Institutional dependencies in the case of Facebook. Big Data & Society 5, 1 (2018), 205395171875725. Google Scholar
Cross Ref
- Corinne Cath, Sandra Wachter, Brent Mittelstadt, Mariarosaria Taddeo, and Luciano Floridi. 2018. Artificial Intelligence and the 'Good Society': the US, EU, and UK approach. Science and Engineering Ethics 24, 2 (2018), 505--528. Google Scholar
Cross Ref
- Raja Chatila, Kay Firth-Butterflied, John C. Havens, and Konstantinos Karachalios. 2017. The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems. IEEE Robotics and Automation Magazine 24, 1 (2017), 110. Google Scholar
Cross Ref
- Kenneth Ward Church. 2017. Emerging trends: I did it, I did it, I did it, but. Natural Language Engineering 23, 3 (2017), 473--480. Google Scholar
Cross Ref
- Danielle Citron and Frank Pasquale. 2014. The Scored Society: Due Process for Automated Predictions. Washington Law Review 89, 1 (2014), 1--34. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Cary Coglianese and David Lehr. 2017. Regulating by robot: Administrative decision making in the Machine-learning era. Georgetown Law Journal 105, 5 (2017), 1147--1223. Google Scholar
Cross Ref
- Kate Crawford. 2016. Can an Algorithm be Agonistic? Ten Scenes from Life in Calculated Publics. Science Technology and Human Values 41, 1 (2016), 77--92. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- K. Crawford and T. Gillespie. 2014. What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media & Society (2014), 1461444814543163-. Google Scholar
Cross Ref
- Andrej Dameski. 2018. A comprehensive ethical framework for AI entities: Foundations. In 11th International Conference Artificial General Intelligence 2018. 42--51. Google Scholar
Cross Ref
- John Danaher. 2016. The Threat of Algocracy: Reality, Resistance and Accommodation. Philosophy and Technology 29, 3 (2016), 245--268. Google Scholar
Cross Ref
- John Danaher, Michael J Hogan, Chris Noone, Rónán Kennedy, Anthony Behan, Aisling De Paor, Heike Felzmann, Muki Haklay, Su-Ming Khoo, John Morison, Maria Helen Murphy, Niall O'Brolchain, Burkhard Schafer, and Kalpana Shankar. 2017. Algorithmic governance: Developing a research agenda through the power of collective intelligence. Big Data & Society 4, 2 (2017), 205395171772655. Google Scholar
Cross Ref
- Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08GGoogle Scholar
- Paul B. de Laat. 2017. Algorithmic Decision-Making Based on Machine Learning from Big Data: Can Transparency Restore Accountability? Philosophy & Technology (2017). Google Scholar
Cross Ref
- S Dekker. 2018. Transparantie van algoritmes in gebruik bij de overheid.Google Scholar
- Lina Dencik, Arne Hintz, Joanna Redden, and Harry Warne. 2018. Data Scores as Governance: Investigating uses of citizen scoring in public services. Technical Report. Data Justice Lab, Cardiff. https://datajustice.files.wordpress.com/2018/12/data-scores-as-governance-project-report2.pdfGoogle Scholar
- Nicholas Diakopoulos. 2015. Accountability in Algorithmic Decision-making: A view from computational journalism. ACM Queue december (2015), 1--24.Google Scholar
- Nicholas Diakopoulos. 2015. Algorithmic Accountability. Digital Journalism 3, 3 (2015), 398--415. Google Scholar
Cross Ref
- Nicholas Diakopoulos. 2015. Algorithmic Accountability: Journalistic investigation of computational power structures. Digital Journalism 3, 3 (2015), 398--415. Google Scholar
Cross Ref
- Nicholas Diakopoulos. 2016. Accountability in Algorithmic Decision-making: A view from computational journalism. Commun. ACM 59, 2 (2016), 56--62. Google Scholar
Digital Library
- Ezekiel Dixon-Román. 2016. Algo-Ritmo: More-Than-Human Performative Acts and the Racializing Assemblages of Algorithmic Architectures. Cultural Studies <-> Critical Methodologies 16, 5 (2016), 482--490. Google Scholar
Cross Ref
- J.E. Dobson. 2015. Can An Algorithm Be Disturbed?: Machine Learning, Intrinsic Criticism, and the Digital Humanities. College Literature: A Journal of Critical Literay Studies 42, 4 (2015), 543--564. Google Scholar
Cross Ref
- Danilo Doneda and Virgilio A.F. Almeida. 2016. What Is Algorithm Governance? IEEE Internet Computing 20, 4 (2016), 60--63. Google Scholar
Cross Ref
- Cat Drew. 2016. Data science ethics in government. Philosophical Transactions of the Royal Society 374, 2083 (2016), 20160119. Google Scholar
Cross Ref
- Marina Drosou, H.V. Jagadish, Evaggelia Pitoura, and Julia Stoyanovich. 2017. Diversity in Big Data: A Review. Big Data 5, 2 (2017), 73--84. Google Scholar
Cross Ref
- ECP Platform voor de InformatieSamenleving. 2018. Artificial Intelligence Impact Assessment.Google Scholar
- Lilian Edwards and Michael Veale. 2017. Enslaving the algorithm : from a 'right to an explanation' to a 'right to better decisions'? (2017), 12 pages.Google Scholar
- Amitai Etzioni and Oren Etzioni. 2017. Incorporating Ethics into Artificial Intelligence. Journal of Ethics 21, 4 (2017), 403--418. Google Scholar
Cross Ref
- Virginia Eubanks. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin's Press, New York.Google Scholar
Digital Library
- European Union. 2016. Regulation 2016/679 of the European parliament and the Council of the European Union. arXiv:arXiv:1011.1669v3Google Scholar
- Katherine Fink. 2018. Opening the government's black boxes: freedom of information and algorithmic accountability. Information Communication and Society 21, 10 (2018), 1453--1471. Google Scholar
Cross Ref
- Batya Friedman and Helen Nissenbaum. 1996. Bias in computer systems. ACM SIGCHI Bulletin 14, 3 (1996), 330 --347. Google Scholar
Digital Library
- Urs Gasser and Virgilio A.F. Almeida. 2017. A Layered Model for AI Governance. IEEE Internet Computing 21, 6 (2017), 58--62. Google Scholar
Cross Ref
- Gemeente Rotterdam. 2018. Betalingsregeling gemeentelijke belastingschuld. https://www.rotterdam.nl/loket/betalingsregeling-belastingschuld/Google Scholar
- Jen Jack Gieseking. 2018. Operating anew: Queering GIS with good enough software. Canadian Geographer 62, 1 (2018), 55--66. Google Scholar
Cross Ref
- A. Goffey. 2008. Algorithm. In Software Studies: a lexicon, Matthew Fuller (Ed.). MIT Press, Cambridge/London, 15--20.Google Scholar
- Mika Gröndahl, Keith Collins, and James Glanz. 2019. The dangerous flaws in Boeing's automated system. https://www.nytimes.com/interactive/2019/03/29/business/boeing-737-max-8-flaws.htmlGoogle Scholar
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Dino Pedreschi, and Fosca Giannotti. 2018. A Survey Of Methods For Explaining Black Box Models. Comput. Surveys 51, 5 (2018). arXiv:1802.01933 http://arxiv.org/abs/1802.01933Google Scholar
- Andrew J. Hawkins. 2019. Deadly Boeing crashes raise questions about airplane automation. https://www.theverge.com/2019/3/15/18267365/boeing-737-max-8-crash-autopilot-automationGoogle Scholar
- Paul Henman. 2017. The computer says 'DEBT': Towards a critical sociology of algorithms and algorithmic governance. In Data for Policy 2017: Government by Algorithm? London. Google Scholar
Cross Ref
- Paul Henman. 2019. Of algorithms, Apps and advice: digital social policy and service delivery. Journal of Asian Public Policy 12, 1 (2019), 71--89. Google Scholar
Cross Ref
- Marc Hijink. 2018. Algoritme voorspelt wie fraude pleegt bij bijstandsuitkering., 8--10 pages. https://www.nrc.nl/nieuws/2018/04/08/algoritme-voorspelt-wie-fraude-pleegt-bij-bijstandsuitkering-a1598669Google Scholar
- Russell Hotten. 2015. Volkswagen: The scandal explained. https://www.bbc.com/news/business-34324772Google Scholar
- Husson University. 2019. What is the software development life cycle? https://online.husson.edu/software-development-cycle/Google Scholar
- IEEE. [n. d.]. IEEE Code of Ethics. https://www.ieee.org/about/corporate/governance/p7-8.htmlGoogle Scholar
- Lucas D Introna. 2016. Algorithms, Governance, and Governmentality : On Governing Academic Writing. Science Technology and Human Values 41, 1 (2016), 17--49. Google Scholar
Cross Ref
- Mathieu Jacomy, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. 2014. ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the Gephi software. PLoS ONE 9, 6 (2014), 1--12. arXiv:arXiv:1209.0748v1 Google Scholar
Cross Ref
- Marijn Janssen and George Kuk. 2016. The challenges and limits of big data algorithms in technocratic governance. Government Information Quarterly 33, 3 (2016), 371--377. Google Scholar
Cross Ref
- Deborah G. Johnson and Helen Nissenbaum. 1995. Computers, ethics & social values. Prentice-Hall, Upper Saddle River.Google Scholar
- Natascha Just and Michael Latzer. 2017. Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture and Society 39, 2 (2017), 238--258. Google Scholar
Cross Ref
- Christian Katzenbach. 2012. Technologies as Institutions: Rethinking the Role of Technology in Media Governance Constellations. In Trends in Communication Policy Research, New Theories, Methods & Subjects, Manuel Puppis and Natascha Just (Eds.). Intellect Books, Bristol, 117--138. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Jakko Kemper and Daan Kolkman. 2018. Transparent to whom? No algorithmic accountability without a critical audience. Information Communication and Society (2018), 1--16. Google Scholar
Cross Ref
- Rob Kitchin. 2019. The ethics of smart cities. https://www.rte.ie/brainstorm/2019/0425/1045602-the-ethics-of-smart-cities/Google Scholar
- Donald E. Knuth. 1984. Literate Programming. Computers and Chemical Engineering 22, 12 (1984), 1745--1747. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Utku Köse. 2017. Are We Safe Enough in the Future of Artificial Intelligence? A Discussion on Machine Ethics and Artificial Intelligence Safety. In Scientific Methods in Academic Research and Teaching International Conference. 184--197.Google Scholar
- Felicitas Kraemer, Kees Van Overveld, and Martin Peterson. 2011. Is there an ethics of algorithms ? Ethics and Information Technology 13, 3 (2011), 251--260. Google Scholar
Digital Library
- Joshua A. Kroll, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable Algorithms. University of Pennsylvania Law Review 165 (2017), 633--705. arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Leer- en Expertisepunt Open Overheid. 2019. Actieplan Open Overheid 2018-2020. https://www.open-overheid.nl/actieplan-open-overheid-2018-2020-open-moet-het-zijn/Google Scholar
- Bruno Lepri, Nuria Oliver, Emmanuel Letouzé, Alex Pentland, and Patrick Vinck. 2017. Fair, Transparent, and Accountable Algorithmic Decision-making Processes. Philosophy & Technology (2017). Google Scholar
Cross Ref
- Lawrence Lessig. 1999. Code: and other laws of cyberspace. Basic Books, New York.Google Scholar
- Alessandro Liberati, Douglas G. Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C. Gøtzsche, John P.A. Ioannidis, Mike Clarke, P. J. Devereaux, Jos Kleijnen, and David Moher. 2009. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine 6, 7 (2009). arXiv:arXiv:1011.1669v3 Google Scholar
Cross Ref
- Michael Lipsky. 1980. Street-level bureaucracy: dilemmas of the individual in public services. Russell Sage Foundation, New York.Google Scholar
- Katharina Loeber. 2018. Big Data, Algorithmic Regulation, and the History of the Cybersyn Project in Chile, 1971--1973. Social Sciences 7, 4 (2018), 65. Google Scholar
Cross Ref
- Caitlin Lustig, Katie Pine, Bonnie Nardi, Lilly Irani, Min Kyung Lee, Dawn Nafus, and Christian Sandvig. 2016. Algorithmic Authority: The Ethics, Politics, and Economics of Algorithms that Interpret, Decide, and Manage. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '16 (2016), 1057--1062. Google Scholar
Digital Library
- Lei Ma, Zhongqiu Zhang, and Nana Zhang. 2018. Ethical Dilemma of Artificial Intelligence and its Research Progress. In IOP Conference Series: Materials Science and Engineering, Vol. 392. Google Scholar
Cross Ref
- Vidushi Marda. 2018. Artificial intelligence policy in India: A framework for engaging the limits of data-driven decision-making. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences (2018). Google Scholar
Cross Ref
- Kirsten Martin. 2018. Ethical Implications and Accountability of Algorithms. Journal of Business Ethics 0, 0 (2018), 1--16. Google Scholar
Cross Ref
- James McGrath and Ankur Gupta. 2018. Writing a Moral Code: Algorithms for Ethical Reasoning by Humans and Machines. Religions 9, 8 (2018), 240--259.Google Scholar
Cross Ref
- Dan McQuillan. 2018. People's Councils for Ethical Machine Learning. Social Media and Society 4, 2 (2018), 1--10. Google Scholar
Cross Ref
- Eden Medina. 2015. Rethinking algorithmic regulation. Kybernetes 44, 6-7 (2015), 1005--1019. Google Scholar
Cross Ref
- Thomas Metzinger. 2019. Ethics washing made in Europe. https://www.tagesspiegel.de/politik/eu-guidelines-ethics-washing-made-in-europe/24195496.htmlGoogle Scholar
- Brent Daniel Mittelstadt, Patrick Allo, Mariarosaria Taddeo, Sandra Wachter, and Luciano Floridi. 2016. The ethics of algorithms: Mapping the debate. Big Data & Society 3, 2 (2016), 205395171667967. Google Scholar
Cross Ref
- Annemarie Mol. 2002. The body multiple: ontology in medical practice. Duke University Press, Durham.Google Scholar
- Joshua New and Daniel Castro. 2018. How Policymakers can foster Algorithmic Accountability. Technical Report. Center for Data Innovation, Washington.Google Scholar
- New York City. 2018. Automated Decision Systems Task Force. https://www1.nyc.gov/site/adstaskforce/index.pageGoogle Scholar
- Daniel Neyland. 2016. Bearing Account-able Witness to the Ethical Algorithmic System. Science, Technology, & Human Values 41, 1 (2016), 50--76. Google Scholar
Cross Ref
- Daniel Neyland and Norma Möllers. 2017. Algorithmic IF ... THEN rules and the conditions and consequences of power. Information Communication and Society 20, 1 (2017), 45--62. Google Scholar
Cross Ref
- Helen Nissenbaum. 1994. Computing and accountability. Commun. ACM 37, 1 (1994), 72--80. Google Scholar
Digital Library
- Dietmar Offenhuber. 2017. Waste is information: infrastructure legibility and governance. MIT Press, Cambridge/London.Google Scholar
- Partnership on AI. 2019. About. https://www.partnershiponai.org/about/Google Scholar
- Partnership on AI. 2019. Partnership on AI. https://www.partnershiponai.org/Google Scholar
- Frank Pasquale. 2015. Digital Star Chamber. https://aeon.co/essays/judge-jury-and-executioner-the-unaccountable-algorithmGoogle Scholar
- David J. Pauleen, David Rooney, and Ali Intezari. 2017. Big data, little wisdom: trouble brewing? Ethical implications for the information systems discipline. Social Epistemology 31, 4 (2017), 400--416. Google Scholar
Cross Ref
- Wolter Pieters. 2011. Explanation and trust: What to tell the user in security and AI? Ethics and Information Technology 13, 1 (2011), 53--64. Google Scholar
Digital Library
- Craig Plain. 2007. Build an Affinity for K-J Method. Quality Progress 40, 3 (2007), 88.Google Scholar
- Iyad Rahwan. 2018. Society-in-the-loop: programming the algorithmic social contract. Ethics and Information Technology 20, 1 (2018), 5--14. arXiv:1707.07232 Google Scholar
Digital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In KDD '16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, San Francisco, 1135--1144. arXiv:1602.04938 http://arxiv.org/abs/1602.04938Google Scholar
Digital Library
- Rijksoverheid. 2014. Besluit SUWI. https://wetten.overheid.nl/BWBR0013267/2019-01-01{#}Hoofdstuk5aGoogle Scholar
- Alex Rosenblat, Tamara Kneese, and Danah Boyd. 2014. Algorithmic Accountability. In The Social, Cultural & Ethical Dimensions of "Big Data". Data & Society Research Institute, New York. https://datasociety.net/pubs/2014-0317/AlgorithmicAccountabilityPrimer.pdfGoogle Scholar
- Florian Saurwein, Natascha Just, and Michael Latzer. 2015. Governance of algorithms: Options and limitations. Info 17, 6 (2015), 35--49. Google Scholar
Cross Ref
- Nick Seaver. 2017. Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Big Data & Society 4, 2 (2017), 205395171773810. Google Scholar
Cross Ref
- Security.nl. 2018. Kamervragen over algoritmen voor opsporen bijstandsfraude. https://www.security.nl/posting/557836/Kamervragen+over+algoritmen+voor+opsporen+bijstandsfraude?channel=rssGoogle Scholar
- Bernd Carsten Stahl and David Wright. 2018. Ethics and Privacy in AI and Big Data: Implementing Responsible Research and Innovation. IEEE Security and Privacy 16, 3 (2018), 26--33. Google Scholar
Cross Ref
- Stats NZ. 2018. Algorithm Assessment Report. Technical Report. Stats NZ, Wellington.Google Scholar
- Daniel Susser. 2019. Ethics Alone Can't Fix Big Tech. https://slate.com/technology/2019/04/ethics-board-google-ai.htmlGoogle Scholar
- Jim Torresen. 2018. A Review of Future and Ethical Perspectives of Robotics and AI. Frontiers in Robotics and AI 4, January (2018). Google Scholar
Cross Ref
- Totta Data Lab. 2019. Gemeentelijke Fraudedetectie. https://www.tottadatalab.nl/portfolio-item/gemeentelijke-fraudedetectie/Google Scholar
- T. Van Ark. 2018. Kamervraag/vragen van het lid Buitenweg (GroenLinks).Google Scholar
- Iris Van der Tuin and Rick Dolphijn. 2012. "Matter feels, converses, suffers, desires, yearns and remembers" Interview with Karen Barad. In New Materialism: Interviews & Cartographies. Open Humanities Press, Ann Arbor, 48--70.Google Scholar
- Michael Veale, Max Van Kleek, and Reuben Binns. 2018. Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making. (2018), 1--14. arXiv:1802.01029 Google Scholar
Digital Library
- Anton Vedder and Laurens Naudts. 2017. Accountability for the use of algorithms in a big data environment. International Review of Law, Computers and Technology 31, 2 (2017), 206--224. Google Scholar
Digital Library
- Ben Wagner. 2016. Algorithmic regulation and the global default: Shifting norms in Internet technology. Etikk i Praksis 10, 1 (2016), 5--13. Google Scholar
Cross Ref
- Ben Wagner. 2018. Ethics as an Escape from Regulation: From 'ethics-washing' to ethics-shopping? In Being Profiled, Cogitas Ergo Sum, E. Bayamliogu, I. Baraliuc, L.A.W. Janssens, and M. Hildebrandt (Eds.). Amsterdam University Press, Amsterdam, 84--90.Google Scholar
- Meredith Whittaker, Kate Crawford, Roel Dobbe, Genevieve Fried, Elizabeth Kaziunas, Varoon Mathur, Sarah Myers West, Rashida Richardson, Jason Schultz, and Oscar Schwartz. 2018. AI Now Report 2018. Technical Report. AI Now/New York University, New York.Google Scholar
- Ben Williamson. 2015. Governing software: networks, databases and algorithmic power in the digital governance of public education. Learning, Media and Technology 40, 1 (2015), 83--105. Google Scholar
Cross Ref
- Karen Yeung. 2017. Algorithmic regulation: A critical interrogation. Regulation & Governance April (2017), 1--19. Google Scholar
Cross Ref
- Karen Yeung. 2017. 'Hypernudge': Big Data as a mode of regulation by design. Information, Communication and Society 20, 1 (2017), 118--136. Google Scholar
Cross Ref
- Han Yu, Zhiqi Shen, Chunyan Miao, Cyril Leung, Victor R. Lesser, and Qiang Yang. 2018. Building ethics into artificial intelligence. In IJCAI International Joint Conference on Artificial Intelligence, Vol. 2018-July. 5527--5533. arXiv:1812.02953v1 Google Scholar
Cross Ref
- Tal Zarsky. 2016. The Trouble with Algorithmic Decisions : An Analytic Road Map to Examine Efficiency and Fairness in Automated and Opaque Decision Making. Science Technology and Human Values 41, 1 (2016), 118--132. Google Scholar
Cross Ref
- Malte Ziewitz. 2016. Governing Algorithms: Myth, Mess, and Methods. Science Technology and Human Values 41, 1 (2016), 3--16. Google Scholar
Cross Ref
Index Terms
What to account for when accounting for algorithms




Comments