How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India

ABSTRACT

Accountability, a requisite for responsible AI, can be facilitated through transparency mechanisms such as audits and explainability. However, prior work suggests that the success of these mechanisms may be limited to Global North contexts; understanding the limitations of current interventions in varied socio-political conditions is crucial to help policymakers facilitate wider accountability. To do so, we examined the mediation of accountability in the existing interactions between vulnerable users and a ‘high-risk’ AI system in a Global South setting. We report on a qualitative study with 29 financially-stressed users of instant loan platforms in India. We found that users experienced intense feelings of indebtedness for the ‘boon’ of instant loans, and perceived huge obligations towards loan platforms. Users fulfilled obligations by accepting harsh terms and conditions, over-sharing sensitive data, and paying high fees to unknown and unverified lenders. Users demonstrated a dependence on loan platforms by persisting with such behaviors despite risks of harms such as abuse, recurring debts, discrimination, privacy harms, and self-harm to them. Instead of being enraged with loan platforms, users assumed responsibility for their negative experiences, thus releasing the high-powered loan platforms from accountability obligations. We argue that accountability is shaped by platform-user power relations, and urge caution to policymakers in adopting a purely technical approach to fostering algorithmic accountability. Instead, we call for situated interventions that enhance agency of users, enable meaningful transparency, reconfigure designer-user relations, and prompt a critical reflection in practitioners towards wider accountability. We conclude with implications for responsibly deploying AI in FinTech applications in India and beyond.

References

1. 2021. Non-Banking Financial Company. https://www.rbi.org.in/Scripts/FAQView.aspx?Id=92
2. 2022. Dhani - India’s Trusted Site for Finance, Healthcare and Online Medicines. https://www.dhani.com/
3. 2022. Five ways that AI augments FinTech. https://indiaai.gov.in/article/five-ways-that-ai-augments-fintech
4. 2022. Get line of credit up to Rs. 5 Lakhs - MoneyTap. https://www.moneytap.com
5. 2022. Kissht. https://kissht.com
6. 2022. KreditBee. https://www.kreditbee.in/
7. 2022. SmartCoin - Get Instant Credit. https://smartcoin.co.in/
8. Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–18.
9. Ada Lovelace Institute, AI Now Institute, and Open Government Partnership. 2021. Algorithmic Accountability for the Public Sector. (2021).
10. Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE access 6(2018), 52138–52160.
11. Sray Agarwal. 2020. AI powered FinTech: The drivers of Digital India. https://indiaai.gov.in/article/ai-powered-fintech-the-drivers-of-digital-india
12. Syed Ishtiaque Ahmed, Md Romael Haque, Shion Guha, Md Rashidujjaman Rifat, and Nicola Dell. 2017. Privacy, security, and surveillance in the Global South: A study of biometric mobile SIM registration in Bangladesh. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 906–918.
13. Syed Ishtiaque Ahmed, Steven J Jackson, Nova Ahmed, Hasan Shahid Ferdous, Md Rashidujjaman Rifat, ASM Rizvi, Shamir Ahmed, and Rifat Sabbir Mansur. 2014. Protibadi: A platform for fighting sexual harassment in urban Bangladesh. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2695–2704.
14. Saleema Amershi. 2020. Toward Responsible AI by Planning to Fail. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3607–3607.
15. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13.
16. Amsterdam. 2020. Algorithmic Register Amsterdam. https://algoritmeregister.amsterdam.nl/en/ai-register/
17. Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. new media & society 20, 3 (2018), 973–989.
18. Varsha Bansal. 2021. Shame, suicide and the dodgy loan apps plaguing Google’s Play Store. Wired UK (2021). https://www.wired.co.uk/article/google-loan-apps-india-deaths
19. Alistair Barr. 2015. Google Mistakenly Tags Black People as ‘Gorillas,’ Showing Limits of Algorithms. Wall Street Journal (jul 2015). https://www.wsj.com/articles/BL-DGB-42522
20. Ruha Benjamin. 2019. Race after technology: Abolitionist tools for the new jim code. Social Forces (2019).
21. Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. ’It’s Reducing a Human Being to a Percentage’ Perceptions of Justice in Algorithmic Decisions. In Proceedings of the 2018 Chi conference on human factors in computing systems. 1–14.
22. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016), 4349–4357.
23. Danah Boyd and Kate Crawford. 2012. Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, communication & society 15, 5 (2012), 662–679.
24. Virginia Braun and Victoria Clarke. 2012. Thematic analysis.(2012).
25. Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
26. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.
27. Jenna Burrell. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 2053951715622512.
28. CAC. 2021. Internet Information Service Algorithm Recommendation Management Regulations. http://www.cac.gov.cn/2021-08/27/c_1631652502874117.htm.
29. Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. ” Hello AI”: Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proceedings of the ACM on Human-computer Interaction 3, CSCW(2019), 1–24.
30. Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine learning interpretability: A survey on methods and metrics. Electronics 8, 8 (2019), 832.
31. Husanjot Chahal, Sara Abdulla, Jonathan Murdick, and Ilya Rahkovsky. 2021. Mapping India’s AI Potential. Technical Report.
32. C Chausson. 2016. France opens the source code of tax and benefits calculators to increase transparency.
33. Sasha Costanza-Chock. 2020. Design justice: Community-led practices to build the worlds we need.
34. Aman Dalmia, Jerome White, Ankit Chaurasia, Vishal Agarwal, Rajesh Jain, Dhruvin Vora, Balasaheb Dhame, Raghu Dharmaraju, and Rahul Panicker. 2020. Pest Management In Cotton Farms: An AI-System Case Study from the Global South. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3119–3127.
35. Nicholas Diakopoulos. 2015. Algorithmic accountability: Journalistic investigation of computational power structures. Digital journalism 3, 3 (2015), 398–415.
36. Nicholas Diakopoulos. 2017. Enabling accountability of algorithmic media: transparency as a constructive and critical lens. In Transparent data mining for Big and Small Data. Springer, 25–43.
37. Nicholas Diakopoulos and Michael Koliska. 2017. Algorithmic transparency in the news media. Digital journalism 5, 7 (2017), 809–828.
38. Catherine D’ignazio and Lauren F Klein. 2020. Data feminism. MIT press.
39. Paul Dourish. 2010. HCI and environmental sustainability: the politics of design and the design of politics. In Proceedings of the 8th ACM conference on designing interactive systems. 1–10.
40. Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances (2018).
41. Michaelanne Dye, David Nemer, Laura R Pina, Nithya Sambasivan, Amy S Bruckman, and Neha Kumar. 2017. Locating the Internet in the Parks of Havana. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 3867–3878.
42. Upol Ehsan, Q Vera Liao, Michael Muller, Mark O Riedl, and Justin D Weisz. 2021. Expanding explainability: Towards social transparency in ai systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.
43. Upol Ehsan, Samir Passi, Q Vera Liao, Larry Chan, I Lee, Michael Muller, Mark O Riedl, 2021. The Who in Explainable AI: How AI Background Shapes Perceptions of AI Explanations. arXiv preprint arXiv:2107.13509(2021).
44. MC Elish and EA Watkins. 2020. Repairing innovation: A study of integrating AI in clinical care. Unpublished Manuscript(2020).
45. Isil Erel, Lea H Stern, Chenhao Tan, and Michael S Weisbach. 2018. Could machine learning help companies select better board directors?Harvard Business Review 1, 5 (2018).
46. Nel Escher and Nikola Banovic. 2020. Exposing Error in Poverty Management Technology: A Method for Auditing Government Benefits Screening Tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1(2020), 1–20.
47. Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. ” I always assumed that I wasn’t really that close to [her]” Reasoning about Invisible Algorithms in News Feeds. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 153–162.
48. Motahhare Eslami, Kristen Vaccaro, Min Kyung Lee, Amit Elazari Bar On, Eric Gilbert, and Karrie Karahalios. 2019. User attitudes towards algorithmic opacity and transparency in online reviewing platforms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.
49. ET Goverment. 2021. Odisha launches AI based online life certificate system for pensioners.
50. Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.
51. Simson Garfinkel, Jeanna Matthews, Stuart S Shapiro, and Jonathan M Smith. 2017. Toward algorithmic transparency and accountability.
52. Susan Wharton Gates, Vanessa Gail Perry, and Peter M Zorn. 2002. Automated underwriting in mortgage lending: Good news for the underserved?Housing Policy Debate 13, 2 (2002), 369–391.
53. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Commun. ACM 64, 12 (2021), 86–92.
54. Tarleton Gillespie. 2014. The relevance of algorithms. Media technologies: Essays on communication, materiality, and society 167, 2014(2014), 167.
55. Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 80–89.
56. Government of Tamil Nadu. 2020. Tamil Nadu Safe and Ethical Artificial Intelligence Policy. Technical Report.
57. Darrell Grissen 2019. Behavior Revealed in Mobile Phone Usage Predicts Loan Repayment. Technical Report. arXiv. org.
58. Varun Gulshan, Lily Peng, Marc Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316, 22 (2016), 2402–2410.
59. M Haataja, L van de Fliert, and P Rautio. 2020. Public AI Registers: Realising AI transparency and civic participation in government use of AI. (2020).
60. Alexa Hagerty and Igor Rubinov. 2019. Global AI ethics: a review of the social impacts and ethical implications of artificial intelligence. arXiv preprint arXiv:1907.07892(2019).
61. Sara Hajian, Francesco Bonchi, and Carlos Castillo. 2016. Algorithmic bias: From discrimination discovery to fairness-aware data mining. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2125–2126.
62. Anne Henriksen, Simon Enni, and Anja Bechmann. 2021. Situated accountability: Ethical principles, certification standards, and explanation methods in applied AI. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 574–585.
63. Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 22, 7 (2019), 900–915.
64. Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021. Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 560–575.
65. ICF IIPS. 2017. India National Family Health Survey NFHS-4 2015–16. Mumbai: IIPS and ICF(2017).
66. Google India. 2022. Helping users stay safe online. https://forindia.withgoogle.com/intl/en/trust-and-safety/
67. Internet Freedom Foundation. [n.d.]. Project Panoptic: Facial Recognition Systems in India.
68. Lilly C Irani and M Six Silberman. 2013. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In Proceedings of the SIGCHI conference on human factors in computing systems. 611–620.
69. Mohammad Hossein Jarrahi, Gemma Newlands, Min Kyung Lee, Christine T Wolf, Eliscia Kinder, and Will Sutherland. 2021. Algorithmic management in a work context. Big Data & Society 8, 2 (2021), 20539517211020332.
70. Maria K, Styliani Kleanthous, Pınar Barlas, and Jahna Otterbacher. 2021. I Agree with the Decision, but They Didn’t Deserve This: Future Developers’ Perception of Fairness in Algorithmic Decisions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 690–700. https://doi.org/10.1145/3442188.3445931
71. Shivaram Kalyanakrishnan, Rahul Alex Panicker, Sarayu Natarajan, and Shreya Rao. 2018. Opportunities and challenges for artificial intelligence in India. In Proceedings of the 2018 AAAI/ACM conference on AI, Ethics, and Society. 164–170.
72. Vaishnav Kameswaran and Srihari Hulikal Muralidhar. 2019. Cash, Digital Payments and Accessibility: A Case Study from Metropolitan India. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–23.
73. Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu SP, and Nithya Sambasivan. 2022. ”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India. In CHI Conference on Human Factors in Computing Systems (CHI ’22).
74. Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Daniella Raz, and PM Krafft. 2020. Toward situated interventions for algorithmic equity: lessons from the field. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 45–55.
75. Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
76. Jakko Kemper and Daan Kolkman. 2019. Transparent to whom? No algorithmic accountability without a critical audience. Information, Communication & Society 22, 14 (2019), 2081–2096.
77. Os Keyes, Jevan Hutson, and Meredith Durbin. 2019. A mulching proposal: Analysing and improving an algorithmic system for turning the elderly into high-nutrient slurry. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.
78. Bran Knowles, Lynne Blair, Mike Hazas, and Stuart Walker. 2013. Exploring sustainability research in computing: where we are and where we go next. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 305–314.
79. Steinar Kvale. 2008. Doing interviews. Sage.
80. Ms Amina Lahreche, Ms Sumiko Ogawa, Ms Kimberly Beaton, Purva Khera, Majid Bazarbash, Mr Ulric Eriksson von Allmen, Ms Ratna Sahay, 2020. The Promise of Fintech: Financial Inclusion in the Post COVID-19 Era. Technical Report. International Monetary Fund.
81. Min Kyung Lee, Daniel Kusbit, Evan Metsky, and Laura Dabbish. 2015. Working with machines: The impact of algorithmic and data-driven management on human workers. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 1603–1612.
82. Min Kyung Lee and Katherine Rich. 2021. Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
83. Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: informing design practices for explainable AI user experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15.
84. Ryan Mac. 2021. Facebook Apologizes After A.I. Puts ‘Primates’ Label on Video of Black Men. The New York Times (sep 2021). https://www.nytimes.com/2021/09/03/technology/facebook-ai-race-primates.html
85. Vidushi Marda. 2018. Artificial Intelligency Policy in India: A Framework for Engaging the Limits of Data-Driven Decision Making. Technical Report.
86. Vidushi Marda and Shivangi Narayan. 2020. Data in New Delhi’s predictive policing system. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 317–324.
87. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) 54, 6 (2021), 1–35.
88. Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. 2021. Algorithmic impact assessments and accountability: The co-construction of impacts. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 735–746.
89. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.
90. Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf. 2021. Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Available at SSRN 3877437(2021).
91. NITI Aayog. 2021. Responsible AI #AIFORALL. (2021).
92. Safiya Umoja Noble. 2018. Algorithms of oppression. New York University Press.
93. Chinasa T Okolo, Srujana Kamath, Nicola Dell, and Aditya Vashistha. 2021. “It cannot do all of my work”: Community Health Worker Perceptions of AI-Enabled Mobile Health Applications in Rural India. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–20.
94. Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.
95. Jacki O’neill, Anupama Dhareshwar, and Srihari H Muralidhar. 2017. Working digital money into a cash economy: The collaborative work of loan payment. Computer Supported Cooperative Work (CSCW) 26, 4 (2017), 733–768.
96. Google PAIR. 2019. People + AI Guidebook. https://design.google/ai-guidebook
97. Joyojeet Pal, Priyank Chandra, Vaishnav Kameswaran, Aakanksha Parameshwar, Sneha Joshi, and Aditya Johri. 2018. Digital payment and its discontents: Street shops and the Indian government’s push for cashless transactions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
98. Frank Pasquale. 2015. The black box society. Harvard University Press.
99. S Pénicaud. 2021. Building Public Algorithm Registers: Lessons Learned from the French Approach. (2021).
100. Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations as mechanisms for supporting algorithmic transparency. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–13.
101. Inioluwa Deborah Raji and Joy Buolamwini. 2019. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 429–435.
102. Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 33–44.
103. Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where Responsible AI Meets Reality: Practitioner Perspectives on Enablers for Shifting Organizational Practices. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 7 (apr 2021), 23 pages. https://doi.org/10.1145/3449081
104. Padmini Ray Murray, Naveen L Bagalkot, Shreyas Srivatsa, and Paul Anthony. 2021. Design Beku: Toward Decolonizing Design and Technology through Collaborative and Situated Care-in-Practices. Global Perspectives 2, 1 (2021), 26132.
105. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386(2016).
106. Rashida Richardson. 2021. Defining and Demystifying Automated Decision Systems. Maryland Law Review, Forthcoming(2021).
107. A Ruhaak. 2021. When One Affects Many: The Case For Collective Consent. Mozilla Foundation 20(2021).
108. Johnny Saldaña. 2015. The coding manual for qualitative researchers. Sage.
109. Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021. Re-imagining algorithmic fairness in india and beyond. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 315–328.
110. Nithya Sambasivan, Ed Cutrell, Kentaro Toyama, and Bonnie Nardi. 2010. Intermediated technology use in developing communities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2583–2592.
111. Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
112. Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. [n.d.]. Auditing algorithms: Research methods for detecting discrimination on internet platforms. ([n. d.]).
113. Alison Smith-Renner, Ron Fan, Melissa Birchfield, Tongshuang Wu, Jordan Boyd-Graber, Daniel S Weld, and Leah Findlater. 2020. No explainability without accountability: An empirical study of explanations and feedback in interactive ml. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
114. Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press.
115. The Artificial Intelligence Task Force. 2018. No Report of Task Force on Artificial Intelligence. Technical Report. https://dpiit.gov.in/whats-new/report-task-force-artificial-intelligence
116. Harry Tucker. 2016. Australian Uber drivers say the company is manipulating their ratings to boost its fees. Business Insider Australia 20 (2016).
117. James Vincent. 2016. Twitter taught {Microsoft}’s friendly {AI} chatbot to be a racist asshole in less than a day. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist
118. John Vines, Mark Blythe, Stephen Lindsay, Paul Dunphy, Andrew Monk, and Patrick Olivier. 2012. Questionable concepts: critique as resource for designing with eighty somethings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1169–1178.
119. Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
120. Ruotong Wang, F. Maxwell Harper, and Haiyi Zhu. 2020. Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376813
121. Maranke Wieringa. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 1–18.
122. Allison Woodruff, Sarah E Fox, Steven Rousso-Schindler, and Jeffrey Warshaw. 2018. A qualitative exploration of perceptions of algorithmic fairness. In Proceedings of the 2018 chi conference on human factors in computing systems. 1–14.
123. Kyra Yee, Uthaipon Tantipongpipat, and Shubhanshu Mishra. 2021. Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency. arXiv preprint arXiv:2105.08667(2021).