Abstract
The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators? subjectivity as a major cause of biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.
- Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination Through Optimization : How Facebook 's Ad Delivery Can Lead to Biased Outcomes. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW (Nov. 2019), 199:1--199:30. https://doi.org/10.1145/3359301Google Scholar
Digital Library
- Ali Alkhatib and Michael Bernstein. 2019. Street-Level Algorithms : A Theory at the Gaps Between Policy and Decisions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 530:1--530:13. https://doi.org/10.1145/3290605.3300760Google Scholar
Digital Library
- Luis Araujo and Martin Spring. 2006. Services, Products, and the Institutional Structure of Production. Industrial Marketing Management, Vol. 35, 7 (Oct. 2006), 797--805. https://doi.org/10.1016/j.indmarman.2006.05.013Google Scholar
Cross Ref
- Paul Baker and Amanda Potts. 2013. `Why Do White People Have Thin Lips?' Google and the Perpetuation of Stereotypes via Auto-Complete Search Forms. Critical Discourse Studies, Vol. 10, 2 (May 2013), 187--204. https://doi.org/10.1080/17405904.2012.744320Google Scholar
Cross Ref
- Chelsea Barabas, Colin Doyle, JB Rubinovitz, and Karthik Dinakar. 2020. Studying up: Reorienting the Study of Algorithmic Fairness around Issues of Power. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT * '20). Association for Computing Machinery, Barcelona, Spain, 167--176. https://doi.org/10.1145/3351095.3372859Google Scholar
- Solon Barocas and Andrew D. Selbst. 2016. Big Data 's Disparate Impact. California Law Review, Vol. 104, 3 (2016), 671--732. https://doi.org/10.15779/Z38BG31Google Scholar
- Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing : Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics, Vol. 6 (2018), 587--604. https://doi.org/10.1162/tacl_a_00041Google Scholar
Cross Ref
- Jeanette Blomberg and Chuck Darrah. 2015. Toward an Anthropology of Services. The Design Journal, Vol. 18, 2 (2015), 171--192. https://doi.org/10.2752/175630615X14212498964196Google Scholar
Cross Ref
- Pierre Bourdieu. 1977. Outline of a Theory of Practice .Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511812507Google Scholar
- Pierre Bourdieu. 1985. The Social Space and the Genesis of Groups. Theory and Society, Vol. 14, 6 (1985), 723--744. https://doi.org/10.1007/BF00174048Google Scholar
Cross Ref
- Pierre Bourdieu. 1989. Social Space and Symbolic Power. Sociological Theory, Vol. 7, 1 (1989), 14--25. https://doi.org/10.2307/202060Google Scholar
Cross Ref
- Pierre Bourdieu. 1990. The logic of practice reprinted ed.). Polity Press, Cambridge.Google Scholar
- Pierre Bourdieu. 1992. Language and Symbolic Power new ed.). Blackwell Publishers, Cambridge.Google Scholar
- Pierre Bourdieu. 2000. Pascalian Meditations .Stanford University Press, Stanford, Calif.Google Scholar
- Geoffrey C. Bowker. 2000. Biodiversity Datadiversity. Social Studies of Science, Vol. 30, 5 (Oct. 2000), 643--683. https://doi.org/10.1177/030631200030005001Google Scholar
Cross Ref
- Geoffrey C. Bowker and Susan Leigh Star. 1999. Sorting Things out: Classification and Its Consequences .MIT Press, Cambridge, Mass. BD175 .B68 1999Google Scholar
Digital Library
- danah boyd and Kate Crawford. 2012. Critical Questions for Big Data : Provocations for a Cultural, Technological, and Scholarly Phenomenon. Information, Communication & Society, Vol. 15, 5 (June 2012), 662--679. https://doi.org/10.1080/1369118X.2012.678878Google Scholar
Cross Ref
- C. E. Brodley and M. A. Friedl. 1999. Identifying Mislabeled Training Data. Journal of Artificial Intelligence Research, Vol. 11 (Aug. 1999), 131--167. https://doi.org/10.1613/jair.606Google Scholar
Cross Ref
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades : Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, Vol. 81. PMLR, 77--91.Google Scholar
- Ryan Burns. 2019. New Frontiers of Philanthro -capitalism: Digital Technologies and Humanitarianism. Antipode, Vol. 51, 4 (April 2019), 1101--1122. https://doi.org/10.1111/anti.12534Google Scholar
- Kathy Charmaz. 2006. Constructing Grounded Theory : A Practical Guide through Qualitative Analysis .Sage Publications, London ; Thousand Oaks, Calif. H61.24 .C45 2006Google Scholar
- Justin Cheng and Dan Cosley. 2013. How Annotation Styles Influence Content and Preferences. In Proceedings of the 24th ACM Conference on Hypertext and Social Media - HT '13. Association for Computing Machinery, Paris, France, 214--218. https://doi.org/10.1145/2481492.2481519Google Scholar
Digital Library
- Angèle Christin. 2016. From Daguerreotypes to Algorithms : Machines, Expertise, and Three Forms of Objectivity. SIGCAS Computers and Society, Vol. 46, 1 (2016), 27--32. https://doi.org/10.1145/2908216.2908220Google Scholar
Digital Library
- Danielle Keats Citron and Frank Pasquale. 2014. The Scored Society : Due Process for Automated Predictions. Washington Law Review, Vol. 89, 1 (2014).Google Scholar
- Juliet M. Corbin and Anselm L. Strauss. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory fourth edition ed.). SAGE, Los Angeles. HA29 .C7724 2015Google Scholar
- Nick Couldry and Ulises A. Mejias. 2019. Data Colonialism : Rethinking Big Data 's Relation to the Contemporary Subject. Television & New Media, Vol. 20, 4 (May 2019), 336--349. https://doi.org/10.1177/1527476418796632Google Scholar
Cross Ref
- Kate Crawford and Trevor Paglen. 2019. Excavating AI. https://www.excavating.ai.Google Scholar
- Ciaran Cronin. 1996. Bourdieu and Foucault on Power and Modernity. Philosophy & Social Criticism, Vol. 22, 6 (Nov. 1996), 55--85. https://doi.org/10.1177/019145379602200603Google Scholar
Cross Ref
- Hannah Davis. 2020. A Dataset Is a Worldview. https://towardsdatascience.com/a-dataset-is-a-worldview-5328216dd44d.Google Scholar
- Catherine D'Ignazio and Lauren F. Klein. 2020. Data Feminism .The MIT Press, Cambridge, Massachusetts. HQ1190 .D574 2020Google Scholar
- Ravit Dotan and Smitha Milli. 2020. Value-Laden Disciplinary Shifts in Machine Learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT * '20). Association for Computing Machinery, Barcelona, Spain, 294. https://doi.org/10.1145/3351095.3373157Google Scholar
Digital Library
- Emile Durkheim and Marcel Mauss. 1963. Primitive Classification. University of Chicago Press.Google Scholar
- M. C. Elish and danah boyd. 2018. Situating Methods in the Magic of Big Data and AI. Communication Monographs, Vol. 85, 1 (Jan. 2018), 57--80. https://doi.org/10.1080/03637751.2017.1375130Google Scholar
- Virginia Eubanks. 2018. Automating Inequality : How High -Tech Tools Profile, Police, and Punish the Poor .St. Martin's Press, New York.Google Scholar
- Melanie Feinberg. 2017. A Design Perspective on Data. In CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, Denver, Colorado, USA, 2952--2963. https://doi.org/10.1145/3025453.3025837Google Scholar
Digital Library
- Tim Finin, Will Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, and Mark Dredze. 2010. Annotating Named Entities in Twitter Data with Crowdsourcing. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon 's Mechanical Turk (CSLDAMT '10). Association for Computational Linguistics, Los Angeles, California, 80--88. https://doi.org/10.5555/1866696.1866709Google Scholar
Digital Library
- Marion Fourcade and Kieran Healy. 2013. Classification Situations: Life -Chances in the Neoliberal Era. Accounting, Organizations and Society, Vol. 38, 8 (Nov. 2013), 559--572. https://doi.org/10.1016/j.aos.2013.11.002Google Scholar
Cross Ref
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for Datasets. arXiv:1803.09010 (March 2018). arxiv: 1803.09010Google Scholar
- R. Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage in, Garbage out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT * '20). Association for Computing Machinery, Barcelona, Spain, 325--336. https://doi.org/10.1145/3351095.3372862Google Scholar
Digital Library
- Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, and Klaus Mueller. 2020. Measuring Social Biases of Crowd Workers Using Counterfactual Queries. In Workshop on Fair & Responsible AI at ACM CHI Conference on Human Factors in Computing Systems. Honolulu, HI, USA.Google Scholar
- Tarleton Gillespie and Tarleton Gillespie. 2014. The Relevance of Algorithms. In Media Technologies : Essays on Communication, Materiality, and Society,, Pablo J. Boczkowski and Kirsten A. Foot (Eds.). The MIT Press, 167--194. https://doi.org/10.7551/mitpress/9780262525374.003.0009Google Scholar
- Lisa Gitelman (Ed.). 2013. "Raw Data" Is an Oxymoron .The MIT Press, Cambridge, Massachusetts ; London, England. Q355 .R385 2013Google Scholar
- Barney G. Glaser and Anselm L. Strauss. 1998. Grounded theory: Strategien qualitativer Forschung. Huber, Bern.Google Scholar
- Mary L. Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, Boston.Google Scholar
- Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M. Branham. 2018. Gender Recognition or Gender Reductionism - The Social Implications of Embedded Gender Recognition Systems.. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, 1--13. https://doi.org/10.1145/3173574.3173582Google Scholar
- Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. 2020. Towards a Critical Race Methodology in Algorithmic Fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT * '20). Association for Computing Machinery, Barcelona, Spain, 501--512. https://doi.org/10.1145/3351095.3372826Google Scholar
Digital Library
- Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018. The Dataset Nutrition Label : A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 (2018).Google Scholar
- Christoph Hube, Besnik Fetahu, and Ujwal Gadiraju. 2019. Understanding and Mitigating Worker Biases in the Crowdsourced Collection of Subjective Judgments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300637Google Scholar
Digital Library
- Gunay Kazimzade and Milagros Miceli. 2020. Biased Priorities, Biased Outcomes : Three Recommendations for Ethics -Oriented Data Annotation Practices. In Proceedings of the AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society. (AIES '20). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3375627.3375809Google Scholar
Digital Library
- Lucy Kimbell and Jeanette Blomberg. 2017. The Object of Service Design. In Designing for Service : Key Issues and New Directions. Bloomsbury Publishing, 81--94.Google Scholar
- Rob Kitchin. 2017. Thinking Critically about and Researching Algorithms. Information, Communication & Society, Vol. 20, 1 (Jan. 2017), 14--29. https://doi.org/10.1080/1369118X.2016.1154087Google Scholar
- Gary Klein, Jennifer K. Phillips, Erica L. Rall, and Deborah A. Peluso. 2007. A Data-Frame Theory of Sensemaking. In Expertise out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 113--155.Google Scholar
- Ulrike Klinger and Jakob Svensson. 2018. The End of Media Logics? On Algorithms and Agency. New Media & Society, Vol. 20, 12 (Dec. 2018), 4653--4670. https://doi.org/10.1177/1461444818779750Google Scholar
Cross Ref
- Natalia M Libakova and Ekaterina A Sertakova. 2015. The Method of Expert Interview as an Effective Research Procedure of Studying the Indigenous Peoples of the North. Journal of Siberian Federal University. Humanities & Social Sciences, Vol. 8, 1 (2015), 114--129. https://doi.org/10.17516/1997--1370--2015--8--1--114--129Google Scholar
- Michael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1--14. https://doi.org/10.1145/3313831.3376445Google Scholar
Digital Library
- Astrid Mager. 2012. Algorithmic Ideology : How Capitalist Society Shapes Search Engines. Information, Communication & Society, Vol. 15, 5 (June 2012), 769--787. https://doi.org/10.1080/1369118X.2012.676056Google Scholar
Cross Ref
- Steffen Mau. 2019. The Metric Society: On the Quantification of the Social .Polity, Cambridge ; Medford, MA. MR 2800 M447 M5Google Scholar
- Frauke Mörike. 2019. Ethnography for Human Factors Researchers. Collecting and Interweaving Threads of HCI.Google Scholar
- Michael Muller. 2014. Curiosity, Creativity, and Surprise as Analytic Tools : Grounded Theory Method. In Ways of Knowing in HCI,, Judith S. Olson and Wendy A. Kellogg (Eds.). Springer, New York, NY, 25--48. https://doi.org/10.1007/978--1--4939-0378--8_2Google Scholar
- Michael Muller, Shion Guha, Eric P.S. Baumer, David Mimno, and N. Sadat Shami. 2016. Machine Learning and Grounded Theory Method : Convergence, Divergence, and Combination. In Proceedings of the 19th International Conference on Supporting Group Work (GROUP '16). Association for Computing Machinery, Sanibel Island, Florida, USA, 3--8. https://doi.org/10.1145/2957276.2957280Google Scholar
- Michael Muller, Ingrid Lange, Dakuo Wang, David Piorkowski, Jason Tsay, Q. Vera Liao, Casey Dugan, and Thomas Erickson. 2019. How Data Science Workers Work with Data : Discovery, Capture, Curation, Design, Creation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, Glasgow, Scotland Uk, 1--15. https://doi.org/10.1145/3290605.3300356Google Scholar
Digital Library
- Safiya Umoja Noble. 2018. Algorithms of Oppression : How Search Engines Reinforce Racism .NYU Press, New York.Google Scholar
- Cathy O'Neil. 2017. Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy .PENGUIN BOOKS, London.Google Scholar
- Juho P"a"akkönen, Matti Nelimarkka, Jesse Haapoja, and Airi Lampinen. 2020. Bureaucracy as a Lens for Analyzing and Designing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA., 1--14. https://doi.org/10.1145/3313831.3376780Google Scholar
- Samir Passi and Solon Barocas. 2019. Problem Formulation and Fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT * '19). Association for Computing Machinery, Atlanta, GA, USA, 39--48. https://doi.org/10.1145/3287560.3287567Google Scholar
Digital Library
- Samir Passi and Steven Jackson. 2017. Data Vision : Learning to See Through Algorithmic Abstraction. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). Association for Computing Machinery, Portland, Oregon, USA, 2436--2447. https://doi.org/10.1145/2998181.2998331Google Scholar
Digital Library
- Samir Passi and Steven J. Jackson. 2018. Trust in Data Science : Collaboration, Translation, and Accountability in Corporate Data Science Projects. Proc. ACM Hum.-Comput. Interact., Vol. 2, CSCW (Nov. 2018), 1--28. https://doi.org/10.1145/3274405Google Scholar
- Kathleen H. Pine and Max Liboiron. 2015. The Politics of Measurement and Action. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 3147--3156. https://doi.org/10.1145/2702123.2702298Google Scholar
Digital Library
- Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d'Alché-Buc, Emily Fox, and Hugo Larochelle. 2020. Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program ). arXiv:2003.12206 (April 2020). arxiv: 2003.12206Google Scholar
- Alex Rosenblat, Tamara Kneese, and Danah Boyd. 2014. Networked Employment Discrimination. SSRN Electronic Journal (2014). https://doi.org/10.2139/ssrn.2543507Google Scholar
- Morgan Klaus Scheuerman, Jacob M. Paul, and Jed R. Brubaker. 2019. How Computers See Gender : An Evaluation of Gender Classification in Commercial Facial Analysis Services. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW (Nov. 2019). https://doi.org/10.1145/3359246Google Scholar
- Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R Brubaker. 2020. How We 've Taught Algorithms to See Identity : Constructing Race and Gender in Image Databases for Facial Analysis. Proc. ACM Hum.-Comput. Interact., Vol. 4, CSCW1 (2020). https://doi.org/10.1145/3392866Google Scholar
- Nick Seaver. 2019. Knowing Algorithms. In digitalSTS : A Field Guide for Science & Technology Studies. Princeton University Press, PRINCETON; OXFORD, 412--422.Google Scholar
- Ismaïla Seck, Khouloud Dahmane, Pierre Duthon, and Gaëlle Loosli. 2018. Baselines and a Datasheet for the Cerema AWP Dataset. In Conférence d'Apprentissage CAp (Conférence d'Apprentissage Francophone 2018). Rouen, France. https://doi.org/10.13140/RG.2.2.36360.93448Google Scholar
- Susan Leigh Star and Anselm Strauss. 1999. Layers of Silence, Arenas of Voice : The Ecology of Visible and Invisible Work. Computer Supported Cooperative Work, Vol. 8, 1--2 (March 1999), 9--30. https://doi.org/10.1023/A:1008651105359Google Scholar
Digital Library
- Robert Thornberg. 2012. Informed Grounded Theory. Scandinavian Journal of Educational Research, Vol. 56, 3 (June 2012), 243--259. https://doi.org/10.1080/00313831.2011.581686Google Scholar
- Fabian L. Wauthier and Michael I. Jordan. 2011. Bayesian Bias Mitigation for Crowdsourcing. In Proceedings of the 24th International Conference on Neural Information Processing Systems (NIPS '11). Curran Associates Inc., Granada, Spain, 1800--1808.Google Scholar
- Jennifer Wortman Vaughan and Hanna Wallach. 2020. A Human -Centered Agenda for Intelligible Machine Learning. In Machines We Trust : Getting Along with Artificial Intelligence.Google Scholar
- Eviatar Zerubavel. 1993. The Fine Line : Making Distinctions in Everyday Life. 2nd ed. ed.). University of Chicago Press.Google Scholar
- Honglei Zhuang and Joel Young. 2015. Leveraging In -Batch Annotation Bias for Crowdsourced Active Learning. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM '15). Association for Computing Machinery, Shanghai, China, 243--252. https://doi.org/10.1145/2684822.2685301Google Scholar
Digital Library
- Shoshana Zuboff. 2019. The Age of Surveillance Capitalism: The Fight for the Future at the New Frontier of Power .Profile Books, London.Google Scholar
Index Terms
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Recommendations
How Data Science Workers Work with Data: Discovery, Capture, Curation, Design, Creation
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsWith the rise of big data, there has been an increasing need for practitioners in this space and an increasing opportunity for researchers to understand their workflows and design new tools to improve it. Data science is often described as data-driven, ...
On privacy preservation in text and document-based active learning for named entity recognition
PAVLAD '09: Proceedings of the ACM first international workshop on Privacy and anonymity for very large databasesThe preservation of the privacy of persons mentioned in text requires the ability to automatically recognize and identify names. Named entity recognition is a mature field and most current approaches are based on supervised machine learning techniques. ...
A hunt for the Snark: Annotator Diversity in Data Practices
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsDiversity in datasets is a key component to building responsible AI/ML. Despite this recognition, we know little about the diversity among the annotators involved in data production. We investigated the approaches to annotator diversity through 16 semi-...






Comments