review-article

Free access

Datasheets for datasets

Authors:

Timnit Gebru,

Jamie Morgenstern,

Briana Vecchione,

Jennifer Wortman Vaughan,

Hanna Wallach,

Hal Daumé III,

Kate CrawfordAuthors Info & Claims

Communications of the ACM, Volume 64, Issue 12

Pages 86 - 92

https://doi.org/10.1145/3458723

Published: 19 November 2021 Publication History

All formats PDF

Abstract

Documentation to facilitate communication between dataset creators and consumers.

References

[1]

Andrews, D., Bonta, J., and Wormith, J. The recent past and near future of risk and/or need assessment. Crime & Delinquency 52, 1 (2006), 7--27.

Crossref

Google Scholar

[2]

Bender, E. and Friedman, B. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Trans. of the Assoc. for Computational Linguistics 6 (2018), 587--604.

Crossref

Google Scholar

[3]

Bhardwaj, A. et al. DataHub: Collaborative data science & dataset version management at scale. CoRR abs/1409.0798 (2014).

Google Scholar

[4]

Bolukbasi, T., Chang, K., Zou, J., Saligrama, V., and Kalai, A. Man is to computer programmer as woman is to homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems (2016).

Google Scholar

[5]

Buolamwini, J. and Gebru, T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the Conf. on Fairness, Accountability, and Transparency (2018). 77--91.

Google Scholar

[6]

Cao, Y. and Daumé, H. Toward gender-inclusive coreference resolution. In Proceedings of the Conf. of the Assoc. for Computational Linguistics (2020). abs/1910.13913.

Google Scholar

[7]

Cao, Y. and Daumé, H. Toward gender-inclusive coreference resolution. In Proceedings of the Conf. of the Assoc. for Computational Linguistics (2020).

Crossref

Google Scholar

[8]

Cheney, J., Chiticariu, L., and Tan, W. Provenance in databases: Why, how, and where. Foundations and Trends in Databases 1, 4 (2009), 379--474.

Google Scholar

[9]

Chmielinski, K. et al. The dataset nutrition label (2nd Gen): Leveraging context to mitigate harms in artificial intelligence. In NeurIPS Workshop on Dataset Curation and Security, 2020.

Google Scholar

[10]

Choi, E. et al. QuAC: Question answering in context. In Proceedings of the 2018 Conf. on Empirical Methods in Natural Language Processing.

Google Scholar

[11]

Chui, G. Project will use AI to prevent or minimize electric grid failures, 2017.

Google Scholar

[12]

Dastin, J. Amazon scraps secret AI recruiting tool that showed bias against women, 2018; https://reut.rs/3imOH4d.

Google Scholar

[13]

Garvie, C., Bedoya, A., and Frankle, J. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, Washington, D.C., 2016.

Google Scholar

[14]

Hind, M. et al. Varshney. Increasing trust in AI services through supplier's declarations of conformity. CoRR abs/1808.07261 (2018).

Google Scholar

[15]

Holstein, K., Vaughan, J., Daumé, H, Dudík, M., and Wallach, H. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of 2019 ACM CHI Conf. on Human Factors in Computing Systems.

Google Scholar

[16]

Huang, G., Ramesh, M., Berg, T., and Learned-Miller, E. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49. University of Massachusetts Amherst, 2007.

Google Scholar

[17]

Krasin, I. et al. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017.

Google Scholar

[18]

Lin, T. The new investor. UCLA Law Review 60 (2012), 678.

Google Scholar

[19]

Mann, G. and O'Neil, C. Hiring Algorithms Are Not Neutral, 2016; https://hbr.org/2016/12/hiring-algorithms-are-not-neutral.

Google Scholar

[20]

Mitchell, M. et al. Model cards for model reporting. In Proceedings of the Conf. on Fairness, Accountability, and Transparency (2019). 220--229.

Digital Library

Google Scholar

[21]

O'Connor, M. How AI Could Smarten Up Our Water System, 2017.

Google Scholar

[22]

Pang, B. and Lee, L. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42^nd Annual Meeting of the Assoc. for Computational Linguistics. 2004, 271.

Digital Library

Google Scholar

[23]

Seck, I., Dahmane, K., Duthon, P., and Loosli, G. Baselines and a datasheet for the Cerema AWP dataset. CoRR abs/1806.04016 (2018). http://arxiv.org/abs/1806.04016

Google Scholar

[24]

Doha Supply Systems. Facial Recognition, 2017.

Google Scholar

[25]

World Economic Forum Global Future Council on Human Rights 2016--2018. How to Prevent Discriminatory Outcomes in Machine Learning; 2018. https://www.weforum.org/whitepapers/how-to-prevent-discriminatory-outcomes-inmachine-learning.

Google Scholar

[26]

Yagcioglu, S., Erdem, A., Erdem, E., and Ikizler-Cinbis, N. RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conf. on Empirical Methods in Natural Language Processing.

Google Scholar

Cited By

View all

Chicco DFabris AJurman G(2025)The Venus score for the assessment of the quality and trustworthiness of biomedical datasetsBioData Mining10.1186/s13040-024-00412-x18:1Online publication date: 9-Jan-2025
https://doi.org/10.1186/s13040-024-00412-x
Muralikumar MMcDonald D(2025)An Emerging Design Space of How Tools Support Collaborations in AI Design and DevelopmentProceedings of the ACM on Human-Computer Interaction10.1145/37011819:1(1-28)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3701181
Leschanowsky ARusti CQuinlan CPnacek MGorce LHutiri W(2025)A Data Perspective on Ethical Challenges in Voice Biometrics ResearchIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.34468467:1(118-131)Online publication date: Jan-2025
https://doi.org/10.1109/TBIOM.2024.3446846
Show More Cited By

Index Terms

Datasheets for datasets
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory

Recommendations

Datasheets for Energy Datasets: An Ethically-Minded Approach to Documentation
e-Energy '23 Companion: Companion Proceedings of the 14th ACM International Conference on Future Energy Systems

This work presents an argument for the use of specific documentation for the ethical development, use, and sharing of energy datasets, and an evaluation of current practice in the energy AI community. Drawing on a recently developed resource from the ...
Augmented Datasheets for Speech Datasets and Ethical Decision-Making
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

Speech datasets are crucial for training Speech Language Technologies (SLT); however, the lack of diversity of the underlying training data can lead to serious limitations in building equitable and robust SLT products, especially along dimensions of ...
What is in our datasets?: describing a structure of datasets
ACSW '16: Proceedings of the Australasian Computer Science Week Multiconference

In order to facilitate research based on datasets in empirical software engineering, the meaning of data must be able to be interpreted correctly. Datasets contain measurements that are associated with metrics and entities. In some datasets, it is not ...

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 64, Issue 12

December 2021

101 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3502158

Editor:
Andrew A. Chien
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2021

Published in CACM Volume 64, Issue 12

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Review-article
Popular
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

615
Total Citations
View Citations
42,994
Total Downloads

Downloads (Last 12 months)11,851
Downloads (Last 6 weeks)1,106

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chicco DFabris AJurman G(2025)The Venus score for the assessment of the quality and trustworthiness of biomedical datasetsBioData Mining10.1186/s13040-024-00412-x18:1Online publication date: 9-Jan-2025
https://doi.org/10.1186/s13040-024-00412-x
Muralikumar MMcDonald D(2025)An Emerging Design Space of How Tools Support Collaborations in AI Design and DevelopmentProceedings of the ACM on Human-Computer Interaction10.1145/37011819:1(1-28)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3701181
Leschanowsky ARusti CQuinlan CPnacek MGorce LHutiri W(2025)A Data Perspective on Ethical Challenges in Voice Biometrics ResearchIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.34468467:1(118-131)Online publication date: Jan-2025
https://doi.org/10.1109/TBIOM.2024.3446846
Wang YHernández HAlbrecht CZhu X(2025)Feature Guided Masked Autoencoder for Self-Supervised Learning in Remote SensingIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.349323718(321-336)Online publication date: 2025
https://doi.org/10.1109/JSTARS.2024.3493237
Alderman JPalmer JLaws EMcCradden MOrdish JGhassemi MPfohl SRostamzadeh NCole-Lewis HGlocker BCalvert MPollard TGill JGath JAdebajo ABeng JLeung CKuku SFarmer LMatin RMateen BMcKay FHeller KKarthikesalingam ATreanor DMackintosh MOakden-Rayner LPearson RManrai AMyles PKumuthini JKapacee ZSebire NNazer LSeah JAkbari ABerman LGichoya JRighetto LSamuel DWasswa WCharalambides MArora APujari SSummers CSapey EWilkinson SThakker VDenniston ALiu X(2025)Tackling Algorithmic Bias and Promoting Transparency in Health Datasets: The STANDING Together Consensus RecommendationsNEJM AI10.1056/AIp24010882:1Online publication date: Jan-2025
https://doi.org/10.1056/AIp2401088
Giner-Miguelez JGómez ACabot J(2025)On the Readiness of Scientific Data Papers for a Fair and Transparent Use in Machine LearningScientific Data10.1038/s41597-025-04402-412:1Online publication date: 13-Jan-2025
https://doi.org/10.1038/s41597-025-04402-4
Celepija APalmero Aprosio ALepri BKazhamiakin R(2025)AI product cards: a framework for code-bound formal documentation cards in the public administrationData & Policy10.1017/dap.2024.557Online publication date: 8-Jan-2025
https://doi.org/10.1017/dap.2024.55
Pagallo U(2025)LLMs meet the AI Act: Who’s the sorcerer’s apprentice?Cambridge Forum on AI: Law and Governance10.1017/cfl.2024.61Online publication date: 6-Jan-2025
https://doi.org/10.1017/cfl.2024.6
Wei XKumar NZhang H(2025)Addressing Bias in Generative AI: Challenges and Research Opportunities in Information ManagementInformation & Management10.1016/j.im.2025.104103(104103)Online publication date: Jan-2025
https://doi.org/10.1016/j.im.2025.104103
Geng Y(2025)Research on the promotion of intelligent entertainment voice robots in personalized English learning based on data mining and gamified teaching experienceEntertainment Computing10.1016/j.entcom.2024.10081652(100816)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100816
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Datasheets for Energy Datasets: An Ethically-Minded Approach to Documentation

Augmented Datasheets for Speech Datasets and Ethical Decision-Making

What is in our datasets?: describing a structure of datasets

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations