research-article

Children attribute moral standing to a personified agent

Author:

Nathan G. FreierAuthors Info & Claims

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 343 - 352

https://doi.org/10.1145/1357054.1357113

Published: 06 April 2008 Publication History

Abstract

This paper describes the results of a study conducted to answer two questions: (1) Do children generalize their understanding of distinctions between conventional and moral violations in human-human interactions to human-agent interactions? and (2) Does the agent's ability to make claims to its own moral standing influence children's judgments? A two condition, between- and within-subjects study was conducted in which 60 eight and nine year-old children interacted with a personified agent and observed a researcher interacting with the same agent. A semi-structured interview was conducted to investigate the children's judgments and reasoning about the observed interactions as well as hypothetical human-human interactions. Results suggest that children do distinguish between conventional and moral violations in human-agent interactions and that the ability of the agent to express harm and make claims to its own rights significantly increases children's likelihood of identifying an act against the agent as a moral violation.

References

[1]

Bakeman, R. & Gottman, J. M. (1997). Observing Interaction: An Introduction to Sequential Analysis. NY: Cambridge University Press.

[2]

Cassell, J. (2000). Embodied Conversational Agents. The MIT Press.

[3]

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical Methods for Rates and Proportions, 3rd Ed. Hoboken: John Wiley and Son

[4]

Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003). A survey of socially interactive robots. Robotics and Autonomous Systems, 42(3-4), 143--166.

[5]

Freier, N. G. (2007). Children Distinguish Conventional from Moral Violations in Interactions with a Personified Agent. Dissertation Manuscript. The Information School, University of Washington.

Digital Library

[6]

Friedman, B. (Ed.) (1997). Human Values and the Design of Computer Technology. Stanford, CA: CSLI.

Digital Library

[7]

Friedman, B., Kahn, P. H., Jr., & Borning, A. (2006). Value Sensitive Design and information systems. In P. Zhang & D. Galletta (eds.), Human-Computer Interaction in Management Information Systems: Foundations. (pp. 348--372). Armonk, NY: M.E. Sharpe.

[8]

Friedman, B., Kahn, P. H., Jr., & Hagman, J. (2003). Hardware companions? -- What online AIBO discussion forums reveal about the human-robotic relationship. Proceedings of the ACM 2003 Conference on Human Factors in Computing Systems (CHI '03). (pp. 273--280). New York: Association for Computing Machinery.

Digital Library

[9]

Gelman, S. A. & Opfer, J. E. (2002) Development of the animate-inanimate distinction. In Goswami, U. (Ed.) Blackwell Handbook of Childhood Cognitive Development. Malden, MA: Blackwell.

[10]

Isbister, K. (2006). Better Game Characters by Design: A Psychological Approach. Boston: Morgan Kaufmann.

Digital Library

[11]

Kahn, P. H., Jr. (1999). The Human Relationship with Nature: Development and Culture. MIT Press.

[12]

Kahn, P. H., Jr., Friedman, B., Perez-Granados, D., & Freier, N. G. (2006). Robotic pets in the lives of preschool children. Interaction Studies, 7(3), 405--436.

[13]

Killen M., Lee-Kim, J., McGlothlin, H., & Stangor, C. (2002). How children and adolescents evaluate gender and racial exclusion. Monographs for the Society for Research in Child Development (Serial No. 271), 67(4). Oxford, UK: Basil Blackwell.

[14]

Landis, J. R. & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159--174.

[15]

Lester, J., Converse, S., et al. (1997). The persona effect: Affective impact of animated pedagogical agents. CHI '97: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. (pp. 359--366). ACM.

Digital Library

[16]

Nass, C. & Brave, S. (2005). Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship. Cambridge: MIT Press.

Digital Library

[17]

Nass, C., Moon, Y., Morkes, J., Kim, E. Y., & Fogg, B. J. (1997). Computers are social actors: A review of current research. In B. Friedman (Ed.). (1997). Human Values and the Design of Computer Technology (pp. 137--162). Stanford, CA: CSLI.

Digital Library

[18]

Nissenbaum, H. (1996) Accountability in a computerized society. Science and Engineering Ethics 2, 25--42.

[19]

Piaget, J. (1983). Piaget's theory. In W. Kessen (Ed.), P. H. Mussen (Series Ed.), Handbook of Child Psychology: Vol. 1. History, Theory, and Methods (4th ed., pp. 103--128). New York: Wiley.

[20]

Reeves, B. & Nass, C. (1996). The Media Equation: How People Treat Computers, Television, and New Media like Real People and Places. The Center for the Study of Language and Information Publications.

Digital Library

[21]

Smetana, J. (2006). Social-cognitive domain theory: Consistencies and variations in children's moral judgments. In M. Killen & J. Smetana (Eds.), Handbook of Moral Development. (pp. 119--154). Mahwah, NJ: Lawrence Erlbaum Associates.

[22]

Turiel. E., & Davidson, P. (1986). Heterogeneity, inconsistency, and asynchrony in the development of cognitive structures. In I. Levin, (Ed.). Stage & Structure: Reopening the Debate. (pp. 106--143). Ablex, Norwood, N.J.

[23]

Wiener, N. (1950). The Human Use of Human Beings. Boston: Da Copa Press.

Cited By

Sadek MMougenot C(2024)Challenges in Value-Sensitive AI Design: Insights from AI Practitioner InterviewsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2439021(1-18)Online publication date: 18-Dec-2024
https://doi.org/10.1080/10447318.2024.2439021
Kaufman GJarbo KGonzalez C(2024)Bias at Play: Investigating Sensitization and Desensitization to Diversity and Inclusion via Interactions with BotsComputers in Human Behavior Reports10.1016/j.chbr.2024.100493(100493)Online publication date: Sep-2024
https://doi.org/10.1016/j.chbr.2024.100493
Cociancig CHeuer HBreiter A(2024)AI ethics unwrapped: an empirical investigation of ethical principles in collaborative ideation processesAI and Ethics10.1007/s43681-024-00638-9Online publication date: 27-Dec-2024
https://doi.org/10.1007/s43681-024-00638-9
Show More Cited By

Index Terms

Children attribute moral standing to a personified agent
1. Human-centered computing
  1. Interaction design
    1. Interaction design process and methods
      1. User centered design

Recommendations

Robotic pets in the lives of preschool children
CHI EA '04: CHI '04 Extended Abstracts on Human Factors in Computing Systems

This study examined preschool children's reasoning about and behavioral interactions with one of the most advanced robotic pets currently on the retail market, Sony's robotic dog AIBO. Eighty children, equally divided between two age groups, 34-50 ...
Children distinguish conventional from moral violations in interactions with a personified agent
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing Systems

This paper describes the preliminary results of a study conducted to answer two questions: (1) Do children generalize their understanding of distinctions between conventional and moral violations in human-human interactions to human-agent interactions? ...
The moral accountability of a personified agent: young adults' conceptions
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

This paper describes the preliminary results of a study conducted to answer the question: Do users attribute moral accountability to personified agent technologies? A pilot study was conducted in which 20 college students interacted with a personified ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2008

1870 pages

ISBN:9781605580111

DOI:10.1145/1357054

General Chairs:
Mary Czerwinski
Microsoft Research, USA
,
Arnie Lund
Microsoft, USA
,
Program Chair:
Desney Tan
Microsoft Research, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 April 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '08

Sponsor:

CHI '08: CHI Conference on Human Factors in Computing Systems

April 5 - 10, 2008

Florence, Italy

Acceptance Rates

CHI '08 Paper Acceptance Rate 157 of 714 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
732
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sadek MMougenot C(2024)Challenges in Value-Sensitive AI Design: Insights from AI Practitioner InterviewsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2439021(1-18)Online publication date: 18-Dec-2024
https://doi.org/10.1080/10447318.2024.2439021
Kaufman GJarbo KGonzalez C(2024)Bias at Play: Investigating Sensitization and Desensitization to Diversity and Inclusion via Interactions with BotsComputers in Human Behavior Reports10.1016/j.chbr.2024.100493(100493)Online publication date: Sep-2024
https://doi.org/10.1016/j.chbr.2024.100493
Cociancig CHeuer HBreiter A(2024)AI ethics unwrapped: an empirical investigation of ethical principles in collaborative ideation processesAI and Ethics10.1007/s43681-024-00638-9Online publication date: 27-Dec-2024
https://doi.org/10.1007/s43681-024-00638-9
Wester JLee Mvan Berkel N(2023)Moral Transparency as a Mitigator of Moral Bias in Conversational User InterfacesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603752(1-6)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603752
Vernim SBauer HRauch EZiegler MUmbrello S(2022)A value sensitive design approach for designing AI-based worker assistance systems in manufacturingProcedia Computer Science10.1016/j.procs.2022.01.248200(505-516)Online publication date: 2022
https://doi.org/10.1016/j.procs.2022.01.248
Sapraz MHan S(2021)Implicating Human Values for designing a Digital Government Collaborative Platform for Environmental Issues: A Value Sensitive Design ApproachSustainability10.3390/su1311624013:11(6240)Online publication date: 1-Jun-2021
https://doi.org/10.3390/su13116240
Harris JAnthis J(2021)The Moral Consideration of Artificial Entities: A Literature ReviewScience and Engineering Ethics10.1007/s11948-021-00331-827:4Online publication date: 9-Aug-2021
https://doi.org/10.1007/s11948-021-00331-8
King TAggarwal NTaddeo MFloridi L(2021)Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and SolutionsEthics, Governance, and Policies in Artificial Intelligence10.1007/978-3-030-81907-1_13(251-282)Online publication date: 3-Nov-2021
https://doi.org/10.1007/978-3-030-81907-1_13
King TAggarwal NTaddeo MFloridi L(2021)Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and SolutionsThe 2020 Yearbook of the Digital Ethics Lab10.1007/978-3-030-80083-3_14(195-227)Online publication date: 31-Oct-2021
https://doi.org/10.1007/978-3-030-80083-3_14
Aguiar N(2021)A paradigm for assessing adults' and children's concepts of artificially intelligent virtual charactersHuman Behavior and Emerging Technologies10.1002/hbe2.2833:4(618-634)Online publication date: 14-Sep-2021
https://doi.org/10.1002/hbe2.283
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents