skip to main content
research-article

Understanding the Influences of Past Experience on Trust in Human-agent Teamwork

Published:13 September 2019Publication History
Skip Abstract Section

Abstract

People use the knowledge acquired from past experiences in assessing the trustworthiness of a trustee. In a time where the agents are being increasingly accepted as partners in collaborative efforts and activities, it is critical to understand all aspects of human trust development in agent partners. For human-agent virtual ad hoc teams to be effective, humans must be able to trust their agent counterparts. To earn the humans’ trust, agents need to quickly develop an understanding of the expectation of human team members and adapt accordingly. This study empirically investigates the impact of past experience on human trust in and reliance on agent teammates. To do so, we developed a team coordination game, the Game of Trust (GoT), in which two players repeatedly cooperate to complete team tasks without prior assignment of subtasks. The effects of past experience on human trust are evaluated by performing an extensive set of controlled experiments with participants recruited from Amazon Mechanical Turk, a crowdsourcing marketplace. We collect both teamwork performance data as well as surveys to gauge participants’ trust in their agent teammates. The results show that positive (negative) past experience increases (decreases) human trust in agent teammates; lack of past experience leads to higher trust levels compared to positive past experience; positive (negative) past experience facilitates (hinders) reliance on agent teammates; the relationship between trust in and reliance on agent teammates is not always correlated. These findings provide clear and significant evidence of the influence of key factors on human trust in virtual agent teammates and enhance our understanding of the changes in human trust in peer-level agent teammates with respect to past experience.

References

  1. Dimitrios Antos, Celso de Melo, Jonathan Gratch, and Barbara Grosz. 2011. The influence of emotion expression on perceptions of trustworthiness in negotiation. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI’11). 772--778. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Benoit A. Aubert and Barbara L. Kelsey. 2003. Further understanding of trust and performance in virtual teams. Small Group Res. 34, 5 (Oct. 2003), 575--618.Google ScholarGoogle ScholarCross RefCross Ref
  3. Robert Axelrod. 1984. The Evolution of Cooperation. Basic Books.Google ScholarGoogle Scholar
  4. Gary Bente, Odile Baptist, and Haug Leuschner. 2012. To buy or not to buy: Influence of seller photos and reputation on buyer trust and purchase behavior. Int. J. Human-Comput. Stud. 70, 1 (2012), 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Moshe Bitan, Ya’akov Gal, Sarit Kraus, Elad Dokow, and Amos Azaria. 2013. Social rankings in human-computer committees. In Proceedings of the AAAI Conference on Artificial Intelligence. 116--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Grant Blank and William H. Dutton. 2012. Age and trust in the internet: The centrality of experience and attitudes toward technology in britain. Soc. Sci. Comput. Rev. 30, 2 (2012), 135--151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cody Buntain and Jennifer Golbeck. 2015. Trust transfer between contexts. Journal of Trust Management 2, 6 (2015), 1--16.Google ScholarGoogle ScholarCross RefCross Ref
  8. Christiano Castelfranchi and Rino Falcone. 2010. Trust Theory: A Socio-Cognitive and Computational Model. John Wiley 8 Sons. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. E. Chan and P. Vorderer. 2006. Massively multiplayer online games. In Playing Computer Games: Motives, Responses, and Consequences, P. Vorderer and J. Bryant (Eds.). Lawrence Erlbaum Associates Publishers, Mahwah, NJ, 77--88.Google ScholarGoogle Scholar
  10. Ying-Hueih Chen, Shu-Hua Chien, Jyh-Jeng Wu, and Pei-Yin Tsai. 2010. Impact of signals and experience on trust and trusting behavior. Cyberpsych., Behav., Soc. Netw. 13, 5 (Oct. 2010), 539--546.Google ScholarGoogle Scholar
  11. Celso de Melo, Peter Carnevale, and Jonathan Gratch. 2014. Social categorization and cooperation between humans and computers. In Proceedings of the Meeting of the Cognitive Science Society (CogSci’14). 2109--2114.Google ScholarGoogle Scholar
  12. Ewart J. de Visser, Frank Krueger, Patrick McKnight, Steven Scheid, Melissa Smith, Stephanie Chalk, and Raja Parasuraman. 2012. The world is not enough: Trust in cognitive agents. In Proceedings of the Human Factors and Ergonomics Society Meeting 56, 1 (Sept. 2012), 263--267.Google ScholarGoogle ScholarCross RefCross Ref
  13. Hongying Du and M. N. Huhns. 2013. Determining the effect of personality types on human-agent interactions. In Proceedings of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI’13) and Intelligent Agent Technologies (IAT’13). 239--244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jennifer R. Dunn and Maurice E. Schweitzer. 2005. Feeling and believing: The influence of emotion on trust. J. Person. Soc. Psychol. 88, 5 (May 2005), 736--748.Google ScholarGoogle ScholarCross RefCross Ref
  15. William H. Dutton and Adrian Shepherd. 2006. Trust in the internet as an experience technology. Inform., Commun. Soc. 9, 4 (Nov. 2006), 433--451.Google ScholarGoogle ScholarCross RefCross Ref
  16. Mary T. Dzindolet, Scott A. Peterson, Regina A. Pomranky, Linda G. Pierce, and Hall P. Beck. 2003. The role of trust in automation reliance. Int. J. Human-Comput. Stud. 58, 6 (2003), 697--718. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Rino Falcone and Cristiano Castelfranchi. 2004. Trust dynamics: How trust is influenced by direct experiences and by trust itself. In Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems. IEEE Computer Society, Washington, DC, 740--747. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Xiaocong Fan, Sooyoung Oh, Michael McNeese, John Yen, Haydee Cuevas, Laura Strater, and Mica R. Endsley. 2008. The influence of agent reliability on trust in human-agent collaboration. In Proceedings of the 15th European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction. ACM, New York, NY, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ernst Fehr and Klaus M. Schmidt. 1999. A theory of fairness, competition, and cooperation. Quart. J. Econ. 114, 3 (Aug. 1999), 817--868.Google ScholarGoogle ScholarCross RefCross Ref
  20. Mark A. Fuller, Mark A. Serva, and John “Skip” Benamati. 2007. Seeing is believing: The transitory influence of reputation information on e-commerce trust and decision making. Decis. Sci. 38, 4 (2007), 675--699.Google ScholarGoogle Scholar
  21. Alyssa Glass, Deborah L. McGuinness, and Michael Wolverton. 2008. Toward establishing trust in adaptive agents. In Proceedings of the 13th International Conference on Intelligent User Interfaces. ACM, New York, NY, 227--236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Feyza Merve Hafizoglu and Sandip Sen. 2015. Evaluating trust levels in human-agent teamwork in virtual environments. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’15). 1--16.Google ScholarGoogle Scholar
  23. Galit Haim, Yaakov Kobi Gal, Sarit Kraus, and Michele Gelfand. 2012. A cultural sensitive agent for human-computer negotiation. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), Conitzer, Winikoff, Padgham, and van der Hoek (Eds.). International Foundation for Autonomous Agents and Multiagent Systems, 451--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Nader Hanna and Deborah Richards. 2014. “Building a bridge”: Communication, trust and commitment in human-intelligent virtual agent teams. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’14).Google ScholarGoogle Scholar
  25. Sebastian Hergeth, Lutz Lorenz, and Josef F. Krems. 2017. Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Human Fact. 59, 3 (May 2017), 457--470.Google ScholarGoogle ScholarCross RefCross Ref
  26. Kevin Anthony Hoff and Masooda Bashir. 2015. Trust in automation: Integrating empirical evidence on factors that influence trust. Human Fact. 57, 3 (May 2015), 407--434.Google ScholarGoogle ScholarCross RefCross Ref
  27. Gareth R. Jones and Jennifer M. George. 1998. The experience and evolution of trust: Implications for cooperation and teamwork. Acad. Manag. Rev. 23, 3 (July 1998).Google ScholarGoogle ScholarCross RefCross Ref
  28. Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The future of crowd work. In Proceedings of the Conference on Computer Supported Cooperative Work. ACM, 1301--1318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sherrie Y. X. Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quart. 30, 4 (Dec. 2006), 941--960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human Fact.: J. Human Fact. Ergon. Soc. 46, 1 (Spring 2004), 50--80.Google ScholarGoogle ScholarCross RefCross Ref
  31. R. B. Lount Jr. 2010. The impact of positive mood on trust in interpersonal and intergroup interactions. J. Person. Soc. Psychol. 98, 3 (2010), 420--433.Google ScholarGoogle ScholarCross RefCross Ref
  32. Dietrich Manzey, Juliane Reichenbach, and Linda Onnasch. 2012. Human performance consequences of automated decision aids: The impact of degree of automation and system experience. J. Cog. Eng. Decis. Mak. 6 (Jan. 2012), 57--87.Google ScholarGoogle ScholarCross RefCross Ref
  33. Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An integrative model of organizational trust. Acad. Manag. Rev. 20, 3 (July 1995), 709--734.Google ScholarGoogle ScholarCross RefCross Ref
  34. Daniel J. McAllister. 1995. Affect- and cognition-based trust as foundations for interpersonal cooperation in organizations. Acad. Manag. J. 38, 1 (Feb. 1995), 24--59.Google ScholarGoogle Scholar
  35. Stephanie M. Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but i don’t know why effects of implicit attitudes toward automation on trust in an automated system. Human Fact.: J. Human Fact. Ergon. Soc. 55, 3 (June 2013), 520--534.Google ScholarGoogle ScholarCross RefCross Ref
  36. Tim Robert Merritt, Kian Boon Tan, Christopher Ong, Aswin Thomas Abraham, Teong Leong Chuah, and Kevin McGee. 2011. Are artificial team-mates scapegoats in computer games? In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW’11). ACM, 685--688. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Kathleen L. Mosier, Linda J. Skitka, Susan Heers, and Mark Burdick. 1998. Automation bias: Decision making and performance in high-tech cockpits. Int. J. Aviat. Psychol. 8, 1 (1998), 47--63.Google ScholarGoogle ScholarCross RefCross Ref
  38. Florian Nothdurft, Tobias Heinroth, and Wolfgang Minker. 2013. The impact of explanation dialogues on human-computer trust. In Human-Computer Interaction. Users and Contexts of Use, Masaaki Kurosu (Ed.). Springer, Berlin, 59--67.Google ScholarGoogle Scholar
  39. Christopher Ong, Kevin McGee, and Teong Leong Chuah. 2012. Closing the human-AI team-mate gap: How changes to displayed information impact player behavior towards computer teammates. In Proceedings of the 24th Australian Computer-Human Interaction Conference. ACM, New York, NY, 433--439. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. R. Pak, N. Fink, M. Price, B. Bass, and L. Sturre. 2012. Decision support aids with anthropomorphic characteristics influence trust and performance in younger and older adults. Ergonomics 55, 9 (July 2012), 1059--1072.Google ScholarGoogle ScholarCross RefCross Ref
  41. Paul Robinette, Alan R. Wagner, and Ayanna M. Howard. 2013. Building and maintaining trust between humans and guidance robots in an emergency. In Proceedings of the AAAI Spring Symposium: Trust and Autonomous Systems. 78--83.Google ScholarGoogle Scholar
  42. Juergen Sauer, Alain Chavaillaz, and David Wastell. 2016. Experience of automation failures in training: Effects on trust, automation bias, complacency and performance. Ergonomics 59, 6 (May 2016), 767--780.Google ScholarGoogle ScholarCross RefCross Ref
  43. Sandip Sen. 2013. A comprehensive approach to trust management. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems. 797--800. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. C. K. Stokes, J. B. Lyons, K. Littlejohn, J. Natarian, E. Case, and N. Speranza. 2010. Accounting for the human in cyberspace: Effects of mood on trust in automation. In Proceedings of the International Symposium on Collaborative Technologies and Systems (CTS’10). 180--187.Google ScholarGoogle Scholar
  45. Piotr Sztompka. 1999. Trust: A Sociological Theory. Cambridge University Press, Cambridge, UK.Google ScholarGoogle Scholar
  46. A. van Wissen, Y. Gal, B. A. Kamphorst, and M. V. Dignum. 2012. Human-agent teamwork in dynamic environments. Comput. Human Behav. 28, 1 (2012), 23--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Arlette van Wissen, Jurriaan van Diggelen, and Virginia Dignum. 2009. The effects of cooperative agent behavior on human cooperativeness. In Proceedings of 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’09). 1179--1180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Frank Verberne, Jaap Ham, and Cees J. H. Midden. 2012. Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Human Fact.: J. Human Fact. Ergon. Soc. 54, 5 (May 2012), 799--810.Google ScholarGoogle ScholarCross RefCross Ref
  49. Frank M. F. Verberne, Jaap Ham, and Cees J. H. Midden. 2014. Familiar faces: Trust in a facially similar agent. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’14).Google ScholarGoogle Scholar
  50. Robert A. Wagner and Michael J. Fischer. 1974. The string-to-string correction problem. J. ACM 21, 1 (Jan. 1974), 168--173. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Understanding the Influences of Past Experience on Trust in Human-agent Teamwork

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Internet Technology
      ACM Transactions on Internet Technology  Volume 19, Issue 4
      Special Section on Trust and AI and Regular Papers
      November 2019
      201 pages
      ISSN:1533-5399
      EISSN:1557-6051
      DOI:10.1145/3362102
      • Editor:
      • Ling Liu
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 13 September 2019
      • Accepted: 1 April 2019
      • Revised: 1 March 2019
      • Received: 1 November 2018
      Published in toit Volume 19, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!