Abstract
People use the knowledge acquired from past experiences in assessing the trustworthiness of a trustee. In a time where the agents are being increasingly accepted as partners in collaborative efforts and activities, it is critical to understand all aspects of human trust development in agent partners. For human-agent virtual ad hoc teams to be effective, humans must be able to trust their agent counterparts. To earn the humans’ trust, agents need to quickly develop an understanding of the expectation of human team members and adapt accordingly. This study empirically investigates the impact of past experience on human trust in and reliance on agent teammates. To do so, we developed a team coordination game, the Game of Trust (GoT), in which two players repeatedly cooperate to complete team tasks without prior assignment of subtasks. The effects of past experience on human trust are evaluated by performing an extensive set of controlled experiments with participants recruited from Amazon Mechanical Turk, a crowdsourcing marketplace. We collect both teamwork performance data as well as surveys to gauge participants’ trust in their agent teammates. The results show that positive (negative) past experience increases (decreases) human trust in agent teammates; lack of past experience leads to higher trust levels compared to positive past experience; positive (negative) past experience facilitates (hinders) reliance on agent teammates; the relationship between trust in and reliance on agent teammates is not always correlated. These findings provide clear and significant evidence of the influence of key factors on human trust in virtual agent teammates and enhance our understanding of the changes in human trust in peer-level agent teammates with respect to past experience.
- Dimitrios Antos, Celso de Melo, Jonathan Gratch, and Barbara Grosz. 2011. The influence of emotion expression on perceptions of trustworthiness in negotiation. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI’11). 772--778. Google Scholar
Digital Library
- Benoit A. Aubert and Barbara L. Kelsey. 2003. Further understanding of trust and performance in virtual teams. Small Group Res. 34, 5 (Oct. 2003), 575--618.Google Scholar
Cross Ref
- Robert Axelrod. 1984. The Evolution of Cooperation. Basic Books.Google Scholar
- Gary Bente, Odile Baptist, and Haug Leuschner. 2012. To buy or not to buy: Influence of seller photos and reputation on buyer trust and purchase behavior. Int. J. Human-Comput. Stud. 70, 1 (2012), 1--13. Google Scholar
Digital Library
- Moshe Bitan, Ya’akov Gal, Sarit Kraus, Elad Dokow, and Amos Azaria. 2013. Social rankings in human-computer committees. In Proceedings of the AAAI Conference on Artificial Intelligence. 116--122. Google Scholar
Digital Library
- Grant Blank and William H. Dutton. 2012. Age and trust in the internet: The centrality of experience and attitudes toward technology in britain. Soc. Sci. Comput. Rev. 30, 2 (2012), 135--151. Google Scholar
Digital Library
- Cody Buntain and Jennifer Golbeck. 2015. Trust transfer between contexts. Journal of Trust Management 2, 6 (2015), 1--16.Google Scholar
Cross Ref
- Christiano Castelfranchi and Rino Falcone. 2010. Trust Theory: A Socio-Cognitive and Computational Model. John Wiley 8 Sons. Google Scholar
Digital Library
- E. Chan and P. Vorderer. 2006. Massively multiplayer online games. In Playing Computer Games: Motives, Responses, and Consequences, P. Vorderer and J. Bryant (Eds.). Lawrence Erlbaum Associates Publishers, Mahwah, NJ, 77--88.Google Scholar
- Ying-Hueih Chen, Shu-Hua Chien, Jyh-Jeng Wu, and Pei-Yin Tsai. 2010. Impact of signals and experience on trust and trusting behavior. Cyberpsych., Behav., Soc. Netw. 13, 5 (Oct. 2010), 539--546.Google Scholar
- Celso de Melo, Peter Carnevale, and Jonathan Gratch. 2014. Social categorization and cooperation between humans and computers. In Proceedings of the Meeting of the Cognitive Science Society (CogSci’14). 2109--2114.Google Scholar
- Ewart J. de Visser, Frank Krueger, Patrick McKnight, Steven Scheid, Melissa Smith, Stephanie Chalk, and Raja Parasuraman. 2012. The world is not enough: Trust in cognitive agents. In Proceedings of the Human Factors and Ergonomics Society Meeting 56, 1 (Sept. 2012), 263--267.Google Scholar
Cross Ref
- Hongying Du and M. N. Huhns. 2013. Determining the effect of personality types on human-agent interactions. In Proceedings of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI’13) and Intelligent Agent Technologies (IAT’13). 239--244. Google Scholar
Digital Library
- Jennifer R. Dunn and Maurice E. Schweitzer. 2005. Feeling and believing: The influence of emotion on trust. J. Person. Soc. Psychol. 88, 5 (May 2005), 736--748.Google Scholar
Cross Ref
- William H. Dutton and Adrian Shepherd. 2006. Trust in the internet as an experience technology. Inform., Commun. Soc. 9, 4 (Nov. 2006), 433--451.Google Scholar
Cross Ref
- Mary T. Dzindolet, Scott A. Peterson, Regina A. Pomranky, Linda G. Pierce, and Hall P. Beck. 2003. The role of trust in automation reliance. Int. J. Human-Comput. Stud. 58, 6 (2003), 697--718. Google Scholar
Digital Library
- Rino Falcone and Cristiano Castelfranchi. 2004. Trust dynamics: How trust is influenced by direct experiences and by trust itself. In Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems. IEEE Computer Society, Washington, DC, 740--747. Google Scholar
Digital Library
- Xiaocong Fan, Sooyoung Oh, Michael McNeese, John Yen, Haydee Cuevas, Laura Strater, and Mica R. Endsley. 2008. The influence of agent reliability on trust in human-agent collaboration. In Proceedings of the 15th European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction. ACM, New York, NY, 1--8. Google Scholar
Digital Library
- Ernst Fehr and Klaus M. Schmidt. 1999. A theory of fairness, competition, and cooperation. Quart. J. Econ. 114, 3 (Aug. 1999), 817--868.Google Scholar
Cross Ref
- Mark A. Fuller, Mark A. Serva, and John “Skip” Benamati. 2007. Seeing is believing: The transitory influence of reputation information on e-commerce trust and decision making. Decis. Sci. 38, 4 (2007), 675--699.Google Scholar
- Alyssa Glass, Deborah L. McGuinness, and Michael Wolverton. 2008. Toward establishing trust in adaptive agents. In Proceedings of the 13th International Conference on Intelligent User Interfaces. ACM, New York, NY, 227--236. Google Scholar
Digital Library
- Feyza Merve Hafizoglu and Sandip Sen. 2015. Evaluating trust levels in human-agent teamwork in virtual environments. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’15). 1--16.Google Scholar
- Galit Haim, Yaakov Kobi Gal, Sarit Kraus, and Michele Gelfand. 2012. A cultural sensitive agent for human-computer negotiation. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), Conitzer, Winikoff, Padgham, and van der Hoek (Eds.). International Foundation for Autonomous Agents and Multiagent Systems, 451--458. Google Scholar
Digital Library
- Nader Hanna and Deborah Richards. 2014. “Building a bridge”: Communication, trust and commitment in human-intelligent virtual agent teams. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’14).Google Scholar
- Sebastian Hergeth, Lutz Lorenz, and Josef F. Krems. 2017. Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Human Fact. 59, 3 (May 2017), 457--470.Google Scholar
Cross Ref
- Kevin Anthony Hoff and Masooda Bashir. 2015. Trust in automation: Integrating empirical evidence on factors that influence trust. Human Fact. 57, 3 (May 2015), 407--434.Google Scholar
Cross Ref
- Gareth R. Jones and Jennifer M. George. 1998. The experience and evolution of trust: Implications for cooperation and teamwork. Acad. Manag. Rev. 23, 3 (July 1998).Google Scholar
Cross Ref
- Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The future of crowd work. In Proceedings of the Conference on Computer Supported Cooperative Work. ACM, 1301--1318. Google Scholar
Digital Library
- Sherrie Y. X. Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quart. 30, 4 (Dec. 2006), 941--960. Google Scholar
Digital Library
- John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human Fact.: J. Human Fact. Ergon. Soc. 46, 1 (Spring 2004), 50--80.Google Scholar
Cross Ref
- R. B. Lount Jr. 2010. The impact of positive mood on trust in interpersonal and intergroup interactions. J. Person. Soc. Psychol. 98, 3 (2010), 420--433.Google Scholar
Cross Ref
- Dietrich Manzey, Juliane Reichenbach, and Linda Onnasch. 2012. Human performance consequences of automated decision aids: The impact of degree of automation and system experience. J. Cog. Eng. Decis. Mak. 6 (Jan. 2012), 57--87.Google Scholar
Cross Ref
- Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An integrative model of organizational trust. Acad. Manag. Rev. 20, 3 (July 1995), 709--734.Google Scholar
Cross Ref
- Daniel J. McAllister. 1995. Affect- and cognition-based trust as foundations for interpersonal cooperation in organizations. Acad. Manag. J. 38, 1 (Feb. 1995), 24--59.Google Scholar
- Stephanie M. Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but i don’t know why effects of implicit attitudes toward automation on trust in an automated system. Human Fact.: J. Human Fact. Ergon. Soc. 55, 3 (June 2013), 520--534.Google Scholar
Cross Ref
- Tim Robert Merritt, Kian Boon Tan, Christopher Ong, Aswin Thomas Abraham, Teong Leong Chuah, and Kevin McGee. 2011. Are artificial team-mates scapegoats in computer games? In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW’11). ACM, 685--688. Google Scholar
Digital Library
- Kathleen L. Mosier, Linda J. Skitka, Susan Heers, and Mark Burdick. 1998. Automation bias: Decision making and performance in high-tech cockpits. Int. J. Aviat. Psychol. 8, 1 (1998), 47--63.Google Scholar
Cross Ref
- Florian Nothdurft, Tobias Heinroth, and Wolfgang Minker. 2013. The impact of explanation dialogues on human-computer trust. In Human-Computer Interaction. Users and Contexts of Use, Masaaki Kurosu (Ed.). Springer, Berlin, 59--67.Google Scholar
- Christopher Ong, Kevin McGee, and Teong Leong Chuah. 2012. Closing the human-AI team-mate gap: How changes to displayed information impact player behavior towards computer teammates. In Proceedings of the 24th Australian Computer-Human Interaction Conference. ACM, New York, NY, 433--439. Google Scholar
Digital Library
- R. Pak, N. Fink, M. Price, B. Bass, and L. Sturre. 2012. Decision support aids with anthropomorphic characteristics influence trust and performance in younger and older adults. Ergonomics 55, 9 (July 2012), 1059--1072.Google Scholar
Cross Ref
- Paul Robinette, Alan R. Wagner, and Ayanna M. Howard. 2013. Building and maintaining trust between humans and guidance robots in an emergency. In Proceedings of the AAAI Spring Symposium: Trust and Autonomous Systems. 78--83.Google Scholar
- Juergen Sauer, Alain Chavaillaz, and David Wastell. 2016. Experience of automation failures in training: Effects on trust, automation bias, complacency and performance. Ergonomics 59, 6 (May 2016), 767--780.Google Scholar
Cross Ref
- Sandip Sen. 2013. A comprehensive approach to trust management. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems. 797--800. Google Scholar
Digital Library
- C. K. Stokes, J. B. Lyons, K. Littlejohn, J. Natarian, E. Case, and N. Speranza. 2010. Accounting for the human in cyberspace: Effects of mood on trust in automation. In Proceedings of the International Symposium on Collaborative Technologies and Systems (CTS’10). 180--187.Google Scholar
- Piotr Sztompka. 1999. Trust: A Sociological Theory. Cambridge University Press, Cambridge, UK.Google Scholar
- A. van Wissen, Y. Gal, B. A. Kamphorst, and M. V. Dignum. 2012. Human-agent teamwork in dynamic environments. Comput. Human Behav. 28, 1 (2012), 23--33. Google Scholar
Digital Library
- Arlette van Wissen, Jurriaan van Diggelen, and Virginia Dignum. 2009. The effects of cooperative agent behavior on human cooperativeness. In Proceedings of 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’09). 1179--1180. Google Scholar
Digital Library
- Frank Verberne, Jaap Ham, and Cees J. H. Midden. 2012. Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Human Fact.: J. Human Fact. Ergon. Soc. 54, 5 (May 2012), 799--810.Google Scholar
Cross Ref
- Frank M. F. Verberne, Jaap Ham, and Cees J. H. Midden. 2014. Familiar faces: Trust in a facially similar agent. In Proceedings of the HAIDM Workshop at the Autonomous Agents and Multiagent Systems (AAMAS’14).Google Scholar
- Robert A. Wagner and Michael J. Fischer. 1974. The string-to-string correction problem. J. ACM 21, 1 (Jan. 1974), 168--173. Google Scholar
Digital Library
Index Terms
Understanding the Influences of Past Experience on Trust in Human-agent Teamwork
Recommendations
The Effects of Past Experience on Trust in Repeated Human-Agent Teamwork
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent SystemsFor human-agent virtual ad hoc teams to be effective, humans must be able to trust their agent counterparts. To earn the human's trust, agents need to quickly develop an understanding of the expectation of human team members and adapt accordingly. This ...
Reputation Based Trust In Human-Agent Teamwork Without Explicit Coordination
HAI '18: Proceedings of the 6th International Conference on Human-Agent InteractionInteracting with strangers and agents through computer networks has become a routine aspect of our daily lives. In such environments, reputation plays a critical role in determining our future interactions and satisfaction derived from them. This paper ...
Comparing Human Trust Attitudes Towards Human and Agent Teammates
HAI '20: Proceedings of the 8th International Conference on Human-Agent InteractionAgents' roles in our lives increasingly matter as they engage with people in a variety of important tasks. To achieve successful human-agent teamwork, it is critical to know the differences and similarities in people's attitudes towards human and agent ...






Comments