research-article

Public Access

In Search of the Dream Team: Temporally Constrained Multi-Armed Bandits for Identifying Effective Team Structures

Authors:

Melissa Valentine,

Michael S. BernsteinAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 108, Pages 1 - 13

https://doi.org/10.1145/3173574.3173682

Published: 19 April 2018 Publication History

Abstract

Team structures---roles, norms, and interaction patterns---define how teams work. HCI researchers have theorized ideal team structures and built systems nudging teams towards them, such as those increasing turn-taking, deliberation, and knowledge distribution. However, organizational behavior research argues against the existence of universally ideal structures. Teams are diverse and excel under different structures: while one team might flourish under hierarchical leadership and a critical culture, another will flounder. In this paper, we present DreamTeam: a system that explores a large space of possible team structures to identify effective structures for each team based on observable feedback. To avoid overwhelming teams with too many changes, DreamTeam introduces multi-armed bandits with temporal constraints: an algorithm that manages the timing of exploration--exploitation trade-offs across multiple bandits simultaneously. A field experiment demonstrated that DreamTeam teams outperformed self-managing teams by 38%, manager-led teams by 46%, and teams with unconstrained bandits by 41%. This research advances computation as a powerful partner in establishing effective teamwork.

References

[1]

Mark S Ackerman. 2000. The intellectual challenge of CSCW: the gap between social requirements and technical feasibility. Human-computer interaction 15, 2 (2000), 179--203.

Digital Library

[2]

Shipra Agrawal and Navin Goyal. 2012. Analysis of Thompson sampling for the multi-armed bandit problem. In Conference on Learning Theory. 39--1.

[3]

Deborah G Ancona, Gerardo A Okhuysen, and Leslie A Perlow. 2001. Taking time to integrate temporal research. Academy of Management Review 26, 4 (2001), 512--529.

[4]

Holly Arrow. 1997. Stability, bistability, and instability in small group influence patterns. Journal of Personality and Social Psychology 72, 1 (1997), 75.

[5]

Sasha A Barab and Thomas Duffy. 2000. From practice fields to communities of practice. Theoretical foundations of learning environments 1, 1 (2000), 25--55.

[6]

Jeremy B Bernerth, H Jack Walker, and Stanley G Harris. 2011. Change fatigue: Development and initial validation of a new measure. Work & Stress 25, 4 (2011), 321--337.

[7]

Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, and Katrina Panovich. 2015. Soylent: A Word Processor with a Crowd Inside. Commun. ACM 58,

Digital Library

[8]

(July 2015), 85--94. 8. Omar Besbes, Yonatan Gur, and Assaf Zeevi. 2015. Non-stationary stochastic optimization. Operations research 63, 5 (2015), 1227--1244.

[9]

Peter Michael Blau. 1974. On the nature of organizations. John Wiley & Sons.

[10]

J Stuart Bunderson and Peter Boumgarden. 2010. Structure and learning in self-managed teams: Why "bureaucratic" teams can be better learners. Organization Science 21, 3 (2010), 609--624.

Digital Library

[11]

Tom E Burns and George Macpherson Stalker. 1961. The management of innovation. (1961).

[12]

Stéphane Caron and Smriti Bhagat. 2013. Mixing bandits: A recipe for improved cold-start recommendations in a social network. In Proceedings of the 7th Workshop on Social Network Mining and Analysis. ACM.

Digital Library

[13]

Nicolo Cesa-Bianchi, Claudio Gentile, and Giovanni Zappella. 2013. A gang of bandits. In Advances in Neural Information Processing Systems. 737--745.

Digital Library

[14]

Joel Chan, Steven Dang, and Steven P Dow. 2016. Improving crowd innovation with expert facilitation. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 1223--1235.

Digital Library

[15]

Hans De Bruijn and Ernst et al. Ten Heuvelhof. 2010. Process management: why project management fails in complex decision making processes. Springer Science & Business Media.

[16]

C. K. W. De Dreu and M. A. West. 2001. Minority dissent and team innovation: The importance of participation in decision making. Journal of Applied Psychology 86, 6 (2001), 1191--1201.

[17]

Lex Donaldson. 1999. The normal science of structural contingency theory. SAGE Publications Ltd. 51--70 pages.

[18]

Paul Dourish and Victoria Bellotti. 1992. Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work. ACM, 107--114.

Digital Library

[19]

A. Edmondson. 1999. Psychological safety and learning behavior in work teams. Administrative Science Quarterly 44, 2 (1999), 350--383.

[20]

Thomas Erickson and Wendy A Kellogg. 2000. Social translucence: an approach to designing systems that support social processes. ACM transactions on computer-human interaction (TOCHI) 7, 1 (2000), 59--83.

Digital Library

[21]

Ethan Fast, Binbin Chen, and Michael S Bernstein. 2016. Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4647--4657.

Digital Library

[22]

Colin M Fisher. 2017. An ounce of prevention or a pound of cure? Two experiments on in-process interventions in decision-making groups. Organizational Behavior and Human Decision Processes 138 (2017), 59--73.

[23]

Victor Gabillon, Mohammad Ghavamzadeh, Alessandro Lazaric, and Sébastien Bubeck. 2011. Multi-bandit best arm identification. In Advances in Neural Information Processing Systems. 2222--2230.

Digital Library

[24]

Nicolas Galichet, Michele Sebag, and Olivier Teytaud. 2013. Exploration vs exploitation vs safety: Risk-aware multi-armed bandits. In Asian Conference on Machine Learning. 245--260.

[25]

R.T. Golembiewski. 2000. Handbook of Organizational Behavior, Second Edition, Revised and Expanded. Taylor and Francis.

[26]

Google. 2016. re:Work with Google. https://rework. withgoogle.com/print/guides/5721312655835136/. (2016).

[27]

Alvin Ward Gouldner. 1959. Organizational analysis. Basic Books.

[28]

Carl Gutwin and Saul Greenberg. 2002. A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work (CSCW) 11, 3 (2002), 411--446.

Digital Library

[29]

J Richard Hackman. 2012. From causes to conditions in group research. Journal of Organizational Behavior 33, 3 (2012), 428--444.

[30]

Nathan Hahn, Joseph Chang, Ji Eun Kim, and Aniket Kittur. 2016. The Knowledge Accelerator: Big picture thinking in small pieces. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2258--2270.

Digital Library

[31]

Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 119--125.

Digital Library

[32]

Daniel R Ilgen, John R Hollenbeck, Michael Johnson, and Dustin Jundt. 2005. Teams in organizations: From input-process-output models to IMOI models. Annu. Rev. Psychol. 56 (2005), 517--543.

[33]

Farnaz Jahanbakhsh, Wai-Tat Fu, Karrie Karahalios, Darko Marinov, and Brian Bailey. 2017. You Want Me to Work with Who?: Stakeholder Perceptions of Automated Team Formation in Project-based Courses. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3201--3212.

Digital Library

[34]

Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E Kraut. 2011. Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 43--52.

Digital Library

[35]

Aniket Kittur, Bongwon Suh, Bryan A Pendleton, and Ed H Chi. 2007. He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 453--462.

Digital Library

[36]

Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012. Is this what you meant?: promoting listening on the web with reflect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1559--1568.

Digital Library

[37]

Paul R Lawrence and Jay W Lorsch. 1967. Differentiation and integration in complex organizations. Administrative science quarterly (1967), 1--47.

[38]

Ioanna Lykourentzou, Robert E. Kraut, and Steven P. Dow. 2017. Team Dating Leads to Better Online Ad Hoc Collaborations. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 2330--2343.

Digital Library

[39]

Thomas W Malone, Kevin Crowston, and George Arthur Herman. 2003. Organizing business knowledge: The MIT process handbook. MIT press.

Digital Library

[40]

Andrew Mao, Winter Mason, Siddharth Suri, and Duncan J Watts. 2016. An experimental study of team size and performance on a complex task. PloS one 11, 4 (2016).

[41]

Michelle A Marks, John E Mathieu, and Stephen J Zaccaro. 2001. A temporally based framework and taxonomy of team processes. Academy of management review 26, 3 (2001), 356--376.

[42]

Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods 44, 1 (2012), 1--23.

[43]

Joseph Edward McGrath. 1984. Groups: Interaction and performance. Vol. 14. Prentice-Hall Englewood Cliffs, NJ.

[44]

Gerardo A Okhuysen and Mary J Waller. 2002. Focusing on midpoint transitions: An analysis of boundary conditions. Academy of Management Journal 45, 5 (2002), 1056--1065.

[45]

Gary M Olson and Judith S Olson. 2000. Distance matters. Human-computer interaction 15, 2 (2000), 139--178.

Digital Library

[46]

Derek Salman Pugh and David J Hickson. 1976. Organizational structure in its context. Saxon House Westmead, Farnborough.

[47]

Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter S Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine, and Michael S Bernstein. 2014. Expert crowdsourcing with flash teams. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 75--85.

Digital Library

[48]

Horst WJ Rittel and Melvin M Webber. 1973. Dilemmas in a general theory of planning. Policy sciences 4, 2 (1973), 155--169.

[49]

Sandra L Robinson and Anne M O'Leary-Kelly. 1998. Monkey see, monkey do: The influence of work groups on the antisocial behavior of employees. Academy of Management Journal 41, 6 (1998), 658--672.

[50]

Niloufar Salehi, Andrew McCabe, Melissa Valentine, and Michael Bernstein. 2017. Huddler: Convening Stable and Familiar Crowd Teams Despite Unpredictable Availability. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 1700--1713.

Digital Library

[51]

Jane E Salk and Mary Yoko Brannen. 2000. National culture, networks, and individual influence in a multinational management team. Academy of Management journal 43, 2 (2000), 191--202.

[52]

Amir Sani, Alessandro Lazaric, and Rémi Munos. 2012. Risk-aversion in multi-armed bandits. In Advances in Neural Information Processing Systems. 3275--3283.

Digital Library

[53]

Michaéla C Schippers, Amy C Edmondson, and Michael A West. 2014. Team reflexivity as an antidote to team information-processing failures. Small Group Research 45, 6 (2014), 731--769.

[54]

W.R. Scott and G.F. Davis. 2015. Organizations and Organizing: Rational, Natural and Open Systems Perspectives. Taylor & Francis.

[55]

Pao Siangliulue, Joel Chan, Steven P Dow, and Krzysztof Z Gajos. 2016. IdeaHound: Improving Large-scale Collaborative Ideation with Crowd-powered Real-time Semantic Modeling. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 609--624.

Digital Library

[56]

Jane Siegel, Vitaly Dubrovsky, Sara Kiesler, and Timothy W McGuire. 1986. Group processes in computer-mediated communication. Organizational behavior and human decision processes 37, 2 (1986), 157--187.

[57]

HA Simon. 1947. Administrative behavior; a study of decision-making processes in administrative organization. Macmillan.

[58]

Christine A Sprigg, Paul R Jackson, and Sharon K Parker. 2000. Production teamworking: The importance of interdependence and autonomy for employee strain and satisfaction. Human Relations 53, 11 (2000), 1519--1543.

[59]

Bradley R Staats, Katherine L Milkman, and Craig R Fox. 2012. The team scaling fallacy: Underestimating the declining efficiency of larger teams. Organizational Behavior and Human Decision Processes 118, 2 (2012), 132--142.

[60]

James D Thompson. 1967. Organizations in action: Social science bases of administrative theory. Transaction publishers.

[61]

H. H. M. Tse, M. T. Dasborough, and N. M. Ashkanasy. 2008. A multi-level analysis of team climate and interpersonal exchange relationships at work. Leadership Quarterly 19, 2 (2008), 195--211.

[62]

Rajan Vaish, Snehalkumar Neil S Gaikwad, Geza Kovacs, Andreas Veit, Ranjay Krishna, Imanol Arrieta Ibarra, Camelia Simoiu, Michael Wilber, Serge Belongie, Sharad Goel, James Davis, and Michael S Bernstein. 2017. Crowd Research: Open and Scalable University Laboratories. In Proceedings of the 32nd annual ACM Symposium on User Interface Software and Technology.

Digital Library

[63]

Sattar Vakili and Qing Zhao. 2016. Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure. IEEE Journal of Selected Topics in Signal Processing 10, 6 (2016), 1093--1111.

[64]

Gina Venolia, John Tang, Ruy Cervantes, Sara Bly, George Robertson, Bongshin Lee, and Kori Inkpen. 2010. Embodied social proxy: mediating interpersonal connection in hub-and-satellite teams. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1049--1058.

Digital Library

[65]

Fernanda B Viégas and Judith S Donath. 1999. Chat circles. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 9--16.

Digital Library

[66]

Ruth Wageman. 1995. Interdependence and group effectiveness. Administrative science quarterly (1995), 145--180.

[67]

Ruth Wageman and Frederick M Gordon. 2005. As the twig is bent: How group values shape emergent task interdependence in groups. Organization Science 16, 6 (2005), 687--700.

Digital Library

[68]

Ruth Wageman, J Richard Hackman, and Erin Lehman. 2005. Team diagnostic survey: Development of an instrument. The Journal of Applied Behavioral Science 41, 4 (2005), 373--398.

[69]

Lan Wang, Jian Han, Colin M Fisher, and Yan Pan. 2017. Learning to share: Exploring temporality in shared leadership and team learning. Small Group Research 48, 2 (2017), 165--189.

[70]

Terry Winograd. 1986. A language/action perspective on the design of cooperative work. In Proceedings of the 1986 ACM conference on Computer-supported cooperative work. ACM, 203--220.

Digital Library

[71]

Yingce Xia, Haifang Li, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2015. Thompson Sampling for Budgeted Multi-Armed Bandits. In IJCAI. 3960--3966.

Digital Library

[72]

Lixiu Yu, Aniket Kittur, and Robert E Kraut. 2014. Distributed analogical idea generation: inventing with crowds. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 1245--1254.

Digital Library

[73]

Amy X Zhang, Lea Verou, and David R Karger. 2017b. Wikum: Bridging Discussion Forums and Wikis Using Recursive Summarization. In CSCW. 2082--2096.

Digital Library

[74]

Haoqi Zhang, Matthew W Easterday, Elizabeth M Gerber, Daniel Rees Lewis, and Leesha Maliakal. 2017a. Agile Research Studios: Orchestrating Communities of Practice to Advance Research Training. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 45--48.

Digital Library

Cited By

Ferguson SVan De Zande GOlechowski A(2024)No Risk, No Reward: Towards An Automated Measure of Psychological Safety from Online CommunicationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650923(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650923
Hao JYi WRen MAi CQi TGuo L(2024)Clustering Based Collaborative Learning Grouping for Knowledge BuildingWeb and Big Data10.1007/978-981-97-7244-5_14(210-223)Online publication date: 28-Aug-2024
https://doi.org/10.1007/978-981-97-7244-5_14
Correia AGrover ASchneider DPimentel AChaves Rde Almeida MFonseca B(2023)Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine InteractionApplied Sciences10.3390/app1304219813:4(2198)Online publication date: 8-Feb-2023
https://doi.org/10.3390/app13042198
Show More Cited By

Index Terms

In Search of the Dream Team: Temporally Constrained Multi-Armed Bandits for Identifying Effective Team Structures
1. Human-centered computing
  1. Collaborative and social computing

Recommendations

A "Distance Matters" Paradox: Facilitating Intra-Team Collaboration Can Harm Inter-Team Collaboration
CSCW1

By identifying the socio-technical conditions required for teams to work effectively remotely, the Distance Matters framework has been influential in CSCW since its introduction in 2000. Advances in collaboration technology and practices have since ...
Automatic search for related-key differential characteristics in byte-oriented block ciphers: application to AES, Camellia, Khazad and others
EUROCRYPT'10: Proceedings of the 29th Annual international conference on Theory and Applications of Cryptographic Techniques

While differential behavior of modern ciphers in a single secret key scenario is relatively well understood, and simple techniques for computation of security lower bounds are readily available, the security of modern block ciphers against related-key ...
Forms of collaboration in high performance computing: exploring implications for learning
CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work

Successful collaboration is not only an occasion for the accomplishment of shared goals, but also provides opportunities for individual collaborators to learn from each other. Extended interaction allows for participants to resolve personal and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

DARPA
National Science Foundation
Office of Naval Research

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,888
Total Downloads

Downloads (Last 12 months)338
Downloads (Last 6 weeks)38

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ferguson SVan De Zande GOlechowski A(2024)No Risk, No Reward: Towards An Automated Measure of Psychological Safety from Online CommunicationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650923(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650923
Hao JYi WRen MAi CQi TGuo L(2024)Clustering Based Collaborative Learning Grouping for Knowledge BuildingWeb and Big Data10.1007/978-981-97-7244-5_14(210-223)Online publication date: 28-Aug-2024
https://doi.org/10.1007/978-981-97-7244-5_14
Correia AGrover ASchneider DPimentel AChaves Rde Almeida MFonseca B(2023)Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine InteractionApplied Sciences10.3390/app1304219813:4(2198)Online publication date: 8-Feb-2023
https://doi.org/10.3390/app13042198
Park SKulkarni C(2023)Retrospector: Rapid collaborative reflection to improve collaborative practicesProceedings of the ACM on Human-Computer Interaction10.1145/36100847:CSCW2(1-20)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610084
Chiang CLu ZLi ZYin M(2023)Are Two Heads Better Than One in AI-Assisted Decision Making? Comparing the Behavior and Performance of Groups and Individuals in Human-AI Collaborative Recidivism Risk AssessmentProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581015(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581015
Yin BLiu YXu BChen HTang S(2023)TTAF: A two-tier task assignment framework for cooperative unit-based crowdsourcing systemsJournal of Network and Computer Applications10.1016/j.jnca.2023.103719218(103719)Online publication date: Sep-2023
https://doi.org/10.1016/j.jnca.2023.103719
Zhao MEadeh FNguyen TGupta PAdmoni HGonzalez CWoolley A(2023)Teaching agents to understand teamworkComputers in Human Behavior10.1016/j.chb.2022.107524139:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.chb.2022.107524
Vinella FHu JLykourentzou IMasthoff J(2022)Crowdsourcing Team Formation With Worker-Centered ModelingFrontiers in Artificial Intelligence10.3389/frai.2022.8185625Online publication date: 27-May-2022
https://doi.org/10.3389/frai.2022.818562
Jain RGarg NKhera S(2022)Adoption of AI-Enabled Tools in Social Development Organizations in India: An Extension of UTAUT ModelFrontiers in Psychology10.3389/fpsyg.2022.89369113Online publication date: 20-Jun-2022
https://doi.org/10.3389/fpsyg.2022.893691
Gómez-Zará DDas APawlow BContractor N(2022)In search of diverse and connected teams: A computational approach to assemble diverse teams based on members’ social networksPLOS ONE10.1371/journal.pone.027606117:11(e0276061)Online publication date: 9-Nov-2022
https://doi.org/10.1371/journal.pone.0276061
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents