Hive: Collective Design Through Network Rotation

Abstract

Collectives gather online around challenges they face, but frequently fail to envision shared outcomes to act on together. Prior work has developed systems for improving collective ideation and design by exposing people to each others' ideas and encouraging them to intermix those ideas. However, organizational behavior research has demonstrated that intermixing ideas does not result in meaningful engagement with those ideas. In this paper, we introduce a new class of collective design system that intermixes people instead of ideas: instead of receiving mere exposure to others' ideas, participants engage deeply with other members of the collective who represent those ideas, increasing engagement and influence. We thus present Hive: a system that organizes a collective into small teams, then intermixes people by rotating team membership over time. At a technical level, Hive must balance two competing forces: (1) networks are better at connecting diverse perspectives when network efficiency is high, but (2) moving people diminishes tie strength within teams. Hive balances these two needs through network rotation: an optimization algorithm that computes who should move where, and when. A controlled study compared network rotation to alternative rotation systems which maximize only tie strength or network efficiency, finding that network rotation produced higher-rated proposals. Hive has been deployed by Mozilla for a real-world open design drive to improve Firefox accessibility.

References

1. Aris Anagnostopoulos, Luca Becchetti, Carlos Castillo, Aristides Gionis, and Stefano Leonardi. 2012. Online team formation in social networks. In Proceedings of the 21st international conference on World Wide Web. ACM, 839--848.
2. Karen A Bantel and Susan E Jackson. 1989. Top management and innovations in banking: Does the composition of the top team make a difference? Strategic management journal , Vol. 10, S1 (1989), 107--124.
3. Michael S Bernstein, Jaime Teevan, Susan Dumais, Daniel Liebling, and Eric Horvitz. 2012. Direct answers for search queries in the long tail. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 237--246.
4. Osvald M Bjelland and Robert Chapman Wood. 2008. An inside view of IBM's `Innovation Jam'. MIT Sloan management review , Vol. 50, 1 (2008), 32.
5. Kevin J Boudreau and Karim R Lakhani. 2015. “Open” disclosure of innovations, incentives and follow-on reuse: Theory on processes of cumulative innovation and a field experiment in computational biology. Research Policy , Vol. 44, 1 (2015), 4--19.
6. Vincent Brown, Michael Tumeo, Timothy S Larey, and Paul B Paulus. 1998. Modeling cognitive interactions during group brainstorming. Small group research , Vol. 29, 4 (1998), 495--526.
7. Moira Burke and Robert E Kraut. 2014. Growing closer on facebook: changes in tie strength through social network site use. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 4187--4196.
8. Joel Chan, Steven Dang, and Steven P. Dow. 2016a. Comparing Different Sensemaking Approaches for Large-Scale Ideation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2717--2728.
9. Joel Chan, Steven Dang, and Steven P. Dow. 2016b. IdeaGens: Enabling Expert Facilitation of Crowd Brainstorming. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW '16 Companion). ACM, New York, NY, USA, 13--16.
10. Henry William Chesbrough. 2006. Open innovation: The new imperative for creating and profiting from technology .Harvard Business Press.
11. Hoon-Seok Choi and Leigh Thompson. 2005. Old wine in a new bottle: Impact of membership change on group creativity. Organizational Behavior and human decision processes , Vol. 98, 2 (2005), 121--132.
12. Marie A Cini. 2001. Group newcomers: From disruption to innovation. Group Facilitation 3 (2001), 3.
13. Mihaly Csikszentmihalyi. 1996. Flow and the psychology of discovery and invention. New Yprk: Harper Collins (1996).
14. Amy C Edmondson. 1999. Psychological Safety and Learning Behavior in Work Teams . Administrative Science Quarterly , Vol. 44, 2 (1999), 350.
15. Amy C Edmondson. 2002. Managing the risk of learning: Psychological safety in work teams . International Handbook of Organizational Teamwork, (2002), 1--38.
16. Amy C Edmondson. 2013. Teaming to innovate .John Wiley & Sons.
17. John E Ettlie. 1980. Manpower flows and the innovation process. Management Science , Vol. 26, 11 (1980), 1086--1095.
18. Craig L Feldbaum, Terry E Christenson, and Edgar C O'Neal. 1980. An observational study of the assimilation of the newcomer to the preschool. Child Development (1980), 497--507.
19. Herman Gadon. 1988. The newcomer and the ongoing work group. Team building: Blueprints for productivity and satisfaction (1988), 161--175.
20. DR Garrison. 2006. Online collaboration principles. Journal of Asynchronous Learning Networks , Vol. 10, 1 (2006), 25--34.
21. Eric Gilbert and Karrie Karahalios. 2009. Predicting tie strength with social media. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 211--220.
22. Deborah H Gruenfeld, Elizabeth A Mannix, Katherine Y Williams, and Margaret A Neale. 1996. Group composition and decision making: How member familiarity and information distribution affect process and performance. Organizational behavior and human decision processes , Vol. 67, 1 (1996), 1--15.
23. Jerold Heiss and Dennison Nash. 1967. The stranger in laboratory culture revisited. Human Organization , Vol. 26, 1--2 (1967), 47--51.
24. Joshua Introne, Robert Laubacher, Gary Olson, and Thomas Malone. 2011. The Climate CoLab: Large scale model-based collaborative planning. In Collaboration Technologies and Systems (CTS), 2011 International Conference on. IEEE, 40--47.
25. Susan E Jackson. 1996. The consequences of diversity in multidisciplinary work teams. Handbook of work group psychology (1996), 53--75.
26. Farnaz Jahanbakhsh, Wai-Tat Fu, Karrie Karahalios, Darko Marinov, and Brian Bailey. 2017. You Want Me to Work with Who?: Stakeholder Perceptions of Automated Team Formation in Project-based Courses. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3201--3212.
27. Ralph Katz and Thomas J. Allen. 1982. Investigating the Not Invented Here (NIH) syndrome: A look at the performance, tenure, and communication patterns of 50 R&D Project Groups. R&D Management , Vol. 12, 1 (1982), 7--20.
28. Joy Kim, Justin Cheng, and Michael S Bernstein. 2014. Ensemble: exploring complementary strengths of leaders and crowds in creative collaboration. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 745--755.
29. Joy Kim, Sarah Sterman, Allegra Argent Beal Cohen, and Michael S. Bernstein. 2017. Mechanical Novel: Crowdsourcing Complex Work Through Reflection and Revision. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 233--245.
30. Mark Klein and Ana Cristina Bicharra Garcia. 2015. High-speed idea filtering with the bag of lemons. Decision Support Systems , Vol. 78 (2015), 39--50.
31. Anand Kulkarni, Prayag Narula, David Rolnitzky, and Nathan Kontny. 2014. Wish: Amplifying creative ability with expert crowds. In Second AAAI Conference on Human Computation and Crowdsourcing .
32. Karim R Lakhani, Hila Lifshitz-Assaf, and Michael Tushman. 2013. Open innovation and organizational boundaries: task decomposition, knowledge distribution and the locus of innovation. Handbook of economic organization: Integrating economic and organizational theory (2013), 355--382.
33. Theodoros Lappas, Kun Liu, and Evimaria Terzi. 2009. Finding a team of experts in social networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 467--476.
34. Vito Latora and Massimo Marchiori. 2001. Efficient behavior of small-world networks. Physical review letters , Vol. 87, 19 (2001), 198701.
35. Diane Wei Liang, Richard Moreland, and Linda Argote. 1995. Group versus individual training and group performance: The mediating role of transactive memory. Personality and Social Psychology Bulletin , Vol. 21, 4 (1995), 384--393.
36. Hila Lifshitz-Assaf. 2014. From problem solvers to solution seekers: dismantling knowledge boundaries at NASA. (2014).
37. Xiaodong Lin and Daniel L Schwartz. 2003. Reflection at the crossroads of cultures. Mind, Culture, and Activity , Vol. 10, 1 (2003), 9--25.
38. Kurt Luther, Jari-Lee Tolentino, Wei Wu, Amy Pavel, Brian P Bailey, Maneesh Agrawala, Björn Hartmann, and Steven P Dow. 2015. Structuring, aggregating, and evaluating crowdsourced design critique. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 473--485.
39. Ioanna Lykourentzou, Angeliki Antoniou, Yannick Naudet, and Steven P Dow. 2016. Personality matters: Balancing for personality types leads to better outcomes for crowd teams. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 260--273.
40. Ioanna Lykourentzou, Robert E Kraut, and Steven P Dow. 2017. Team Dating Leads to Better Online Ad Hoc Collaborations.. In CSCW . 2330--2343.
41. Narges Mahyar, Michael R James, Michelle M Ng, Reginald A Wu, and Steven P Dow. 2018. CommunityCrit: Inviting the Public to Improve and Evaluate Urban Design Ideas through Micro-Activities. (2018).
42. Thomas W Malone, Robert Laubacher, and Chrysanthos Dellarocas. 2009. Harnessing crowds: Mapping the genome of collective intelligence. (2009).
43. Thomas W Malone, Jeffrey V Nickerson, Robert J Laubacher, Laur Hesse Fisher, Patrick De Boer, Yue Han, and Ben Towne. 2017. Putting the Pieces Back Together Again: Contest Webs for Large-Scale Problem Solving. (2017).
44. Peter V Marsden and Karen E Campbell. 1984. Measuring tie strength. Social forces , Vol. 63, 2 (1984), 482--501.
45. E Mcgrath. 1995. Methodology matters: Doing research in the behavioral and social sciences. In Readings in Human-Computer Interaction: Toward the Year 2000 (2nd ed .
46. Andrés Monroy-Hernández and Mitchel Resnick. 2008. Empowering kids to create and share programmable media. Interactions , Vol. 15, 2 (2008), 50--53.
47. Dennison J Nash and Alvin W Wolfe. 1957. The stranger in laboratory culture. American Sociological Review , Vol. 22, 4 (1957), 400--405.
48. Henry Navarro, Giovanna Miritello, Arturo Canales, and Esteban Moro. 2017. Temporal patterns behind the strength of persistent ties. EPJ Data Science , Vol. 6, 1 (2017), 31.
49. Charlan J Nemeth. 1986. Differential contributions of majority and minority influence. Psychological review , Vol. 93, 1 (1986), 23.
50. Charlan Jeanne Nemeth. 1992. Minority dissent as a stimulant to group performance. Group process and productivity , Vol. 95 (1992), 111.
51. Charlan Jeanne Nemeth and Barry M Staw. 1989. The tradeoffs of social control and innovation in groups and organizations. Advances in experimental social psychology , Vol. 22 (1989), 175--210.
52. Jeffrey V Nickerson. 2015. Collective design: remixing and visibility. In Design Computing and Cognition . Springer, 263--276.
53. Nicole Lynn Oxley, Mary T Dzindolet, and Paul B Paulus. 1996. The effects of facilitators on the performance of brainstorming groups. Journal of social behavior and personality , Vol. 11, 4 (1996), 633.
54. Paul B Paulus and Huei-Chuan Yang. 2000. Idea Generation in Groups: A Basis for Creativity in Organizations. Organizational Behavior and Human Decision Processes , Vol. 82, 1 (2000), 76 -- 87.
55. Fabrizio Perretti and Giacomo Negro. 2007. Mixing genres and matching people: A study in innovation and team composition in Hollywood. Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior , Vol. 28, 5 (2007), 563--586.
56. Hasso Plattner. 2010. An introduction to design thinking process guide. The Institute of Design at Stanford: Stanford (2010).
57. James L Price. 1977. The study of turnover .Iowa State Press.
58. Ray Reagans, Linda Argote, and Daria Brooks. 2005. Individual experience and experience working together: Predicting learning rates from knowing who knows what and knowing how to work together. Management science , Vol. 51, 6 (2005), 869--881.
59. Christoph Riedl, Ivo Blohm, Jan Marco Leimeister, and Helmut Krcmar. 2010. Rating scales for collective intelligence in innovation communities: Why quick and easy decision making does not get it right. (2010).
60. Christoph Riedl and Anita Williams Woolley. 2017. Teams vs. Crowds: A Field Test of the Relative Contribution of Incentives, Member Ability, and Emergent Collaboration to Crowd-Based Problem Solving Performance. Academy of Management Discoveries , Vol. 3, 4 (2017), 382--403.
61. Floor Rink, Aimée A Kane, Naomi Ellemers, and Gerben Van der Vegt. 2013. Team receptivity to newcomers: Five decades of evidence and future research themes. The Academy of Management Annals , Vol. 7, 1 (2013), 247--293.
62. Floor A Rink and Naomi Ellemers. 2009. Temporary versus permanent group membership: How the future prospects of newcomers affect newcomer acceptance and newcomer influence. Personality and Social Psychology Bulletin , Vol. 35, 6 (2009), 764--775.
63. Everett M Rogers. 2010. Diffusion of innovations .Simon and Schuster.
64. Niloufar Salehi, Lilly C Irani, Michael S Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, et almbox. 2015. We are dynamo: Overcoming stalling and friction in collective action for crowd workers. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, 1621--1630.
65. Niloufar Salehi, Andrew McCabe, Melissa Valentine, and Michael Bernstein. 2017. Huddler: Convening Stable and Familiar Crowd Teams Despite Unpredictable Availability. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 1700--1713.
66. Pao Siangliulue, Kenneth C Arnold, Krzysztof Z Gajos, and Steven P Dow. 2015a. Toward collaborative ideation at scale: Leveraging ideas from others to generate more creative and diverse ideas. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 937--945.
67. Pao Siangliulue, Joel Chan, Steven P Dow, and Krzysztof Z Gajos. 2016. IdeaHound: Improving Large-scale Collaborative Ideation with Crowd-powered Real-time Semantic Modeling. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 609--624.
68. Pao Siangliulue, Joel Chan, Krzysztof Z Gajos, and Steven P Dow. 2015b. Providing timely examples improves the quantity and quality of generated ideas. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 83--92.
69. Enno Siemsen, Aleda V Roth, Sridhar Balasubramanian, and Gopesh Anand. 2009. The influence of psychological safety and confidence in knowledge on employee knowledge sharing. Manufacturing & Service Operations Management , Vol. 11, 3 (2009), 429--447.
70. Joachim Stempfle and Petra Badke-Schaub. 2002. Thinking in design teams-an analysis of team communication. Design studies , Vol. 23, 5 (2002), 473--496.
71. Robert I. Sutton and Andrew Hargadon. 1996. Brainstorming Groups in Context: Effectiveness in a Product Design Firm. Administrative Science Quarterly , Vol. 41, 4 (1996), 685--718. http://www.jstor.org/stable/2393872
72. Jerry O Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory modeling with collaborative design spaces. ACM Transactions on Graphics-TOG , Vol. 28, 5 (2009), 167.
73. Maryam Tohidi, William Buxton, Ronald Baecker, and Abigail Sellen. 2006. Getting the right design and the design right. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 1243--1252.
74. Helen Wauck, Yu-Chun Yen, Wai-Tat Fu, Elizabeth Gerber, Steven P Dow, and Brian P Bailey. 2017. From in the Class or in the Wild? Peers Provide Better Design Feedback Than External Crowds. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems .
75. Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 1433--1444.
76. Anbang Xu, Huaming Rao, Steven P Dow, and Brian P Bailey. 2015. A classroom study of using crowd feedback in the iterative design process. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1637--1648.
77. Robert M Yerkes and John D Dodson. 1908. The relation of strength of stimulus to rapidity of habit-formation. Journal of comparative neurology and psychology , Vol. 18, 5 (1908), 459--482.
78. Lixiu Yu, Aniket Kittur, and Robert E Kraut. 2014a. Distributed analogical idea generation: inventing with crowds. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 1245--1254.
79. Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014b. Searching for Analogical Ideas with Crowds. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1225--1234.
80. Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2016a. Encouraging “Outside- The- Box” Thinking in Crowd Innovation Through Identifying Domains of Expertise. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 1214--1222.
81. Lixiu Yu, Robert E. Kraut, and Aniket Kittur. 2016b. Distributed Analogical Idea Generation with Multiple Constraints. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 1236--1245.
82. Lixiu Yu and Jeffrey V Nickerson. 2011. Cooks or cobblers?: crowd creativity through combination. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1393--1402.
83. Kathryn Zyskowski, Meredith Ringel Morris, Jeffrey P Bigham, Mary L Gray, and Shaun K Kane. 2015. Accessible crowdwork?: Understanding the value in and challenge of microtask employment for people with disabilities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1682--1693.

Index Terms

• Human-centered computing

  • Collaborative and social computing

    • Collaborative and social computing theory, concepts and paradigms

      • Computer supported cooperative work