skip to main content
research-article

Automatically Analyzing Brainstorming Language Behavior with Meeter

Published:07 November 2019Publication History
Skip Abstract Section

Abstract

Language both influences and indicates group behavior, and we need tools that let us study the content of what is communicated. While one could annotate these spoken dialogue acts by hand, this is a tedious, not scalable process. We present Meeter, a tool for automatically detecting information sharing, shared understanding, word counts, and group activation in spoken interactions. The contribution of our work is two-fold: (1) We validated the tool by showing that the measures computed by Meeter align with human-generated labels, and (2) we demonstrated the value of Meeter as a research tool by quantifying aspects of group behavior using those measures and deriving novel findings from that. Our tool is valuable for researchers conducting group science, as well as those designing groupware systems.

References

  1. Michael Baker, Tia Hansen, Richard Joiner, and David Traum. 1999. The role of grounding in collaborative learning tasks. Collaborative learning: Cognitive and computational approaches 31 (1999), 63.Google ScholarGoogle Scholar
  2. Tony Bergstrom and Karrie Karahalios. 2007. Conversation Clock: Visualizing audio patterns in co-located groups. In System Sciences, 2007. HICSS 2007. 40th Annual Hawaii International Conference on. IEEE, 78--78.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al. 2005. The AMI meeting corpus: A pre-announcement. In International Workshop on Machine Learning for Multimodal Interaction. Springer, 28--39.Google ScholarGoogle Scholar
  4. Senthil Chandrasegaran, Chris Bryan, Hidekazu Shidara, Tung-Yen Chuang, and Kwan-Liu Ma. 2019. TalkTraces: Real-Time Capture and Visualization of Verbal Content in Meetings. In Proceedings of ACM CHI Conference on Human Factors in Computing Systems (CHI 2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of machine learning research 12, Aug (2011), 2493--2537.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Gregorio Convertino, Helena M Mentis, Mary Beth Rosson, John M Carroll, Aleksandra Slavkovic, and Craig H Ganoe. 2008. Articulating common ground in cooperative work: content and process. In proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1637--1646.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Carsten KW De Dreu, Bernard A Nijstad, and Daan van Knippenberg. 2008. Motivated information processing in group judgment and decision making. Personality and Social Psychology Review 12, 1 (2008), 22--49.Google ScholarGoogle ScholarCross RefCross Ref
  8. Barbara Di Eugenio, Pamela W Jordan, Richmond H Thomason, and Johanna D Moore. 2000. The agreement process: An empirical investigation of human--human computer-mediated collaborative dialogs. International Journal of Human-Computer Studies 53, 6 (2000), 1017--1076.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Joan Morris DiMicco, Katherine J Hollenbach, Anna Pandolfo, and Walter Bender. 2007. The impact of increased awareness while face-to-face. Human-Computer Interaction 22, 1 (2007), 47--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gijsbert Erkens and Jeroen Janssen. 2008. Automatic coding of dialogue acts in collaboration protocols. International journal of computer-supported collaborative learning 3, 4 (2008), 447--470.Google ScholarGoogle ScholarCross RefCross Ref
  11. Florian Eyben, Klaus R Scherer, Björn W Schuller, Johan Sundberg, Elisabeth André, Carlos Busso, Laurence Y Devillers, Julien Epps, Petri Laukka, and Shrikanth S Narayanan. 2016. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE Transactions on Affective Computing 7, 2 (2016), 190--202.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 18th ACM international conference on Multimedia. ACM, 1459--1462.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Connie JG Gersick. 1988. Time and transition in work teams: Toward a new model of group development. Academy of Management journal 31, 1 (1988), 9--41.Google ScholarGoogle ScholarCross RefCross Ref
  14. Arthur C Graesser, Mark W Conley, and Andrew Olney. 2012. Intelligent tutoring systems. (2012).Google ScholarGoogle Scholar
  15. Saul Greenberg and Michael Rounding. 2001. The notification collage: posting information to public and personal displays. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 514--521.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gahgene Gweon, Soojin Jun, Joonhwan Lee, Susan Finger, and Carolyn Penstein Rosé. 2011. A framework for assessment of student project groups on-line and off-line. In Analyzing interactions in CSCL. Springer, 293--317.Google ScholarGoogle Scholar
  17. Verlin B Hinsz, R Scott Tindale, and David A Vollrath. 1997. The emerging conceptualization of groups as information processors. Psychological bulletin 121, 1 (1997), 43.Google ScholarGoogle Scholar
  18. Jesse Hoey, Tobias Schröder, Jonathan Morgan, Kimberly B Rogers, Deepak Rishi, and Meiyappan Nagappan. 2018. Artificial Intelligence and Social Simulation: Studying Group Dynamics on a Massive Scale. Small Group Research 49, 6 (2018), 647--683.Google ScholarGoogle ScholarCross RefCross Ref
  19. David H Jonassen and Hyug Kwon. 2001. Communication patterns in computer mediated versus face-to-face group problem solving. Educational technology research and development 49, 1 (2001), 35.Google ScholarGoogle Scholar
  20. Norbert L Kerr and Scott Tindale. 2014. Methods of small group research. (2014).Google ScholarGoogle Scholar
  21. Joseph Kim and Julie A Shah. 2016. Improving Team's Consistency of Understanding in Meetings. IEEE Transactions on Human-Machine Systems 46, 5 (2016), 625--637.Google ScholarGoogle ScholarCross RefCross Ref
  22. Taemie Kim, Agnes Chang, Lindsey Holland, and Alex Sandy Pentland. 2008. Meeting mediator: enhancing group collaboration using sociometric feedback. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 457--466. , Vol. 1, No. 1, Article . Publication date: June 2018.Google ScholarGoogle Scholar
  23. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014).Google ScholarGoogle Scholar
  24. Young Ji Kim, David Engel, Anita Williams Woolley, Jeffrey Yu-Ting Lin, Naomi McArthur, and Thomas W Malone. 2017. What Makes a Strong Team?: Using Collective Intelligence to Predict Team Performance in League of Legends.. In CSCW. 2316--2329.Google ScholarGoogle Scholar
  25. David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-László Barabási, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, et al. 2009. Computational social science. Science 323, 5915 (2009), 721--723.Google ScholarGoogle Scholar
  26. Linda Lebie, Jonathan A Rhoades, and Joseph E McGrath. 1995. Interaction process in computer-mediated and face-to-face groups. Computer Supported Cooperative Work (CSCW) 4, 2--3 (1995), 127--152.Google ScholarGoogle Scholar
  27. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827 (2016).Google ScholarGoogle Scholar
  28. Gilly Leshed, Jeffrey T Hancock, Dan Cosley, Poppy L McLeod, and Geri Gay. 2007. Feedback for guiding reflection on teamwork practices. In Proceedings of the 2007 international ACM conference on Supporting group work. ACM, 217--220.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Gilly Leshed, Diego Perez, Jeffrey T Hancock, Dan Cosley, Jeremy Birnholtz, Soyoung Lee, Poppy L McLeod, and Geri Gay. 2009. Visualizing real-time language-based feedback on teamwork behavior in computer-mediated groups. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 537--546.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Joseph F McCarthy, Ben Congleton, and F Maxwell Harper. 2008. The context, content & community collage: sharing personal digital media in the physical workplace. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 97--106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Joseph E McGrath. 1997. Small group research, that once and future field: An interpretation of the past with an eye to the future. Group Dynamics: Theory, Research, and Practice 1, 1 (1997), 7.Google ScholarGoogle ScholarCross RefCross Ref
  32. Kate G Niederhoffer and James W Pennebaker. 2002. Linguistic style matching in social interaction. Journal of Language and Social Psychology 21, 4 (2002), 337--360.Google ScholarGoogle ScholarCross RefCross Ref
  33. Eyal Ofek, Shamsi T Iqbal, and Karin Strauss. 2013. Reducing disruption from subtle information delivery during a conversation: mode and bandwidth investigation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3111--3120.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Michael J Owren and Jo-Anne Bachorowski. 2003. Reconsidering the evolution of non-linguistic communication: The case of laughter. Journal of Nonverbal Behavior 27, 3 (2003), 183--200.Google ScholarGoogle ScholarCross RefCross Ref
  35. Wei Pan, Wen Dong, Manuel Cebrian, Taemie Kim, James H Fowler, and Alex Sandy Pentland. 2012. Modeling dynamical influence in human interaction: Using data to make better inferences about influence within social systems. IEEE Signal Processing Magazine 29, 2 (2012), 77--86.Google ScholarGoogle ScholarCross RefCross Ref
  36. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001.Google ScholarGoogle Scholar
  37. Alex Pentland. 2004. Social dynamics: Signals and behavior. In Proceedings of the third international conference on developmental learning (ICDL'04). Salk Institute, San Diego. UCSD Institute for Neural Computation. 263--267.Google ScholarGoogle Scholar
  38. Yan Qu and Derek L Hansen. 2008. Building shared understanding in collaborative sense making. In Proceedings of CHI 2008 Sensemaking Workshop.Google ScholarGoogle Scholar
  39. Marc Schröder. 2003. Experimental study of affect bursts. Speech communication 40, 1--2 (2003), 99--116.Google ScholarGoogle Scholar
  40. J Bryan Sexton and Robert L Helmreich. 2000. Analyzing cockpit communications: the links between language, performance, error, and workload. Journal of Human Performance in Extreme Environments 5, 1 (2000), 6.Google ScholarGoogle Scholar
  41. Yang Shi, Yang Wang, Ye Qi, John Chen, Xiaoyao Xu, and Kwan-Liu Ma. 2017. IdeaWall: Improving creative collaboration through combinatorial visual stimuli. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 594--603.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Yla R Tausczik and James W Pennebaker. 2013. Improving teamwork using real-time language feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 459--468.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Ottokar Tilk and Tanel Alumäe. 2016. Bidirectional Recurrent Neural Network with Attention Mechanism for Punctuation Restoration. In Interspeech 2016.Google ScholarGoogle ScholarCross RefCross Ref
  44. Anita Williams Woolley, Christopher F Chabris, Alex Pentland, Nada Hashmi, and Thomas W Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. science 330, 6004 (2010), 686--688.Google ScholarGoogle Scholar

Index Terms

  1. Automatically Analyzing Brainstorming Language Behavior with Meeter

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!