skip to main content
research-article
Public Access

Threading is Sticky: How Threaded Conversations Promote Comment System User Retention

Published:06 December 2017Publication History
Skip Abstract Section

Abstract

The Guardian ---the fifth most widely read online newspaper in the world as of 2014---changed conversations on its commenting platform by altering its design from non-threaded to single-level threaded in 2012. We studied this naturally occurring experiment to investigate the impact of conversation threading on user retention as mediated by several potential changes in conversation structure and style. Our analysis shows that the design change made new users significantly more likely to comment a second time, and that this increased stickiness is due in part to a higher fraction of comments receiving responses after the design change. In mediation analysis, other anticipated mechanisms such as reciprocal exchanges and comment civility did not help to explain users' decision to return to the commenting system; indeed, civility did not increase after the design change and reciprocity declined. These analyses show that even simple design choices can have a significant impact on news forums' stickiness. Further, they suggest that this influence is more powerfully shaped by affordances---the new system made responding easier---than by changes in users' attention to social norms of reciprocity or civility. This has an array of implications for designers.

Skip Supplemental Material Section

Supplemental Material

References

  1. Ashley A. Anderson, Dominique Brossard, Dietram A. Scheufele, Michael A. Xenos, and Peter Ladwig. 2014. The Nasty Effect: Online Incivility and Risk Perceptions of Emerging Technologies. Journal of Computer-Mediated Communication Vol. 19, 3 (2014), 373--387. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Pablo Aragón, Vicencc Gómez, and Andreas Kaltenbrunner. 2017. To Thread or Not to Thread: The Impact of Conversation Threading on Online Discussion Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15--18, 2017. AAAI Press, 12--21. https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15609Google ScholarGoogle Scholar
  3. Jaime Arguello, Brian S. Butler, Elisabeth Joyce, Robert Kraut, Kimberly S. Ling, Carolyn Rosé, and Xiaoqing Wang. 2006. Talk to me: foundations for successful individual-group interactions in online communities. In CHI '06. ACM, 959--968. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Patrick J. Bateman, Peter H. Gray, and Brian S. Butler. 2006. Community Commitment: How Affect, Obligation, and Necessity Drive Online Behaviors Proceedings of the International Conference on Information Systems, ICIS 2006, Milwaukee, Wisconsin, USA, December 10--13, 2006. Association for Information Systems, 63. http://aisel.aisnet.org/icis2006/63Google ScholarGoogle Scholar
  5. Gerard Beenen, Kimberly S. Ling, Xiaoqing Wang, Klarissa Chang, Dan Frankowski, Paul Resnick, and Robert E. Kraut. 2004. Using social psychology to motivate contributions to online communities Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, CSCW 2004, Chicago, Illinois, USA, November 6--10, 2004. ACM, 212--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Martin Belam. 2012. An experiment with threaded comments. (29 February. 2012). https://www.theguardian.com/help/insideguardian/2012/feb/29/threaded-commentsGoogle ScholarGoogle Scholar
  7. Roy Bendor, Susanna Haas Lyons, and John Robinson. 2012. What's there not to 'like'' sustainability deliberations on facebook. JeDEM-eJournal of eDemocracy and Open Government, Vol. 4, 1 (2012), 67--88.Google ScholarGoogle ScholarCross RefCross Ref
  8. George Berry and Sean J. Taylor. 2017. Discussion Quality Diffuses in the Digital Public Square Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3--7, 2017. ACM, 1371--1380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Wolfgang Blau. 2015. Welcome to the new Guardian website. https://www.theguardian.com/help/insideguardian/2015/jan/28/welcome-to-the-new-guardian-website. (2015).Google ScholarGoogle Scholar
  10. Petter Bae Brandtzæg and Jan Heim. 2008. User loyalty and online communities: why members of online communities are not faithful 2nd International Conference on INtelligent TEchnologies for interactive enterTAINment, INTETAIN 2008, Cancun, Mexico, January 8--10, 2008. ICST/ACM, 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Ceren Budak and Rakesh Agrawal. 2013. On Participation in Group Chats on Twitter. In Proceedings of the 22Nd International Conference on World Wide Web (WWW '13). ACM, New York, NY, USA, 165--176. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Pew Research Center. 2016. The Political Environment on Social Media. Technical Report. Pew Research Center.Google ScholarGoogle Scholar
  13. Calvin M. L. Chan, Mamata Bhandar, Lih-Bin Oh, and Hock-Chuan Chan. 2004. Recognition and Participation in a Virtual Community HICSS '04. IEEE Computer Society, 70194.2--. http://dl.acm.org/citation.cfm?id=962755.963100 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Yan Chen, Rosta Farzan, Robert Kraut, Iman YeckehZaare, and Ark Fangzhou Zhang. 2017. ExpertIdeas: Incentivizing Domain Experts to Contribute to Wikipedia. (2017).Google ScholarGoogle Scholar
  15. Coye Cheshire and Judd Antin. 2008. The social psychological effects of feedback on the production of Internet information pools. Journal of Computer-Mediated Communication Vol. 13, 3 (2008), 705--727.Google ScholarGoogle ScholarCross RefCross Ref
  16. Daegon Cho and K. Hazel Kwon. 2015. The impacts of identity verification and disclosure of social cues on flaming in online user comments. Computers in Human Behavior Vol. 51, Part A (2015), 363--372. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Robert B. Cialdini and Noah J. Goldstein. 2004. Social Influence: Compliance and Conformity. Annual Review of Psychology Vol. 55, 1 (2004), 591--621.Google ScholarGoogle ScholarCross RefCross Ref
  18. Kevin Coe, Kate Kenski, and Stephen A. Rains. 2014. Online and Uncivil? Patterns and Determinants of Incivility in Newspaper Website Comments. Journal of Communication Vol. 64, 4 (2014), 658--679.Google ScholarGoogle ScholarCross RefCross Ref
  19. Dan Cosley, Dan Frankowski, Loren G. Terveen, and John Riedl. 2007. SuggestBot: using intelligent task routing to help people find work in wikipedia Proceedings of the 12th International Conference on Intelligent User Interfaces, IUI 2007, Honolulu, Hawaii, USA, January 28--31, 2007. ACM, 32--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kushal Dave, Martin Wattenberg, and Michael Muller. 2004. Flash Forums and forumReader: Navigating a New Kind of Large-scale Online Discussion Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW '04). ACM, New York, NY, USA, 232--241. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Nicholas Diakopoulos and Mor Naaman. 2011. Towards Quality Discourse in Online News Comments. Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (CSCW '11). ACM, New York, NY, USA, 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Tawanna R. Dillahunt and Jennifer Mankoff. 2014. Understanding Factors of Successful Engagement Around Energy Consumption Between and Among Households. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, New York, NY, USA, 1246--1257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Justin Ellis. 2015. What happened after 7 news sites got rid of reader comments. (November 7. 2015). http://www.niemanlab.org/2015/09/what-happened-after-7-news-sites-got-rid-of-reader-comments/Google ScholarGoogle Scholar
  24. Rosta Farzan, Laura A. Dabbish, Robert E. Kraut, and Tom Postmes. 2011. Increasing Commitment to Online Communities by Designing for Social Presence Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (CSCW '11). ACM, New York, NY, USA, 321--330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Julian Fitzell. 2013. Commenting now available on our mobile website. https://www.theguardian.com/help/2013/nov/18/commenting-now-available-on-our-mobile-website. (2013).Google ScholarGoogle Scholar
  26. David Fono and Ron Baecker. 2006. Structuring and Supporting Persistent Chat Conversations Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06). ACM, New York, NY, USA, 455--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Rolf Fredheim, Alfred Moore, and John Naughton. 2015. Anonymity and Online Commenting: The Broken Windows Effect and the End of Drive-by Commenting Proceedings of the ACM Web Science Conference, WebSci 2015, Oxford, United Kingdom, June 28 - July 1, 2015. ACM, 11:1--11:8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Deen Freelon. 2015. Discourse architecture, ideology, and democratic norms in online political discussion. New Media & Society, Vol. 17, 5 (2015), 772--791.Google ScholarGoogle ScholarCross RefCross Ref
  29. Dennis Friess and Christiane Eilders. 2015. A systematic review of online deliberation research. Policy & Internet, Vol. 7, 3 (2015), 319--339.Google ScholarGoogle Scholar
  30. Diego Garlaschelli and Maria I Loffredo. 2004. Patterns of link reciprocity in directed networks. Physical review letters Vol. 93, 26 (2004), 268--701.Google ScholarGoogle Scholar
  31. Werner Geyer, Andrew J. Witt, Eric Wilcox, Michael Muller, Bernard Kerr, Beth Brownholtz, and David R. Millen. 2004. Chat Spaces Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '04). ACM, New York, NY, USA, 333--336. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Douglas Gunzler, Tian Chen, Pan Wu, and Hui Zhang. 2013. Introduction to mediation analysis with structural equation modeling. Shanghai archives of psychiatry Vol. 25, 6 (2013), 390.Google ScholarGoogle Scholar
  33. Natalie Hanman. 2012. Threading arrives on Comment is free. (3 December. 2012). https://www.theguardian.com/commentisfree/2012/dec/03/threading-arrives-on-comment-is-freeGoogle ScholarGoogle Scholar
  34. Cliff Lampe and Erik Johnston. 2005. Follow the (slash) dot: effects of feedback on new members in an online community Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work. ACM Press, Sanibel Island, Florida, USA, 11--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Cliff Lampe, Rick Wash, Alcides Velasquez, and Elif Ozkaya. 2010. Motivations to Participate in Online Communities. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 1927--1936. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Eun-Ju Lee. 2012. That's Not the Way It Is: How User-Generated Comments on the News Affect Perceived Media Bias. Journal of Computer-Mediated Communication Vol. 18, 1 (2012), 32--45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Bella Mackie. 2012. A threading experiment on Comment is free. (March 12. 2012). https://www.theguardian.com/commentisfree/2012/mar/12/threading-experiment-comment-is-freeGoogle ScholarGoogle Scholar
  38. David P MacKinnon, Amanda J Fairchild, and Matthew S Fritz. 2007. Mediation Analysis. Annual Review of Psychology Vol. 58 (2007), 593.Google ScholarGoogle Scholar
  39. Momin M. Malik and Jürgen Pfeffer. 2016. Identifying Platform Effects in Social Media Data. Proceedings of the Tenth International Conference on Web and Social Media, Cologne, Germany, May 17--20, 2016. AAAI Press, 241--249. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13163Google ScholarGoogle Scholar
  40. Edith Manosevitch, Nili Steinfeld, and Azi Lev-On. 2014. Promoting online deliberation quality: cognitive cues matter. Information, Communication & Society Vol. 17, 10 (2014), 1177--1195.Google ScholarGoogle ScholarCross RefCross Ref
  41. Adrienne Massanari. 2017. #Gamergate and The Fappening: How Reddit's algorithm, governance, and culture support toxic technocultures. New Media & Society Vol. 19, 3 (2017), 329--346.Google ScholarGoogle ScholarCross RefCross Ref
  42. Diana C. Mutz. 2006. Hearing the other side: deliberative versus participatory democracy. Cambridge University Press, New York.Google ScholarGoogle Scholar
  43. Donald A. Norman. 1990. The design of everyday things. Doubleday, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Deok Gun Park, Simranjit Singh Sachar, Nicholas Diakopoulos, and Niklas Elmqvist. 2016. Supporting Comment Moderators in Identifying High Quality Online News Comments Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, May 7--12, 2016. ACM, 1114--1125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates Vol. 71 (2001), 2001.Google ScholarGoogle Scholar
  46. Robert D. Putnam. 1995. Bowling alone: America's declining social capital. Journal of Democracy, Vol. 6, 1 (1995), 65--78.Google ScholarGoogle ScholarCross RefCross Ref
  47. Stuart Reeves and Barry Brown. 2016. Embeddedness and Sequentiality in Social Media. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 1052--1064. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. S. D. Reicher, R. Spears, and T. Postmes. 1995. A Social Identity Model of Deindividuation Phenomena. European Review of Social Psychology Vol. 6, 1 (1995), 161--198.Google ScholarGoogle ScholarCross RefCross Ref
  49. June W Rhee and Eun-mee Kim. 2009. Deliberation on the net: Lessons from a field experiment. Online deliberation: Design, research, and practice, bibfieldeditorTodd Davies and Seeta Peśa Gangadharan (Eds.). CSLI Publications, Standford, CA, 223--232.Google ScholarGoogle Scholar
  50. Yves Rosseel. 2012. Lavaan: an R package for structural equation modeling. JOURNAL OF STATISTICAL SOFTWARE Vol. 48, 2 (2012), 1--36.Google ScholarGoogle ScholarCross RefCross Ref
  51. Marie K. Shanahan. 2013. More news organizations try civilizing online comments with the help of social media. (2013). http://www.poynter.org/news/mediawire/218284/more-news-organizations-try-civilizing-online-comments-with-the-help-of-social-media/Google ScholarGoogle Scholar
  52. Leandro Araújo Silva, Mainack Mondal, Denzil Correa, Fabr'ıcio Benevenuto, and Ingmar Weber. 2016. Analyzing the Targets of Hate in Online Social Media Proceedings of the Tenth International Conference on Web and Social Media, Cologne, Germany, May 17--20, 2016. AAAI Press, 687--690. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13147Google ScholarGoogle Scholar
  53. Marc Smith, Jonathan J. Cadiz, and Byron Burkhalter. 2000. Conversation trees and threaded chats. In CSCW 2000, Proceeding on the ACM 2000 Conference on Computer Supported Cooperative Work, Philadelphia, PA, USA, December 2--6, 2000. ACM, 97--105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Marc A. Smith and Andrew T. Fiore. 2001. Visualization Components for Persistent Conversations Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 136--143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Lee Sproull and Sara Kiesler. 1998. Connections: new ways of working in the networked organization. The MIT Press, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Tiziano Squartini, Francesco Picciolo, Franco Ruzzenenti, and Diego Garlaschelli. 2013. Reciprocity of weighted networks. Scientific Reports Vol. 3 (2013), 2729.Google ScholarGoogle ScholarCross RefCross Ref
  57. Natalie Jomini Stroud, Joshua M. Scacco, Ashley Muddiman, and Alexander L. Curry. 2015. Changing Deliberative Norms on News Organizations' Facebook Sites. Journal of Computer-Mediated Communication Vol. 20, 2 (2015), 188--203.Google ScholarGoogle ScholarCross RefCross Ref
  58. The Guardian. 2009 a. Community standards and participation guidelines. (2009). https://www.theguardian.com/community-standardsGoogle ScholarGoogle Scholar
  59. The Guardian. 2009 b. Frequently asked questions about community on the Guardian website. (2009). https://www.theguardian.com/community-faqsGoogle ScholarGoogle Scholar
  60. The Guardian. 2017. Inside the Guardian blog. https://www.theguardian.com/help/insideguardian. (2017).Google ScholarGoogle Scholar
  61. Hannah Waldram. 2012. An update to news comments - what do you think? (22 November. 2012). https://www.theguardian.com/news/blog/2012/nov/22/news-community-comments-threadingGoogle ScholarGoogle Scholar
  62. Kevin Wise, Brian Hamman, and Kjerstin Thorson. 2006. Moderation, Response Rate, and Message Interactivity: Features of Online Communities and Their Effects on Intent to Participate. Journal of Computer-Mediated Communication Vol. 12, 1 (2006), 24--41.Google ScholarGoogle ScholarCross RefCross Ref
  63. Ka-Ping Yee. 2002. Zest: Discussion mapping for mailing lists. (2002). http://zesty.ca/pubs/cscw-2002-zest.pdfGoogle ScholarGoogle Scholar

Index Terms

  1. Threading is Sticky: How Threaded Conversations Promote Comment System User Retention

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!