skip to main content
research-article

Who Has the Last Word? Understanding How to Sample Online Discussions

Authors Info & Claims
Published:03 June 2021Publication History
Skip Abstract Section

Abstract

In online debates, as in offline ones, individual utterances or arguments support or attack each other, leading to some subset of arguments (potentially from different sides of the debate) being considered more relevant than others. However, online conversations are much larger in scale than offline ones, with often hundreds of thousands of users weighing in, collaboratively forming large trees of comments by starting from an original post and replying to each other. In large discussions, readers are often forced to sample a subset of the arguments being put forth. Since such sampling is rarely done in a principled manner, users may not read all the relevant arguments to get a full picture of the debate from a sample. This article is interested in answering the question of how users should sample online conversations to selectively favour the currently justified or accepted positions in the debate. We apply techniques from argumentation theory and complex networks to build a model that predicts the probabilities of the normatively justified arguments given their location in idealised online discussions of comments and replies, which we represent as trees. Our model shows that the proportion of replies that are supportive, the distribution of the number of replies that comments receive, and the locations of comments that do not receive replies (i.e., the “leaves” of the reply tree) all determine the probability that a comment is a justified argument given its location. We show that when the distribution of the number of replies is homogeneous along the tree length, for acrimonious discussions (with more attacking comments than supportive ones), the distribution of justified arguments depends on the parity of the tree level, which is the distance from the root expressed as number of edges. In supportive discussions, which have more supportive comments than attacks, the probability of having justified comments increases as one moves away from the root. For discussion trees that have a non-homogeneous in-degree distribution, for supportive discussions we observe the same behaviour as before, while for acrimonious discussions we cannot observe the same parity-based distribution. This is verified with data obtained from the online debating platform Kialo. By predicting the locations of the justified arguments in reply trees, we can therefore suggest which arguments readers should sample, to grasp the currently accepted opinions in such discussions. Our models have important implications for the design of future online debating platforms.

References

  1. Pushkal Agarwal, Miriam Redi, Nishanth Sastry, Edward Wood, and Andrew Blick. 2020. Wikipedia and Westminster: Quality and dynamics of Wikipedia pages about UK politicians. In Proceedings of the 31st ACM Conference on Hypertext and Social Media. 161–166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Pushkal Agarwal, Nishanth Sastry, and Edward Wood. 2019. Tweeting MPs: Digital engagement between citizens and members of parliament in the UK. Proceedings of the International AAAI Conference on Web and Social Media. 26–37. Retrieved from https://www.aaai.org/ojs/index.php/ICWSM/article/view/3359.Google ScholarGoogle Scholar
  3. Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. J. Econ. Perspect. 31, 2 (2017), 211–36.Google ScholarGoogle ScholarCross RefCross Ref
  4. Leila Amgoud and Claudette Cayrol. 2002. Inferring from inconsistency in preference-based argumentation frameworks. J. Autom. Reason. 29, 2 (2002), 125–169. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Albert-László Barabási and Réka Albert. 1999. Emergence of scaling in random networks. Science 286, 5439 (1999), 509–512.Google ScholarGoogle Scholar
  6. Tom Bosc, Elena Cabrio, and Serena Villata. 2016. Tweeties squabbling: Positive and negative results in applying argument mining on social media. In Proceedings of the International Conference on Computational Models of Argument (COMMA’16). 21–32.Google ScholarGoogle Scholar
  7. Elena Cabrio and Serena Villata. 2012. Combining textual entailment and argumentation theory for supporting online debates interactions. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. 208–212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Elena Cabrio and Serena Villata. 2013. A natural language bipolar argumentation approach to support users in online debate interactions. Argument Comput. 4, 3 (2013), 209–230.Google ScholarGoogle ScholarCross RefCross Ref
  9. Claudette Cayrol and Marie-Christine Lagasquie-Schiex. 2005. On the acceptability of arguments in bipolar argumentation frameworks. In Proceedings of the European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty. Springer, 378–389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Claudette Cayrol and Marie-Christine Lagasquie-Schiex. 2013. Bipolarity in argumentation graphs: Towards a better understanding. Int. J. Approx. Reason. 54, 7 (2013), 876–899.Google ScholarGoogle ScholarCross RefCross Ref
  11. Oana Cocarascu and Francesca Toni. 2017. Identifying attack and support argumentative relations using deep learning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1374–1379.Google ScholarGoogle ScholarCross RefCross Ref
  12. Marcos Cramer and Mathieu Guillaume. 2018. Directionality of attacks in natural language argumentation. In CEUR Workshop Proceedings. RWTH Aachen University.Google ScholarGoogle Scholar
  13. Marcos Cramer and Mathieu Guillaume. 2019. Empirical study on human evaluation of complex argumentation frameworks. In Proceedings of the European Conference on Logics in Artificial Intelligence. Springer, 102–115.Google ScholarGoogle ScholarCross RefCross Ref
  14. Marcos Cramer and Leon van der Torre. 2019. SCF2-an argumentation semantics for rational human judgments on argument acceptability. In Proceedings of the 8th Workshop on Dynamics of Knowledge and Belief (DKB’19) and the 7th Workshop KI and Kognition (KIK’19) co-located with 44nd German Conference on Artificial Intelligence (KI’19). 24–35.Google ScholarGoogle Scholar
  15. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications. Synth. Lect. Hum. Lang. Technol. 6, 4 (2013), 1–220.Google ScholarGoogle ScholarCross RefCross Ref
  16. Sam H. Dekay. 2012. How large companies react to negative Facebook comments. Corp. Commun. Int. J. 17, 3 (2012), 289–299.Google ScholarGoogle ScholarCross RefCross Ref
  17. Nicholas Diakopoulos and Mor Naaman. 2011. Towards quality discourse in online news comments. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. ACM, 133–142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Phan Minh Dung. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming, and \(\)-person games. Artific. Intell. 77, 2 (1995), 321–357. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Paul E. Dunne and Michael Wooldridge. 2009. Complexity of abstract argumentation. In Argumentation in Artificial Intelligence. Springer, 85–104.Google ScholarGoogle Scholar
  20. Sandra. L. Emerson. 1983. Usenet—A bulletin board for Unix users. Byte 8, 10 (1983), 219.Google ScholarGoogle Scholar
  21. Erlin, Susandri, and Helda Yenni. 2016. Social network analysis for online discussion: Number of links vs. sum of weight. In Proceedings of the International Conference on Communication and Information Systems (ICCIS’16). Association for Computing Machinery, New York, NY, 82–86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Yong Gao. 2017. A random model for argumentation framework: Phase transitions, empirical hardness, and heuristics. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’17). 503–509. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Sherice Gearhart and Seok Kang. 2014. Social media in television news: The effects of Twitter and Facebook comments on journalism. Electron. News 8, 4 (2014), 243–259.Google ScholarGoogle ScholarCross RefCross Ref
  24. Vicenç Gómez, Hilbert J. Kappen, and Andreas Kaltenbrunner. 2011. Modeling the structure and evolution of discussion cascades. In Proceedings of the 22nd ACM Conference on Hypertext and Hypermedia. 181–190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Vicenç Gómez, Hilbert J. Kappen, Nelly Litvak, and Andreas Kaltenbrunner. 2013. A likelihood-based framework for the analysis of discussion threads. World Wide Web 16, 5–6 (2013), 645–675. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Jeffery Gottfried and Elisa Shearer. 2017. News Use Across Social Media Platforms. Retrieved from http://www.journalism.org/2017/09/07/news-use-across-social-media-platforms-2017/.Google ScholarGoogle Scholar
  27. Dmytro Karamshuk, Tetyana Lokot, Oleksandr Pryymak, and Nishanth Sastry. 2016. Identifying partisan slant in news articles and Twitter during political crises. In Proceedings of the International Conference on Social Informatics. Springer, 257–272.Google ScholarGoogle ScholarCross RefCross Ref
  28. Zsolt Katona. 2006. Levels of a scale-free tree. Random Struct. Algor. 29, 2 (2006), 194–207. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Joseph M. Kayany. 1998. Contexts of uninhibited online behavior: Flaming in social newsgroups on Usenet. J. Amer. Soc. Info. Sci. 49, 12 (1998), 1135–1141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Barbara Konat, John Lawrence, Joonsuk Park, Katarzyna Budzynska, and Chris Reed. 2016. A corpus of argument networks: Using graph properties to analyse divisive issues. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC’16). 3899–3906.Google ScholarGoogle Scholar
  31. Paul L. Krapivsky and Sidney Redner. 2001. Organization of growing random networks. Phys. Rev. E 63 (May 2001), 066123. Issue 6. DOI:https://doi.org/10.1103/PhysRevE.63.066123Google ScholarGoogle ScholarCross RefCross Ref
  32. Ravi Kumar, Mohammad Mahdian, and Mary McGlohon. 2010. Dynamics of conversations. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10). Association for Computing Machinery, New York, NY, 553–562. DOI:https://doi.org/10.1145/1835804.1835875 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. John Lawrence and Chris Reed. 2020. Argument mining: A survey. Comput. Linguist. 45, 4 (2020), 765–818.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Joao Leite and Joao Martins. 2011. Social abstract argumentation. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marco Lippi and Paolo Torroni. 2016. Argumentation mining: State of the art and emerging trends. ACM Trans. Internet Technol. 16, 2 (2016), 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Alexey N. Medvedev, Jean-Charles Delvenne, and Renaud Lambiotte. 2019. Modelling structure and predicting dynamics of discussion threads in online boards. J. Complex Netw. 7, 1 (2019), 67–82.Google ScholarGoogle ScholarCross RefCross Ref
  37. Johannes Müller and Christina Kuttler. 2015. Methods and Models in Mathematical Biology. Springer.Google ScholarGoogle Scholar
  38. Julia Murphy and Max Roser. 2019. Internet. Our World in Data. Retrieved from https://ourworldindata.org/internet.Google ScholarGoogle Scholar
  39. Bryan Pfaffenberger. 1996. “If I want it, it’s ok”: Usenet and the (outer) limits of free speech. Info. Soc. 12, 4 (1996), 365–386.Google ScholarGoogle Scholar
  40. Antonio Rago, Kristijonas Cyras, and Francesca Toni. 2016. Adapting the DF-QuAD Algorithm to Bipolar Argumentation. In Proceedings of the 1st International Workshop on Systems and Algorithms for Formal Argumentation (SAFA’16), Vol. 1672. CEUR-WS.org, 34–39.Google ScholarGoogle Scholar
  41. Iyad Rahwan, Mohammed I. Madakkatel, Jean-François Bonnefon, Ruqiyabi N. Awan, and Sherief Abdallah. 2010. Behavioral experiments for assessing the abstract argumentation semantics of reinstatement. Cogn. Sci. 34, 8 (2010), 1483–1502.Google ScholarGoogle ScholarCross RefCross Ref
  42. Iyad Rahwan and Guillermo R. Simari. 2009. Argumentation in Artificial Intelligence. Vol. 47. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Stefan Siersdorfer, Sergiu Chelaru, Wolfgang Nejdl, and Jose San Pedro. 2010. How useful are your comments? Analyzing and predicting YouTube comments and comment ratings. In Proceedings of the 19th International Conference on World Wide Web. ACM, 891–900. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Veronika Strnadová-Neeley, David Jurgens, and Tsai-Ching Lu. 2013. Characterizing online discussions in microblogs using network analysis. In Proceedings of the AAAI Spring Symposium: Analyzing Microtext.Google ScholarGoogle Scholar
  45. John Suler. 2004. The online disinhibition effect. Cyberpsychol. Behav. 7, 3 (2004), 321–326.Google ScholarGoogle ScholarCross RefCross Ref
  46. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web. 613–624. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Damian Trilling. 2015. Two different debates? Investigating the relationship between a political debate on TV and simultaneous comments on Twitter. Soc. Sci. Comput. Rev. 33, 3 (2015), 259–276. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Manos Tsagkias, Wouter Weerkamp, and Maarten De Rijke. 2010. News comments: Exploring, modeling, and online prediction. In Proceedings of the European Conference on Information Retrieval. Springer, 191–203. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Zhongyu Wei, Yang Liu, and Yi Li. 2016. Is this post persuasive? Ranking argumentative comments in online forum. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 195–200.Google ScholarGoogle ScholarCross RefCross Ref
  50. Michael Wooldridge. 2009. An Introduction to Multiagent Systems. John Wiley & Sons. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Anthony P. Young. 2018. Notes on abstract argumentation theory. Retrieved from https://arXiv:1806.07709.Google ScholarGoogle Scholar
  52. Anthony P. Young, Sagar Joglekar, Gioia Boschi, and Nishanth Sastry. 2020. Ranking comment sorting policies in online debates. Argu. & Comput. 1–21.Google ScholarGoogle Scholar
  53. Anthony P. Young, Sagar Joglekar, Kiran Garimella, and Nishanth Sastry. 2018. Approximations to truth in online comment networks. In Proceedings of the Workshop on Argumentation and Society at the 7th International Conference on Computational Models of Argument.Google ScholarGoogle Scholar
  54. Justine Zhang, Cristian Danescu-Niculescu-Mizil, Christina Sauper, and Sean J. Taylor. 2018. Characterizing online public discussions through patterns of participant interactions. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 198 (Nov. 2018), 27 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Who Has the Last Word? Understanding How to Sample Online Discussions

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on the Web
        ACM Transactions on the Web  Volume 15, Issue 3
        August 2021
        162 pages
        ISSN:1559-1131
        EISSN:1559-114X
        DOI:10.1145/3462273
        Issue’s Table of Contents

        Copyright © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 3 June 2021
        • Revised: 1 December 2020
        • Accepted: 1 December 2020
        • Received: 1 May 2020
        Published in tweb Volume 15, Issue 3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!