skip to main content
research-article

Using Argumentation to Improve Classification in Natural Language Problems

Published:12 July 2017Publication History
Skip Abstract Section

Abstract

Argumentation has proven successful in a number of domains, including Multi-Agent Systems and decision support in medicine and engineering. We propose its application to a domain yet largely unexplored by argumentation research: computational linguistics. We have developed a novel classification methodology that incorporates reasoning through argumentation with supervised learning. We train classifiers and then argue about the validity of their output. To do so, we identify arguments that formalise prototypical knowledge of a problem and use them to correct misclassifications. We illustrate our methodology on two tasks. On the one hand, we address cross-domain sentiment polarity classification, where we train classifiers on one corpus, for example, Tweets, to identify positive/negative polarity and classify instances from another corpus, for example, sentences from movie reviews. On the other hand, we address a form of argumentation mining that we call Relation-based Argumentation Mining, where we classify pairs of sentences based on whether the first sentence attacks or supports the second or whether it does neither. Whenever we find that one sentence attacks/supports the other, we consider both to be argumentative, irrespective of their stand-alone argumentativeness. For both tasks, we improve classification performance when using our methodology, compared to using standard classifiers only.

References

  1. Teresa Alsinet, Carlos Iván Chesñevar, Lluis Godo, and Guillermo Ricardo Simari. 2008. A logic programming framework for possibilistic argumentation: Formalization and logical properties. Fuzzy Sets Syst. 159 (2008), 1208--1228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Leila Amgoud, Philippe Besnard, and Anthony Hunter. 2015. Representing and reasoning about arguments mined from texts and dialogues. In Proceedings of European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, LNCS, Vol. 9161. Springer, 60--71.Google ScholarGoogle ScholarCross RefCross Ref
  3. Leila Amgoud, Claudette Cayrol, Marie-Christine Lagasquie-Schiex, and Pierre Livet. 2008. On bipolarity in argumentation frameworks. Int. J. Intell. Syst. 23, 10 (2008), 1062--1093. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Leila Amgoud and Mathieu Serrurier. 2008a. Agents that argue and explain classifications. Auton. Agents Multi-Agent Syst. 16, 2 (2008), 187--209. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Leila Amgoud and Mathieu Serrurier. 2008b. Arguing and explaining classifications. In Argumentation in Multi-Agent Systems. Springer, 164--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Emma Barker and Robert Gaizauskas. 2016. Summarizing multi-party argumentative conversations in reader comment on news. In Proceedings of the 3rd Workshop on Argumentation Mining. 12.Google ScholarGoogle ScholarCross RefCross Ref
  7. Pietro Baroni, Marco Romano, Francesca Toni, Marco Aurisicchio, and Giorgio Bertanza. 2015. Automatic evaluation of design alternatives with quantitative argumentation. Arg. Comput. 6, 1 (2015), 24--49.Google ScholarGoogle ScholarCross RefCross Ref
  8. Sahbi Benlamine, Maher Chaouachi, Serena Villata, Elena Cabrio, Claude Frasson, and Fabien Gandon. 2015. Emotions in argumentation: An empirical evaluation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’15). 156--163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Floris Bex, Sanjay Modgil, Henry Prakken, and Chris Reed. 2013. On logical specifications of the argument interchange format. J. Logic Comput. 23 (2013), 951--989.Google ScholarGoogle ScholarCross RefCross Ref
  10. Danushka Bollegala, David Weir, and John Carroll. 2011. Using multiple sources to construct a sentiment sensitive thesaurus for cross-domain sentiment classification. In Proceedings of the 49th Annual Meeting of the ACL: Human Language Technologies (ACL-HLT)-Volume 1. ACL, 132--141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Danushka Bollegala, David Weir, and John Carroll. 2013. Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE Trans. Knowl. Data Eng. 25, 8 (2013), 1719--1731. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Filip Boltuzic and Jan Šnajder. 2014. Back up your stance: Recognizing arguments in online discussions. In Proceedings of the 1st Workshop on Argumentation Mining. 49--58.Google ScholarGoogle ScholarCross RefCross Ref
  13. Filip Boltuzic and Jan Šnajder. 2015. Identifying prominent arguments in online debates using semantic textual similarity. In Proceedings of the 2nd Workshop on Argumentation Mining. 110--115.Google ScholarGoogle ScholarCross RefCross Ref
  14. Leo Breiman. 2001. Random forests. Mach. Learn. 45, 1 (2001), 5--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Elena Cabrio and Serena Villata. 2014. NoDE: A benchmark of natural language arguments. COMMA 266 (2014), 449--450.Google ScholarGoogle Scholar
  16. Erik Cambria and Bruce White. 2014. Jumping NLP curves: A review of natural language processing research {review article}. IEEE Comput. Intell. Mag. 9, 2 (2014), 48--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Lucas Carstens, Xiuyi Fan, Yang Gao, and Francesca Toni. 2015. An overview of argumentation frameworks for decision support. In Proceedings of the 4th International Workshop Graph Structures for Knowledge Representation and Reasoning (GKR). 32--49.Google ScholarGoogle ScholarCross RefCross Ref
  18. Lucas Carstens and Francesca Toni. 2015a. Improving out-of-domain sentiment polarity classification using argumentation. In SENTIRE. IEEE, 1294--1301. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lucas Carstens and Francesca Toni. 2015b. Towards relation based argumentation mining. In Proceedings of the 2nd Workshop on Argumentation Mining. 29--34.Google ScholarGoogle ScholarCross RefCross Ref
  20. Lucas Carstens, Francesca Toni, and Valentinos Evripidou. 2014. Argument mining and social debates. In Proceedings of Computational Models of Argument (COMMA'14). 451--452.Google ScholarGoogle Scholar
  21. Claudette. Cayrol and Marie-Christine Lagasquie-Schiex. 2005. On the acceptability of arguments in bipolar argumentation frameworks. In ECSQARU. 378--389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3 (2011), 27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Carlos Ivan Chesnevar, Jarred McGinnis, Sanjay Modgil, Iyad Rahwan, Chris Reed, Guillermo Ricardo Simari, Matthew South, Gerard Vreeswijk, and Steven Willmott. 2006. Towards an argument interchange format. Knowl. Eng. Rev. 21, 4 (2006), 293--316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Oana Cocarascu and Francesca Toni. 2016. Argumentation for machine learning: A survey. COMMA (2016), 219.Google ScholarGoogle Scholar
  25. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (2011), 2493--2537. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (1995), 273--297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment (2006), 177--190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Mauro Dragoni, Andrea GB Tettamanzi, and Célia da Costa Pereira. 2015. Propagating and aggregating fuzzy polarities for concept-level sentiment analysis. Cogni. Comput. 7, 2 (2015), 186--197.Google ScholarGoogle Scholar
  29. Phan Minh Dung. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artif. Intell. 77, 2 (1995), 321--357. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Valentinos Evripidou and Francesca Toni. 2014. Quaestio-it. com: A social intelligent debating platform. J. Decision Syst. 23, 3 (2014), 333--349.Google ScholarGoogle ScholarCross RefCross Ref
  31. Yang Gao and Francesca Toni. 2014. Argumentation accelerated reinforcement learning for cooperative multi-agent systems. In Proceedings of European Conference on Artificial Intelligence (ECAI). IOS Press, 333. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mehmet Gönen, Ayse Gönül Tanugur, and Ethem Alpaydin. 2008. Multiclass posterior probability support vector machines. IEEE Trans. Neural Netw. 19, 1 (2008), 130--139. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: An update. ACM SIGKDD Explor. Newslett. 11, 1 (2009), 10--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Alexander Hogenboom, Flavius Frasincar, Franciska de Jong, and Uzay Kaymak. 2015. Using rhetorical structure in sentiment analysis. Commun. ACM 58, 7 (2015), 69--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 168--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the ACL: Human Language Technologies (ACL-HLT)-Volume 1. ACL, 151--160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. George H. John and Pat Langley. 1995. Estimating continuous distributions in Bayesian classifiers. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann, 338--345. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Christian Kirschner, Judith Eckle-Kohler, and Iryna Gurevych. 2015. Linking the thoughts: Analysis of argumentation structures in scientific publications. In Proceedings of the 2nd Workshop on Argumentation Mining. 1--11.Google ScholarGoogle ScholarCross RefCross Ref
  39. Efthymios Kouloumpis, Theresa Wilson, and Johanna Moore. 2011. Twitter sentiment analysis: The good the bad and the omg! In Proceedings of the AAAI Conference on Web and Social Media (ICWSM), Vol. 11. AAAI, 538--541.Google ScholarGoogle Scholar
  40. Robert A Kowalski and Francesca Toni. 1996. Abstract argumentation. Artif. Intell. Law 4, 3--4 (1996), 275--296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. John Lawrence, Floris Bex, Chris Reed, and Mark Snaith. 2012. AIFdb: Infrastructure for the argument web. In Proceedings of Computational Models of Argument (COMMA). IOS Press, 515--516.Google ScholarGoogle Scholar
  42. John Lawrence and Chris Reed. 2014. AIFdb corpora. Proceedings of Computational Models of Argument (COMMA) 266 (2014), 465.Google ScholarGoogle Scholar
  43. Marco Lippi and Paolo Torroni. 2015a. Argument mining: A machine learning perspective. In Theory and Applications of Formal Argumentation. Springer, 163--176.Google ScholarGoogle Scholar
  44. Marco Lippi and Paolo Torroni. 2015b. Context-independent claim detection for argument mining. In Proceedings of the 24th International Conference on Artificial Intelligence (ICAI). AAAI Press, 185--191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Paul-Amaury Matt and Francesca Toni. 2008. A game-theoretic measure of argument strength for abstract argumentation. In Proceedings of the 11th European Conference Logics in Artificial Intelligence (JELIA 2008). 285--297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. Citeseer, 3111--3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Sanjay Modgil, Francesca Toni, Floris Bex, Ivan Bratko, Carlos I. Chesñevar, Wolfgang Dvořák, Marcelo A. Falappa, Xiuyi Fan, Sarah Alice Gaggl, Alejandro J. García, and others. 2013. The added value of argumentation. In Agreement Technologies. Springer, 357--403.Google ScholarGoogle Scholar
  48. Marie-Francine Moens, Erik Boiy, Raquel Palau, and Chris Reed. 2007. Automatic detection of arguments in legal texts. In Proceedings of the 11th International Conference on Artificial Intelligence and Law (ICAIL). ACM, 225--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Martin Mozina, Jure Zabkar, Trevor Bench-Capon, and Ivan Bratko. 2005. Argument based machine learning applied to law. Artif. Intell. Law 13, 1 (2005), 53--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Martin Mozina, Jure Zabkar, and Ivan Bratko. 2007. Argument based machine learning. Artif. Intell. 171, 10--15 (2007), 922--937. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Prakash M. Nadkarni, Lucila Ohno-Machado, and Wendy W. Chapman. 2011. Natural language processing: An introduction. J. Am. Med. Inform. Assoc. 18, 5 (2011), 544--551.Google ScholarGoogle ScholarCross RefCross Ref
  52. Thais Mayumi Oshiro, Pedro Santoro Perez, and José Augusto Baranauskas. 2012. How many trees in a random forest? In Proceedings of the International Conference on Machine Learning and Data Mining (MLDM). Springer, 154--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Raquel Palau and Marie-Francine Moens. 2009. Argumentation mining: The detection, classification and structure of arguments in text. In Proceedings of the 12th International Conference on Artificial Intelligence and Law (ICAIL). ACM, 98--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Bo Pang and Lilian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on ACL. ACL, 115--124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Bo Pang, Lilian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Vol. 10. ACL, 79--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Joonsuk Park and Claire Cardie. 2014. Identifying appropriate support for propositions in online user comments. Proceedings of The 52nd Annual Meeting of the ACL (2014), 29.Google ScholarGoogle ScholarCross RefCross Ref
  57. Andreas Peldszus and Manfred Stede. 2013. From argument diagrams to argumentation mining in texts: A survey. Int.l J. Cogn. Inform. Nat. Intell. 7, 1 (2013), 1--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Georgios Petasis and Vangelis Karkaletsis. 2016. Identifying argument components through textrank. In Proceedings of the 3rd Workshop on Argumentation Mining (2016), 94.Google ScholarGoogle ScholarCross RefCross Ref
  59. John C. Platt. 1999. Probabilities for SV machines. In Advances in Large Margin Classifiers. MIT Press, 61--74.Google ScholarGoogle Scholar
  60. Antonio Rago, Francesca Toni, Marco Aurisicchio, and Pietro Baroni. 2016. Discontinuity-free decision support with quantitative argumentation debates. In Proceedings of the 15th International Conference on Principles and Knowledge Representation and Reasoning (KR). AAAI, 63--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Iyad Rahwan and Chris Reed. 2009. The argument interchange format. In Argumentation in Artificial Intelligence. Springer, 383--402.Google ScholarGoogle Scholar
  62. Chris Reed and Glenn Rowe. 2004. Araucaria: Software for argument analysis, diagramming and representation. Int. J. Artif. Intell. Tools 13, 4 (2004), 961--979.Google ScholarGoogle ScholarCross RefCross Ref
  63. Niall Rooney, Hui Wang, and Fiona Browne. 2012. Applying kernel methods to argumentation mining. In Proceedings of the 25th Florida Artificial Intelligence Research Society Conference (FLAIRS). AAAI.Google ScholarGoogle Scholar
  64. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M. Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. Semeval-2015 task 10: Sentiment analysis in Twitter. In Proceedings of SemEval 2015.Google ScholarGoogle ScholarCross RefCross Ref
  65. Hassan Saif, Miriam Fernandez, Yulan He, and Harith Alani. 2013. Evaluation datasets for twitter sentiment analysis: A survey and a new dataset, the sts-gold. In Proceedings of the 1st Interantional Workshop on Emotion and Sentiment in Social and Expressive Media: Approaches and Perspectives from AI (ESSEM 2’13). AI*IA, 9--21.Google ScholarGoogle Scholar
  66. Gerard Salton and Michael J. McGill. 1986. Introduction to Modern Information Retrieval. McGraw--Hill, Inc. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Niek J. Sanders. 2011. Sanders-Twitter Sentiment Corpus. http://www.sananalytics.com/lab/twitter-sentiment. Accessed: 2017-04-24.Google ScholarGoogle Scholar
  68. Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP, Vol. 1631. ACL, 1642.Google ScholarGoogle Scholar
  69. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Comput. Ling. 37, 2 (2011), 267--307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Orith Toledo-Ronen, Roy Bar-Haim, and Noam Slonim. 2016. Expert stance graphs for computational argumentation. Proceedings of the Third Workshop on Argumentation Mining (2016), 119.Google ScholarGoogle ScholarCross RefCross Ref
  71. Stephen Toulmin. 2003. The Uses of Argument. Cambridge University Press.Google ScholarGoogle Scholar
  72. Angela Charng-Rurng Tsai, Chi-En Wu, Richard Tzong-Han Tsai, and Jane Yung-jen Hsu. 2013. Building a concept-level sentiment dictionary based on commonsense knowledge. IEEE Intell. Syst. (2013), 22--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Peter D. Turney. 2002. Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting on ACL (ACL). ACL, 417--424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Frans H. Van Eemeren and Rob Grootendorst. 2003. A Systematic Theory of Argumentation: The Pragma-dialectical Approach. Vol. 14. Cambridge University Press.Google ScholarGoogle Scholar
  75. Marilyn A. Walker, Jean E. Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012. A corpus for research on deliberation and debate. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC). Citeseer, 812--817.Google ScholarGoogle Scholar
  76. Albert Weichselbraun, Stefan Gindl, and Arno Scharl. 2014. Enriching semantic knowledge bases for opinion mining in big data applications. Knowl.-Based Syst. 69 (2014), 78--85. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004. Learning subjective language. Comput. Ling. 30, 3 (2004), 277--308. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Zhibiao Wu and Martha Palmer. 1994. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting on ACL. 133--138. Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Rui Xia, Chengqing Zong, Xuelei Hu, and Erik Cambria. 2013. Feature ensemble plus sample selection: Domain adaptation for sentiment classification. IEEE Intell. Syst. 28, 3 (2013), 10--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th Conference on Computational Linguistics-Volume 2. ACL, 947--953. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Min Zhang and Xingyao Ye. 2008. A generation model to unify topic relevance and lexicon-based sentiment for opinion retrieval. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 411--418. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Using Argumentation to Improve Classification in Natural Language Problems

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!