skip to main content
research-article

Stance and Sentiment in Tweets

Published:12 June 2017Publication History
Skip Abstract Section

Abstract

We can often detect from a person’s utterances whether he or she is in favor of or against a given target entity—one’s stance toward the target. However, a person may express the same stance toward a target by using negative or positive language. Here for the first time we present a dataset of tweet–target pairs annotated for both stance and sentiment. The targets may or may not be referred to in the tweets, and they may or may not be the target of opinion in the tweets. Partitions of this dataset were used as training and test sets in a SemEval-2016 shared task competition. We propose a simple stance detection system that outperforms submissions from all 19 teams that participated in the shared task. Additionally, access to both stance and sentiment annotations allows us to explore several research questions. We show that although knowing the sentiment expressed by a tweet is beneficial for stance classification, it alone is not sufficient. Finally, we use additional unlabeled data through distant supervision techniques and word embeddings to further improve stance classification.

References

  1. Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani, and Michael Minor. 2011. Cats rule and dogs drool! Classifying stance in online debate. In Proceedings of the Workshop on Computational Approaches to Subjectivity and Sentiment Analysis. 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ion Androutsopoulos and Prodromos Malakasiotis. 2010. A survey of paraphrasing and textual entailment methods. Journal of Artificial Intelligence Research 38, 1, 135--187. Google ScholarGoogle ScholarCross RefCross Ref
  3. Yoshua Bengio, Rejean Ducharme, and Pascal Vincent. 2001. A neural probabilistic language model. In Advances in Neural Information Processing Systems. 1--7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Luisa Bentivogli, Peter Clark, Ido Dagan, Hoa Dang, and Danilo Giampiccolo. 2011. The seventh PASCAL recognizing textual entailment challenge. In Proceedings of the Text Analysis Conference.Google ScholarGoogle Scholar
  5. Adam Bermingham and Alan Smeaton. 2011. On using Twitter to monitor political sentiment and predict election results. In Proceedings of the Workshop on Sentiment Analysis Where AI Meets Psychology. 2--10.Google ScholarGoogle Scholar
  6. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the International Conference on Machine Learning. 160--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Michael Conover, Bruno Gonçalves, Jacob Ratkiewicz, Alessandro Flammini, and Filippo Menczer. 2011a. Predicting the political alignment of Twitter users. In Proceedings of the IEEE International Conference on Privacy, Security, Risk, and Trust. 192--199.Google ScholarGoogle ScholarCross RefCross Ref
  8. Michael Conover, Jacob Ratkiewicz, Matthew R. Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011b. Political polarization on Twitter. In Proceedings of the International AAAI Conference on Weblogs and Social Media. 89--96.Google ScholarGoogle Scholar
  9. Mark Craven and Johan Kumlien. 1999. Constructing biological knowledge bases by extracting information from text sources. In Proceedings of the Conference on Intelligent Systems for Molecular Biology. 77--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: Generic applied modeling of language variability. In Proceedings of the PASCAL Workshop on Text Understanding and Mining. 26--29.Google ScholarGoogle Scholar
  11. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing Textual Entailment: Models and Applications. Morgan 8 Claypool.Google ScholarGoogle Scholar
  12. Lingjia Deng, Janyce Wiebe, and Yoonjung Choi. 2014. Joint inference and disambiguation of implicit sentiments via implicature constraints. In Proceedings of the International Conference on Computational Linguistics. 79--88.Google ScholarGoogle Scholar
  13. Myroslava O. Dzikovska, Rodney D. Nielsen, and Claudia Leacock. 2016. The joint student response analysis and recognizing textual entailment challenge: Making sense of student responses in educational applications. Language Resources and Evaluation 50, 1, 67--93. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Adam Faulkner. 2014. Automated classification of stance in student essays: An approach using stance target information and the Wikipedia link-based measure. In Proceedings of the Flairs Conference.Google ScholarGoogle Scholar
  15. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for Twitter: Annotation, features, and experiments. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter Sentiment Classification Using Distant Supervision. Technical Report. Stanford University, Stanford, CA.Google ScholarGoogle Scholar
  17. Jennifer Golbeck and Derek Hansen. 2011. Computing political preference among Twitter followers. In Proceedings of the Conference on Human Factors in Computing Systems. 1105--1108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kazi Saidul Hasan and Vincent Ng. 2013. Stance classification of ideological debates: Data, models, features, and constraints. In Proceedings of the International Joint Conference on Natural Language Processing. 1348--1356.Google ScholarGoogle Scholar
  19. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 168--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Stephanie D. Husby and Denilson Barbosa. 2012. Topic classification of blog posts using distant supervision. In Proceedings of the Workshop on Semantic Analysis in Social Media. 28--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. 2014b. NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle Scholar
  22. Svetlana Kiritchenko, Xiaodan Zhu, and Saif M. Mohammad. 2014a. Sentiment analysis of short informal texts. 50, 723--762. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Florian Kunneman, Christine Liebrecht, and Antal van den Bosch. 2014. The (un)predictability of emotional hashtags in Twitter. In Proceedings of the Workshop on Language Analysis for Social Media. 26--34.Google ScholarGoogle Scholar
  24. Vasileios Lampos, Daniel Preotiuc-Pietro, and Trevor Cohn. 2013. A user-centric model of voting intention from social media. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 993--1003.Google ScholarGoogle Scholar
  25. Bing Liu. 2015. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press.Google ScholarGoogle ScholarCross RefCross Ref
  26. Bing Liu and Lei Zhang. 2012. A survey of opinion mining and sentiment analysis. In Mining Text Data, C. C. Aggarwal and C.-X. Zhai (Eds.). Springer, 415--463.Google ScholarGoogle Scholar
  27. Walid Magdy, Hassan Sajjad, Tarek El-Ganainy, and Fabrizio Sebastiani. 2015. Bridging social media via distant supervision. Social Network Analysis and Mining 5, 1, 1--12.Google ScholarGoogle ScholarCross RefCross Ref
  28. Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. SemEval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle ScholarCross RefCross Ref
  29. Diana Maynard and Adam Funk. 2011. Automatic detection of political opinions in tweets. In Proceedings of the ESWC Workshop on the Semantic Web. 88--99. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 1003--1011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Andriy Mnih and Geoffrey E. Hinton. 2009. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems. 1081--1088. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Saif M. Mohammad. 2012. #Emotional tweets. In Proceedings of the Joint Conference on Lexical and Computational Semantics. 246--255. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Saif M. Mohammad. 2016a. A practical guide to sentiment annotation: Challenges and solutions. In Proceedings of the Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis.Google ScholarGoogle ScholarCross RefCross Ref
  35. Saif M. Mohammad. 2016b. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion Measurement, H. L. Meiselman (Ed.). Elsevier, 201--237.Google ScholarGoogle Scholar
  36. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 Task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle ScholarCross RefCross Ref
  37. Saif M. Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu. 2013. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle Scholar
  38. Saif M. Mohammad and Peter D. Turney. 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 26--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Saif M. Mohammad, Xiaodan Zhu, Svetlana Kiritchenko, and Joel Martin. 2015. Sentiment, emotion, purpose, and style in electoral tweets. Information Processing and Management 51, 480--499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2, 1--2, 1--135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ankur P. Parikh, Hoifung Poon, and Kristina Toutanova. 2015. Grounded semantic parsing for complex knowledge extraction. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  42. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, et al. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825--2830. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1532--1543.Google ScholarGoogle Scholar
  44. Maria Pontiki, Dimitrios Galanis, Haris Papageogiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. SemEval-2015 Task 12: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle ScholarCross RefCross Ref
  45. Maria Pontiki, Dimitrios Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 Task 4: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle ScholarCross RefCross Ref
  46. Kevin Reschke, Martin Jankowiak, Mihai Surdeanu, Christopher D. Manning, and Daniel Jurafsky. 2014. Event extraction using distant supervision. In Proceedings of the International Conference on Language Resources and Evaluation. 4527--4531.Google ScholarGoogle Scholar
  47. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1524--1534. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M. Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. SemEval-2015 Task 10: Sentiment analysis in Twitter. In Proceedings of the International Workshop on Semantic Evaluations.Google ScholarGoogle ScholarCross RefCross Ref
  49. Parinaz Sobhani, Diana Inkpen, and Stan Matwin. 2015. From argumentation mining to stance classification. In Proceedings of the Workshop on Argumentation Mining. 67--77.Google ScholarGoogle ScholarCross RefCross Ref
  50. Parinaz Sobhani, Saif M. Mohammad, and Svetlana Kiritchenko. 2016. Detecting stance in tweets and analyzing its interaction with sentiment. In Proceedings of the Joint Conference on Lexical and Computational Semantics (*Sem).Google ScholarGoogle ScholarCross RefCross Ref
  51. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 116--124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 1555--1565.Google ScholarGoogle ScholarCross RefCross Ref
  53. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 327--335. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Andranik Tumasjan, Timm O. Sprenger, Philipp G. Sandner, and Isabell M. Welpe. 2010. Election forecasts with Twitter: How 140 characters reflect the political landscape. Social Science Computer Review 29, 4, 402--418. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 384--394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Peter D. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the Association for Computational Linguistics. 417--424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Marilyn A. Walker, Pranav Anand, Robert Abbott, and Ricky Grant. 2012a. Stance classification using dialogic properties of persuasion. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 592--596. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Marilyn A. Walker, Jean E. Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012b. A corpus for research on deliberation and debate. In Proceedings of the International Conference on Language Resources and Evaluation. 812--817.Google ScholarGoogle Scholar
  59. Theresa Wilson, Zornitsa Kozareva, Preslav Nakov, Sara Rosenthal, Veselin Stoyanov, and Alan Ritter. 2013. SemEval-2013 Task 2: Sentiment analysis in Twitter. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle Scholar
  60. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 347--354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Guido Zarrella and Amy Marsh. 2016. MITRE at SemEval-2016 Task 6: Transfer learning for stance detection. In Proceedings of the International Workshop on Semantic Evaluation.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Stance and Sentiment in Tweets

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Internet Technology
            ACM Transactions on Internet Technology  Volume 17, Issue 3
            Special Issue on Argumentation in Social Media and Regular Papers
            August 2017
            201 pages
            ISSN:1533-5399
            EISSN:1557-6051
            DOI:10.1145/3106680
            • Editor:
            • Munindar P. Singh
            Issue’s Table of Contents

            Copyright © 2017 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 12 June 2017
            • Accepted: 1 September 2016
            • Revised: 1 August 2016
            • Received: 1 January 2016
            Published in toit Volume 17, Issue 3

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!