Abstract
We can often detect from a person’s utterances whether he or she is in favor of or against a given target entity—one’s stance toward the target. However, a person may express the same stance toward a target by using negative or positive language. Here for the first time we present a dataset of tweet–target pairs annotated for both stance and sentiment. The targets may or may not be referred to in the tweets, and they may or may not be the target of opinion in the tweets. Partitions of this dataset were used as training and test sets in a SemEval-2016 shared task competition. We propose a simple stance detection system that outperforms submissions from all 19 teams that participated in the shared task. Additionally, access to both stance and sentiment annotations allows us to explore several research questions. We show that although knowing the sentiment expressed by a tweet is beneficial for stance classification, it alone is not sufficient. Finally, we use additional unlabeled data through distant supervision techniques and word embeddings to further improve stance classification.
- Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani, and Michael Minor. 2011. Cats rule and dogs drool! Classifying stance in online debate. In Proceedings of the Workshop on Computational Approaches to Subjectivity and Sentiment Analysis. 1--9. Google Scholar
Digital Library
- Ion Androutsopoulos and Prodromos Malakasiotis. 2010. A survey of paraphrasing and textual entailment methods. Journal of Artificial Intelligence Research 38, 1, 135--187. Google Scholar
Cross Ref
- Yoshua Bengio, Rejean Ducharme, and Pascal Vincent. 2001. A neural probabilistic language model. In Advances in Neural Information Processing Systems. 1--7. Google Scholar
Digital Library
- Luisa Bentivogli, Peter Clark, Ido Dagan, Hoa Dang, and Danilo Giampiccolo. 2011. The seventh PASCAL recognizing textual entailment challenge. In Proceedings of the Text Analysis Conference.Google Scholar
- Adam Bermingham and Alan Smeaton. 2011. On using Twitter to monitor political sentiment and predict election results. In Proceedings of the Workshop on Sentiment Analysis Where AI Meets Psychology. 2--10.Google Scholar
- Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the International Conference on Machine Learning. 160--167. Google Scholar
Digital Library
- Michael Conover, Bruno Gonçalves, Jacob Ratkiewicz, Alessandro Flammini, and Filippo Menczer. 2011a. Predicting the political alignment of Twitter users. In Proceedings of the IEEE International Conference on Privacy, Security, Risk, and Trust. 192--199.Google Scholar
Cross Ref
- Michael Conover, Jacob Ratkiewicz, Matthew R. Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011b. Political polarization on Twitter. In Proceedings of the International AAAI Conference on Weblogs and Social Media. 89--96.Google Scholar
- Mark Craven and Johan Kumlien. 1999. Constructing biological knowledge bases by extracting information from text sources. In Proceedings of the Conference on Intelligent Systems for Molecular Biology. 77--86. Google Scholar
Digital Library
- Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: Generic applied modeling of language variability. In Proceedings of the PASCAL Workshop on Text Understanding and Mining. 26--29.Google Scholar
- Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing Textual Entailment: Models and Applications. Morgan 8 Claypool.Google Scholar
- Lingjia Deng, Janyce Wiebe, and Yoonjung Choi. 2014. Joint inference and disambiguation of implicit sentiments via implicature constraints. In Proceedings of the International Conference on Computational Linguistics. 79--88.Google Scholar
- Myroslava O. Dzikovska, Rodney D. Nielsen, and Claudia Leacock. 2016. The joint student response analysis and recognizing textual entailment challenge: Making sense of student responses in educational applications. Language Resources and Evaluation 50, 1, 67--93. Google Scholar
Digital Library
- Adam Faulkner. 2014. Automated classification of stance in student essays: An approach using stance target information and the Wikipedia link-based measure. In Proceedings of the Flairs Conference.Google Scholar
- Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for Twitter: Annotation, features, and experiments. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. Google Scholar
Digital Library
- Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter Sentiment Classification Using Distant Supervision. Technical Report. Stanford University, Stanford, CA.Google Scholar
- Jennifer Golbeck and Derek Hansen. 2011. Computing political preference among Twitter followers. In Proceedings of the Conference on Human Factors in Computing Systems. 1105--1108. Google Scholar
Digital Library
- Kazi Saidul Hasan and Vincent Ng. 2013. Stance classification of ideological debates: Data, models, features, and constraints. In Proceedings of the International Joint Conference on Natural Language Processing. 1348--1356.Google Scholar
- Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 168--177. Google Scholar
Digital Library
- Stephanie D. Husby and Denilson Barbosa. 2012. Topic classification of blog posts using distant supervision. In Proceedings of the Workshop on Semantic Analysis in Social Media. 28--36. Google Scholar
Digital Library
- Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. 2014b. NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
- Svetlana Kiritchenko, Xiaodan Zhu, and Saif M. Mohammad. 2014a. Sentiment analysis of short informal texts. 50, 723--762. Google Scholar
Digital Library
- Florian Kunneman, Christine Liebrecht, and Antal van den Bosch. 2014. The (un)predictability of emotional hashtags in Twitter. In Proceedings of the Workshop on Language Analysis for Social Media. 26--34.Google Scholar
- Vasileios Lampos, Daniel Preotiuc-Pietro, and Trevor Cohn. 2013. A user-centric model of voting intention from social media. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 993--1003.Google Scholar
- Bing Liu. 2015. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press.Google Scholar
Cross Ref
- Bing Liu and Lei Zhang. 2012. A survey of opinion mining and sentiment analysis. In Mining Text Data, C. C. Aggarwal and C.-X. Zhai (Eds.). Springer, 415--463.Google Scholar
- Walid Magdy, Hassan Sajjad, Tarek El-Ganainy, and Fabrizio Sebastiani. 2015. Bridging social media via distant supervision. Social Network Analysis and Mining 5, 1, 1--12.Google Scholar
Cross Ref
- Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. SemEval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
Cross Ref
- Diana Maynard and Adam Funk. 2011. Automatic detection of political opinions in tweets. In Proceedings of the ESWC Workshop on the Semantic Web. 88--99. Google Scholar
Digital Library
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119. Google Scholar
Digital Library
- Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 1003--1011. Google Scholar
Digital Library
- Andriy Mnih and Geoffrey E. Hinton. 2009. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems. 1081--1088. Google Scholar
Digital Library
- Saif M. Mohammad. 2012. #Emotional tweets. In Proceedings of the Joint Conference on Lexical and Computational Semantics. 246--255. Google Scholar
Digital Library
- Saif M. Mohammad. 2016a. A practical guide to sentiment annotation: Challenges and solutions. In Proceedings of the Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis.Google Scholar
Cross Ref
- Saif M. Mohammad. 2016b. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion Measurement, H. L. Meiselman (Ed.). Elsevier, 201--237.Google Scholar
- Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 Task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
Cross Ref
- Saif M. Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu. 2013. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
- Saif M. Mohammad and Peter D. Turney. 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 26--34. Google Scholar
Digital Library
- Saif M. Mohammad, Xiaodan Zhu, Svetlana Kiritchenko, and Joel Martin. 2015. Sentiment, emotion, purpose, and style in electoral tweets. Information Processing and Management 51, 480--499. Google Scholar
Digital Library
- Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2, 1--2, 1--135. Google Scholar
Digital Library
- Ankur P. Parikh, Hoifung Poon, and Kristina Toutanova. 2015. Grounded semantic parsing for complex knowledge extraction. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics.Google Scholar
Cross Ref
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, et al. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825--2830. Google Scholar
Digital Library
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1532--1543.Google Scholar
- Maria Pontiki, Dimitrios Galanis, Haris Papageogiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. SemEval-2015 Task 12: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
Cross Ref
- Maria Pontiki, Dimitrios Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 Task 4: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
Cross Ref
- Kevin Reschke, Martin Jankowiak, Mihai Surdeanu, Christopher D. Manning, and Daniel Jurafsky. 2014. Event extraction using distant supervision. In Proceedings of the International Conference on Language Resources and Evaluation. 4527--4531.Google Scholar
- Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1524--1534. Google Scholar
Digital Library
- Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M. Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. SemEval-2015 Task 10: Sentiment analysis in Twitter. In Proceedings of the International Workshop on Semantic Evaluations.Google Scholar
Cross Ref
- Parinaz Sobhani, Diana Inkpen, and Stan Matwin. 2015. From argumentation mining to stance classification. In Proceedings of the Workshop on Argumentation Mining. 67--77.Google Scholar
Cross Ref
- Parinaz Sobhani, Saif M. Mohammad, and Svetlana Kiritchenko. 2016. Detecting stance in tweets and analyzing its interaction with sentiment. In Proceedings of the Joint Conference on Lexical and Computational Semantics (*Sem).Google Scholar
Cross Ref
- Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 116--124. Google Scholar
Digital Library
- Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 1555--1565.Google Scholar
Cross Ref
- Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 327--335. Google Scholar
Digital Library
- Andranik Tumasjan, Timm O. Sprenger, Philipp G. Sandner, and Isabell M. Welpe. 2010. Election forecasts with Twitter: How 140 characters reflect the political landscape. Social Science Computer Review 29, 4, 402--418. Google Scholar
Digital Library
- Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 384--394. Google Scholar
Digital Library
- Peter D. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the Association for Computational Linguistics. 417--424. Google Scholar
Digital Library
- Marilyn A. Walker, Pranav Anand, Robert Abbott, and Ricky Grant. 2012a. Stance classification using dialogic properties of persuasion. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 592--596. Google Scholar
Digital Library
- Marilyn A. Walker, Jean E. Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012b. A corpus for research on deliberation and debate. In Proceedings of the International Conference on Language Resources and Evaluation. 812--817.Google Scholar
- Theresa Wilson, Zornitsa Kozareva, Preslav Nakov, Sara Rosenthal, Veselin Stoyanov, and Alan Ritter. 2013. SemEval-2013 Task 2: Sentiment analysis in Twitter. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
- Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 347--354. Google Scholar
Digital Library
- Guido Zarrella and Amy Marsh. 2016. MITRE at SemEval-2016 Task 6: Transfer learning for stance detection. In Proceedings of the International Workshop on Semantic Evaluation.Google Scholar
Cross Ref
Index Terms
Stance and Sentiment in Tweets
Recommendations
Challenges for Sentence Level Opinion Detection in Blogs
ICIS '09: Proceedings of the 2009 Eigth IEEE/ACIS International Conference on Computer and Information ScienceThe Opinion Detection from blogs has always been a challenge for researchers. However with the introduction of Blog track in TREC 2006, a considerable improvement has been seen in this field at document level. But now it is the time when researchers are ...
Tweet Sentiment Analyzer: Sentiment Score Estimation Method for Assessing the Value of Opinions in Tweets
AICTC '16: Proceedings of the International Conference on Advances in Information Communication Technology & ComputingSocial networking applications are prominent among the internet user communities. Many social media websites are used for sharing the information instantly. Twitter is one of the vibrant social networking websites for sharing small textual information ...
Followee recommendation based on topic extraction and sentiment analysis from tweets
iiWAS '15: Proceedings of the 17th International Conference on Information Integration and Web-based Applications & ServicesTwitter has become a popular social media service, accumulating and distributing vast amounts of information for its numerous users. One feature of Twitter is that it enables a user to follow other users, who can obtain the information her/his followees ...






Comments