skip to main content
research-article

Incorporating Multi-Level User Preference into Document-Level Sentiment Classification

Authors Info & Claims
Published:19 November 2018Publication History
Skip Abstract Section

Abstract

Document-level sentiment classification aims to predict a user’s sentiment polarity in a document about a product. Most existing methods only focus on review contents and ignore users who post reviews. In fact, when reviewing a product, different users have different word-using habits to express opinions (i.e., word-level user preference), care about different attributes of the product (i.e., aspect-level user preference), and have different characteristics to score the review (i.e., polarity-level user preference). These preferences have great influence on interpreting the sentiment of text. To address this issue, we propose a model called Hierarchical User Attention Network (HUAN), which incorporates multi-level user preference into a hierarchical neural network to perform document-level sentiment classification. Specifically, HUAN encodes different kinds of information (word, sentence, aspect, and document) in a hierarchical structure and imports user embedding and user attention mechanism to model these preferences. Empirical results on two real-world datasets show that HUAN achieves state-of-the-art performance. Furthermore, HUAN can also mine important attributes of products for different users.

References

  1. Mohammad Al Boni, Keira Qi Zhou, Hongning Wang, and Matthew S. Gerber. 2015. Model adaptation for personalized opinion analysis. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 769--774.Google ScholarGoogle Scholar
  2. Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1650--1659.Google ScholarGoogle ScholarCross RefCross Ref
  3. Dan C. Ciresan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column deep neural networks for image classification. In Proceedings of the 2012 Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 3642--3649. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. George E. Dahl, Dong Yu, Li Deng, and Alex Acero. 2011. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio Speech and Language Processing 20, 1 (2011), 30--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Zi-Yi Dou. 2017. Capturing user and product information for document level sentiment analysis with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 532--537.Google ScholarGoogle ScholarCross RefCross Ref
  7. Wenliang Gao, Naoki Yoshinaga, Nobuhiro Kaji, and Masaru Kitsuregawa. 2013. Modeling user leniency and product popularity for sentiment classification. In Proceedings of the 2016 International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, 1107--1111.Google ScholarGoogle Scholar
  8. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1602--1613.Google ScholarGoogle ScholarCross RefCross Ref
  9. Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2017. An unsupervised neural attention model for aspect extraction. In Proceedings of the 2017 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 388--397.Google ScholarGoogle ScholarCross RefCross Ref
  10. Sepp Hochreiter and Jãijrgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Xia Hu, Lei Tang, Jiliang Tang, and Huan Liu. 2013. Exploiting social relations for sentiment analysis in microblogging. In Proceedings of the 2013 International Conference on Web Search and Data Mining. ACM, 537--546. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In Proceedings of the 2011 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 151--160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1746--1751.Google ScholarGoogle ScholarCross RefCross Ref
  14. Svetlana Kiritchenko, Xiaodan Zhu, and Saif M Mohammad. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50 (2014), 723--762. Google ScholarGoogle ScholarCross RefCross Ref
  15. Jiwei Li, Thang Luong, Dan Jurafsky, and Eduard Hovy. 2015. When are tree structures necessary for deep learning of representations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2304--2314.Google ScholarGoogle ScholarCross RefCross Ref
  16. Junjie Li, Haitong Yang, and Chengqing Zong. 2016. Sentiment classification of social media text considering user attributes. In Proceedings of the 2016 Conference on Natural Language Processing and Chinese Computing. Springer, 583--594.Google ScholarGoogle ScholarCross RefCross Ref
  17. Shoushan Li, Chu-Ren Huang, and Chengqing Zong. 2011. Multi-domain sentiment classification with classifier combination. Journal of Computer Science and Technology 26, 1 (2011), 25--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Shoushan Li, Shengfeng Ju, Guodong Zhou, and Xiaojun Li. 2012. Active learning for imbalanced sentiment classification. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 139--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Shoushan Li, Sophia Yat Mei Lee, Ying Chen, Chu-Ren Huang, and Guodong Zhou. 2010. Sentiment classification and polarity shifting. In Proceedings of the 2010 International Conference on Computational Linguistics. COLING 2010 Organizing Committee, 635--643. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Shoushan Li, Yunxia Xue, Zhongqing Wang, and Guodong Zhou. 2013. Active learning for cross-domain sentiment classification. In Proceedings of the 2013 International Joint Conference on Artificial Intelligence. AAAI Press, 2127--2133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5, 1 (2012), 1--167.Google ScholarGoogle Scholar
  22. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of the 2014 Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 55--60.Google ScholarGoogle ScholarCross RefCross Ref
  23. Tomas Mikolov, Greg Corrado, Kai Chen, Jeffrey Dean, Tomas Mikolov, Greg Corrado, Kai Chen, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781 (2013).Google ScholarGoogle Scholar
  24. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2, 1--2 (2008), 1--135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 79--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Martin F. Porter. 1980. An algorithm for suffix stripping. Program 14, 3 (1980), 130--137.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Lizhen Qu, Georgiana Ifrim, and Gerhard Weikum. 2010. The bag-of-opinions method for review rating prediction from sparse text patterns. In Proceedings of the 2010 International Conference on Computational Linguistics. COLING 2010 Organizing Committee, 913--921. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 1201--1211. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, and Christopher D. Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 151--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1631--1642.Google ScholarGoogle Scholar
  31. Kaisong Song, Shi Feng, Wei Gao, Daling Wang, Ge Yu, and Kam Fai Wong. 2015. Personalized sentiment classification based on latent individuality of microblog users. In Proceedings of the 2015 International Joint Conference on Artificial Intelligence. AAAI Press, 2277--2283. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1556--1566.Google ScholarGoogle Scholar
  33. Chenhao Tan, Lillian Lee, Jie Tang, Long Jiang, Ming Zhou, and Ping Li. 2011. User-level sentiment analysis incorporating social networks. In Proceedings of the 2011 International Conference on Knowledge Discovery and Data Mining. ACM, 1397--1405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1422--1432.Google ScholarGoogle ScholarCross RefCross Ref
  35. Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1014--1023.Google ScholarGoogle ScholarCross RefCross Ref
  36. Duyu Tang, Bing Qin, Yuekui Yang, and Yuekui Yang. 2015. User modeling with neural network for review rating prediction. In Proceedings of the 2015 International Joint Conference on Artificial Intelligence. AAAI Press, 1340--1346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 2014 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1555--1565.Google ScholarGoogle ScholarCross RefCross Ref
  38. Hongning Wang, Yue Lu, and Chengxiang Zhai. 2010. Latent aspect rating analysis on review text data: A rating regression approach. In Proceedings of the 2010 International Conference on Knowledge Discovery and Data Mining. ACM, 783--792. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 47--57.Google ScholarGoogle ScholarCross RefCross Ref
  40. Linlin Wang, Kang Liu, Zhu Cao, Jun Zhao, and Gerard de Melo. 2015. Sentiment-aspect extraction based on restricted Boltzmann machines. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 616--625.Google ScholarGoogle ScholarCross RefCross Ref
  41. Rui Xia, Feng Xu, Jianfei Yu, Yong Qi, and Erik Cambria. 2016. Polarity shift detection, elimination and ensemble: A three-stage model for document-level sentiment analysis. Information Processing and Management 52, 1 (2016), 36--45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Rui Xia, Feng Xu, Chengqing Zong, Qianmu Li, Yong Qi, and Tao Li. 2015. Dual sentiment analysis: Considering two sides of one review. IEEE Transactions on Knowledge and Data Engineering 27, 8 (2015), 2120--2133.Google ScholarGoogle ScholarCross RefCross Ref
  43. Rui Xia and Chengqing Zong. 2010. Exploring the use of word relation features for sentiment classification. In Proceedings of the 2010 International Conference on Computational Linguistics. Association for Computational Linguistics, 1336--1344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Rui Xia, Chengqing Zong, and Shoushan Li. 2011. Ensemble of feature sets and classification algorithms for sentiment classification. Information Sciences 181, 6 (2011), 1138--1152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 1480--1489.Google ScholarGoogle Scholar
  46. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).Google ScholarGoogle Scholar

Index Terms

  1. Incorporating Multi-Level User Preference into Document-Level Sentiment Classification

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 18, Issue 1
      March 2019
      196 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3292011
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 19 November 2018
      • Accepted: 1 June 2018
      • Revised: 1 March 2018
      • Received: 1 November 2017
      Published in tallip Volume 18, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!