skip to main content
research-article

Neural Co-training for Sentiment Classification with Product Attributes

Published:04 August 2020Publication History
Skip Abstract Section

Abstract

Sentiment classification aims to detect polarity from a piece of text. The polarity is usually positive or negative, and the text genre is usually product review. The challenges of sentiment classification are that it is hard to capture semantic of reviews, and the labeled data is hard to annotate. Therefore, we propose neural co-training to learn the semantic representation of each review using the neural network model, and learn the information from unlabeled data using a co-training framework. In particular, we use the attention-based bi-directional Gated Recurrent Unit (Att-BiGRU) to model the semantic content of each review and regard different categories of the target product as different views. We then use a co-training framework to learn and predict the unlabeled reviews with different views. Experiment results with the Yelp dataset demonstrate the effectiveness of our approach.

References

  1. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, June 18--22, 2018, Salt Lake City, UT. 6077--6086.Google ScholarGoogle ScholarCross RefCross Ref
  2. Alina Andreevskaia and Sabine Bergler. 2008. When specialists and generalists work together: Overcoming domain dependence in sentiment tagging. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, June 15--20, 2008, Columbus, Ohio. 290--298.Google ScholarGoogle Scholar
  3. John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.Google ScholarGoogle Scholar
  4. Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009. Semi-supervised learning. IEEE Trans. Neural Networks 20, 3 (2009), 542--542.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, November 1--4, 2016, Austin, Texas. 1650--1659.Google ScholarGoogle ScholarCross RefCross Ref
  6. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 25--29, 2014, Doha, Qatar. 1724--1734.Google ScholarGoogle ScholarCross RefCross Ref
  7. Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. 2015. Attention-based models for speech recognition. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 577--585.Google ScholarGoogle Scholar
  8. Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised sequence learning. In Proceedings of the Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 3079--3087.Google ScholarGoogle Scholar
  9. Francois Denis, Anne Laurent, Rémi Gilleron, and Marc Tommasi. 2003. Text classification and co-training from positive and unlabeled examples. In Proceedings of the ICML 2003 Workshop: The Continuum from Labeled to Unlabeled Data. 80--87.Google ScholarGoogle Scholar
  10. Ann Devitt and Khurshid Ahmad. 2007. Sentiment polarity identification in financial news: A cohesion-based approach. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.Google ScholarGoogle Scholar
  11. Xin Dong and Gerard de Melo. 2018. A helping hand: Transfer learning for deep sentiment analysis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, July 15--20, 2018, Melbourne, Australia. 2524--2534.Google ScholarGoogle ScholarCross RefCross Ref
  12. Zi-Yi Dou. 2017. Capturing user and product information for document level sentiment analysis with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 521--526.Google ScholarGoogle ScholarCross RefCross Ref
  13. Gayatree Ganu, Noemie Elhadad, and Amélie Marian. 2009. Beyond the stars: Improving rating predictions using review text content. In Proceedings of the 12th International Workshop on the Web and Databases, Providence, Rhode Island, June 28, 2009.Google ScholarGoogle Scholar
  14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (1997), 1735--1780.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Rie Johnson and Tong Zhang. 2015. Semi-supervised convolutional neural networks for text categorization via region embedding. In Proceedings of the Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 919--927.Google ScholarGoogle Scholar
  16. Rie Johnson and Tong Zhang. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. In Proceedings of the 33nd International Conference on Machine Learning, June 19--24, 2016, New York City, NY. 526--534.Google ScholarGoogle Scholar
  17. Hiroshi Kanayama, Tetsuya Nasukawa, and Hideo Watanabe. 2004. Deeper sentiment analysis using machine translation technology. In Proceedings of the 20th International Conference on Computational Linguistics, August 23--27, 2004, Geneva, Switzerland.Google ScholarGoogle Scholar
  18. Alistair Kennedy and Diana Inkpen. 2006. Sentiment classification of movie reviews using contextual valence shifters. Comput. Intell. 22, 2 (2006), 110--125.Google ScholarGoogle ScholarCross RefCross Ref
  19. Soo-Min Kim and Eduard H. Hovy. 2004. Determining the sentiment of opinions. In Proceedings of the 20th International Conference on Computational Linguistics, August 23--27, 2004, Geneva, Switzerland.Google ScholarGoogle Scholar
  20. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 25--29, 2014, Doha, Qatar. 1746--1751.Google ScholarGoogle ScholarCross RefCross Ref
  21. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning, June 21--26, 2014, Beijing, China. 1188--1196.Google ScholarGoogle Scholar
  22. Huayu Li, Martin Renqiang Min, Yong Ge, and Asim Kadav. 2017. A context-aware attention network for interactive question answering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 13--17, 2017, Halifax, NS, Canada. 927--935.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Linghui Li, Sheng Tang, Lixi Deng, Yongdong Zhang, and Qi Tian. 2017. Image caption with global-local attention. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, February 4--9, 2017, San Francisco, California. 4133--4139.Google ScholarGoogle Scholar
  24. Shoushan Li, Chu-Ren Huang, Guodong Zhou, and Sophia Yat Mei Lee. 2010. Employing personal/impersonal views in supervised and semi-supervised sentiment classification. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11--16, 2010, Uppsala, Sweden. 414--423.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Shoushan Li, Zhongqing Wang, Guodong Zhou, and Sophia Yat Mei Lee. 2011. Semi-supervised learning for imbalanced sentiment classification. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, July 16--22, 2011, Barcelona, Catalonia, Spain. 1826--1831.Google ScholarGoogle Scholar
  26. Tao Li, Yi Zhang, and Vikas Sindhwani. 2009. A non-negative matrix tri-factorization approach to sentiment classification with lexical prior knowledge. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1. 244--252.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Yi Luan, Mari Ostendorf, and Hannaneh Hajishirzi. 2017. Scientific information extraction with semi-supervised neural tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 2641--2651.Google ScholarGoogle ScholarCross RefCross Ref
  28. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, September 17--21, 2015, Lisbon, Portugal. 1412--1421.Google ScholarGoogle ScholarCross RefCross Ref
  29. Ryan T. McDonald, Kerry Hannan, Tyler Neylon, Mike Wells, and Jeffrey C. Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.Google ScholarGoogle Scholar
  30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems December 5--8, 2013, Lake Tahoe, Nevada. 3111--3119.Google ScholarGoogle Scholar
  31. Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725 (2016).Google ScholarGoogle Scholar
  32. Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. 2015. Distributional smoothing with virtual adversarial training. arXiv preprint arXiv:1507.00677 (2015).Google ScholarGoogle Scholar
  33. Rodrigo Moraes, João Francisco Valiati, and Wilson P. Gavião Neto. 2013. Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Syst. Appl. 40, 2 (2013), 621--633.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Tony Mullen and Nigel Collier. 2004. Sentiment analysis using support vector machines with diverse information sources. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, SIGDAT, July 25--26, 2004, Barcelona, Spain. 412--418.Google ScholarGoogle Scholar
  35. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, July 21--26, 2004, Barcelona, Spain. 271--278.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. ACL, 79--86.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Lizhen Qu, Georgiana Ifrim, and Gerhard Weikum. 2010. The bag-of-opinions method for review rating prediction from sparse text patterns. In Proceedings of the 23rd International Conference on Computational Linguistics, August 23--27, 2010, Beijing, China. 913--921.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jonathon Read. 2005. Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In , Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, June 25--30, 2005, University of Michigan. 43--48.Google ScholarGoogle ScholarCross RefCross Ref
  39. Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 338--348.Google ScholarGoogle ScholarCross RefCross Ref
  40. Yafeng Ren, Ruimin Wang, and Donghong Ji. 2016. A topic-enhanced word embedding for Twitter sentiment classification. Inf. Sci. 369 (2016), 188--198.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Aliaksei Severyn and Alessandro Moschitti. 2015. Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, August 9--13, 2015, Santiago, Chile. 959--962.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Vikas Sindhwani and Prem Melville. 2008. Document-word co-regularization for semi-supervised sentiment analysis. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15--19, 2008, Pisa, Italy. 1025--1030.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 18--21, 2013, Seattle, Washington. 1631--1642.Google ScholarGoogle Scholar
  44. Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, September 17-21, 2015, Lisbon, Portugal. 1422--1432.Google ScholarGoogle ScholarCross RefCross Ref
  45. Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Volume 1: Long Papers, July 26--31, 2015, Beijing, China. 1014--1023.Google ScholarGoogle Scholar
  46. Peter D. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6--12, 2002, Philadelphia, PA. 417--424.Google ScholarGoogle Scholar
  47. Xiaojun Wan. 2009. Co-training for cross-lingual sentiment classification. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, August 2--7, 2009, Singapore. 235--243.Google ScholarGoogle ScholarCross RefCross Ref
  48. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, October 6--8, 2005, Vancouver, British Columbia, Canada. 347--354.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Weidi Xu, Haoze Sun, Chao Deng, and Ying Tan. 2017. Variational autoencoder for semi-supervised text classification. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, February 4--9, 2017, San Francisco, California. 3358--3364.Google ScholarGoogle Scholar
  50. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 12--17, 2016, San Diego, California. 1480--1489.Google ScholarGoogle Scholar
  51. Huaguang Zhang, Zhanshan Wang, and Derong Liu. 2014. A comprehensive review of stability analysis of continuous-time recurrent neural networks. IEEE Trans. Neural Netw. Learning Syst. 25, 7 (2014), 1229--1262.Google ScholarGoogle ScholarCross RefCross Ref
  52. Xiaojin Zhu. 2006. Semi-supervised learning literature survey. Computer Science, University of Wisconsin-Madison 2, 3 (2006), 4.Google ScholarGoogle Scholar

Index Terms

  1. Neural Co-training for Sentiment Classification with Product Attributes

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 19, Issue 5
      September 2020
      278 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3403646
      Issue’s Table of Contents

      Copyright © 2020 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 4 August 2020
      • Online AM: 7 May 2020
      • Accepted: 1 April 2020
      • Revised: 1 February 2020
      • Received: 1 May 2019
      Published in tallip Volume 19, Issue 5

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!