skip to main content
research-article

Short Text Classification of Chinese with Label Information Assisting

Authors Info & Claims
Published:25 March 2023Publication History
Skip Abstract Section

Abstract

As a common language form in oral communication, short text is hard to be used in the applications such as intent understanding, text classification and so on due to its limited content and information, as well as irregular expression and missing components. To increase the availability of short texts in real applications, we propose a Label Information Assisting-based Model (LIAM) for Chinese short text classification. In the model, we jointly use sentence-level features and word-level features to reduce text information loss. And the sentence-level features are fused with relevant label information by the Label Information Extending and Fusion (LIEF) module while the word-level features are also enhanced with assistance of relevant label information. By utilizing the text-related information from labels as extended information, the model enriches and enhances the features of short text, benefiting classification. To verify the correctness and effectiveness of the proposed method, we conduct extensive experiments on four Chinese datasets and six sub-datasets with different models. The experimental results show that LIAM presented can effectively enrich information for text and much improve the performance of short text classification. It performs much better than other methods do. What is more, the less the training set, the greater the advantages of the model.

REFERENCES

  1. [1] Chen Jindong, Hu Yizhou, Liu Jingping, Xiao Yanghua, and Jiang Haiyun. 2019. Deep short text classification with knowledge powered attention. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 62526259.Google ScholarGoogle Scholar
  2. [2] Cui Yiming, Che Wanxiang, Liu Ting, Qin Bing, and Yang Ziqing. 2021. Pre-training with whole word masking for chinese BERT. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 35043514.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Devlin Jacob, Chang Mingwei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). 41714186.Google ScholarGoogle Scholar
  4. [4] Du Cunxiao, Chen Zhaozheng, Feng Fuli, Zhu Lei, Gan Tian, and Nie Liqiang. 2019. Explicit interaction model towards text classification. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 63596366.Google ScholarGoogle Scholar
  5. [5] Goudjil Mohamed, Koudil Mouloud, Bedda Mouldi, and Ghoggali Noureddine. 2018. A novel active learning method using SVM for text classification. Int. J. Autom. Comput. 15, 3 (2018), 290298.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Guo Biyang, Han Songqiao, Han Xiao, Huang Hailiang, and Lu Ting. 2021. Label confusion learning to enhance text classification models. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. 1292912936.Google ScholarGoogle Scholar
  7. [7] Hao Ming, Xu Bo, Liang Jingyi, Zhang Bowen, and Yin Xucheng. 2020. Chinese short text classification with mutual-attention convolutional neural networks. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 5 (2020), 61:1–61:13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Hu Jingxiang, Peng Junjie, Zhang Wenqiang, Qi Lizhe, Hu Miao, and Zhang Huanxiang. 2021. An intention multiple-representation model with expanded information. Comput. Speech Lang. 68 (2021), 101196.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Hu Miao, Peng Junjie, Zhang Wenqiang, Hu Jingxiang, Qi Lizhe, and Zhang Huanxiang. 2022. Text representation model for multiple language forms in spoken chinese expression. International Journal of Pattern Recognition and Artificial Intelligence (2022).Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Johnson Rie and Zhang Tong. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Barzilay Regina and Kan Min-Yen (Eds.). 562570.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Kim Sang-Bum, Han Kyoung-Soo, Rim Hae-Chang, and Myaeng Sung-Hyon. 2006. Some effective techniques for naive bayes text classification. IEEE Trans. Knowl. Data Eng. 18, 11 (2006), 14571466.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Kim Yoon. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. 17461751.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.Google ScholarGoogle Scholar
  14. [14] Li Mei, Xiang Lu, Kang Xiaomian, Zhao Yang, Zhou Yu, and Zong Chengqing. 2021. Medical term and status generation from chinese clinical dialogue with multi-granularity transformer. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 33623374.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Li Xin, Bing Lidong, Lam Wai, and Shi Bei. 2018. Transformation networks for target-oriented sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. 946956.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Li Xianming, Li Zongxi, Xie Haoran, and Li Qing. 2021. Merging statistical feature via adaptive gate for improved text classification. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. 1328813296.Google ScholarGoogle Scholar
  17. [17] Li Yong, Yang Xiaojun, Zuo Min, Jin Qingyu, Li Haisheng, and Cao Qian. 2021. Deep structured learning for natural language processing. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 3 (2021), 53:1–53:14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Liu Huiting, Chen Geng, Li Pei-Pei, Zhao Peng, and Wu Xindong. 2021. Multi-label text classification via joint learning from label embedding and label correlation. Neurocomputing 460 (2021), 385398.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Liu Pengfei, Qiu Xipeng, and Huang Xuanjing. 2016. Recurrent neural network for text classification with multi-task learning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, Kambhampati Subbarao (Ed.). 28732879.Google ScholarGoogle Scholar
  20. [20] Liu Yingying, Li Peipei, and Hu Xuegang. 2022. Combining context-relevant features with multi-stage attention network for short text classification. Comput. Speech Lang. 71 (2022), 101268.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Ma Yinglong, Liu Xiaofeng, Zhao Lijiao, Liang Yue, Zhang Peng, and Jin Beihong. 2022. Hybrid embedding-based text representation for hierarchical multi-label text classification. Expert Syst. Appl. 187 (2022), 115905.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Meng Yu, Zhang Yunyi, Huang Jiaxin, Xiong Chenyan, Ji Heng, Zhang Chao, and Han Jiawei. 2020. Text classification using label names only: A language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020. 90069017.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Minaee Shervin, Kalchbrenner Nal, Cambria Erik, Nikzad Narjes, Chenaghlu Meysam, and Gao Jianfeng. 2021. Deep learning-based text classification: A comprehensive review. ACM Comput. Surv. 54, 3 (2021), 62:1–62:40.Google ScholarGoogle Scholar
  24. [24] Pranckevicius Tomas and Marcinkevicius Virginijus. 2017. Comparison of naive bayes, random forest, decision tree, support vector machines, and logistic regression classifiers for text reviews classification. Balt. J. Mod. Comput. 5, 2 (2017).Google ScholarGoogle Scholar
  25. [25] Ranathunga Surangika and Liyanage Isuru Udara. 2021. Sentiment analysis of sinhala news comments. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 4 (2021), 59:1–59:23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Robertson Stephen. 2004. Understanding inverse document frequency: On theoretical arguments for IDF. J. Documentation 60, 5 (2004), 503520.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Sun Chengai, Lv Liangyu, Tian Gang, and Liu Tailu. 2021. Deep interactive memory network for aspect-level sentiment analysis. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 1 (2021), 3:1–3:12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Tong Xiaohai, Wang Pengfei, Li Chenliang, Xia Long, and Niu ShaoZhang. 2021. Pattern-enhanced contrastive policy learning network for sequential recommendation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021. 15931599.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 59986008.Google ScholarGoogle Scholar
  30. [30] Wang Guoyin, Li Chunyuan, Wang Wenlin, Zhang Yizhe, Shen Dinghan, Zhang Xinyuan, Henao Ricardo, and Carin Lawrence. 2018. Joint embedding of words and labels for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. 23212331.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Wang Haitao, Tian Keke, Wu Zhengjiang, and Wang Lei. 2021. A short text classification method based on convolutional neural network and semantic extension. Int. J. Comput. Intell. Syst. 14, 1 (2021), 367375.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Wang Sida and Manning Christopher D.. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, July 8-14, 2012, Jeju Island, Korea - Volume 2: Short Papers. 9094.Google ScholarGoogle Scholar
  33. [33] Yang Zhilin, Dai Zihang, Yang Yiming, Carbonell Jaime G., Salakhutdinov Ruslan, and Le Quoc V.. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 57545764.Google ScholarGoogle Scholar
  34. [34] Zhang Honglun, Xiao Liqiang, Chen Wenqing, Wang Yongkun, and Jin Yaohui. 2018. Multi-task label embedding for text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. 45454553.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Zhang Shu, Zheng Dequan, Hu Xinchen, and Yang Ming. 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, PACLIC 29, Shanghai, China, October 30 - November 1, 2015.Google ScholarGoogle Scholar
  36. [36] Zhang Weinan, Chen Zhigang, Che Wanxiang, Hu Guoping, and Liu Ting. 2017. The first evaluation of chinese human-computer dialogue technology. CoRR abs/1709.10217 (2017). arXiv:1709.10217 http://arxiv.org/abs/1709.10217Google ScholarGoogle Scholar
  37. [37] Zhou Peng, Shi Wei, Tian Jun, Qi Zhenyu, Li Bingchen, Hao Hongwei, and Xu Bo. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers.Google ScholarGoogle Scholar
  38. [38] Zhou Xujuan, Gururajan Raj, Li Yuefeng, Venkataraman Revathi, Tao Xiaohui, Bargshady Ghazal, Barua Prabal Datta, and Kondalsamy-Chennakesavan Srinivas. 2020. A survey on text classification and its applications. Web Intell. 18, 3 (2020), 205216.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Zhou Yujun, Xu Bo, Xu Jiaming, Yang Lei, Li Changliang, and Xu Bo. 2016. Compositional recurrent neural networks for chinese short text classification. In 2016 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2016, Omaha, NE, USA, October 13-16, 2016. 137144.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Short Text Classification of Chinese with Label Information Assisting

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 4
      April 2023
      682 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3588902
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 25 March 2023
      • Online AM: 8 February 2023
      • Accepted: 21 January 2023
      • Revised: 30 November 2022
      • Received: 16 July 2021
      Published in tallip Volume 22, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!