Abstract
As a common language form in oral communication, short text is hard to be used in the applications such as intent understanding, text classification and so on due to its limited content and information, as well as irregular expression and missing components. To increase the availability of short texts in real applications, we propose a Label Information Assisting-based Model (LIAM) for Chinese short text classification. In the model, we jointly use sentence-level features and word-level features to reduce text information loss. And the sentence-level features are fused with relevant label information by the Label Information Extending and Fusion (LIEF) module while the word-level features are also enhanced with assistance of relevant label information. By utilizing the text-related information from labels as extended information, the model enriches and enhances the features of short text, benefiting classification. To verify the correctness and effectiveness of the proposed method, we conduct extensive experiments on four Chinese datasets and six sub-datasets with different models. The experimental results show that LIAM presented can effectively enrich information for text and much improve the performance of short text classification. It performs much better than other methods do. What is more, the less the training set, the greater the advantages of the model.
- [1] . 2019. Deep short text classification with knowledge powered attention. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 6252–6259.Google Scholar
- [2] . 2021. Pre-training with whole word masking for chinese BERT. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 3504–3514.Google Scholar
Digital Library
- [3] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). 4171–4186.Google Scholar
- [4] . 2019. Explicit interaction model towards text classification. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 6359–6366.Google Scholar
- [5] . 2018. A novel active learning method using SVM for text classification. Int. J. Autom. Comput. 15, 3 (2018), 290–298.Google Scholar
Digital Library
- [6] . 2021. Label confusion learning to enhance text classification models. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. 12929–12936.Google Scholar
- [7] . 2020. Chinese short text classification with mutual-attention convolutional neural networks. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 5 (2020), 61:1–61:13.Google Scholar
Digital Library
- [8] . 2021. An intention multiple-representation model with expanded information. Comput. Speech Lang. 68 (2021), 101196.Google Scholar
Cross Ref
- [9] . 2022. Text representation model for multiple language forms in spoken chinese expression. International Journal of Pattern Recognition and Artificial Intelligence (2022).Google Scholar
Cross Ref
- [10] . 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, and (Eds.). 562–570.Google Scholar
Cross Ref
- [11] . 2006. Some effective techniques for naive bayes text classification. IEEE Trans. Knowl. Data Eng. 18, 11 (2006), 1457–1466.Google Scholar
Digital Library
- [12] . 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. 1746–1751.Google Scholar
Cross Ref
- [13] . 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.Google Scholar
- [14] . 2021. Medical term and status generation from chinese clinical dialogue with multi-granularity transformer. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 3362–3374.Google Scholar
Digital Library
- [15] . 2018. Transformation networks for target-oriented sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. 946–956.Google Scholar
Cross Ref
- [16] . 2021. Merging statistical feature via adaptive gate for improved text classification. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. 13288–13296.Google Scholar
- [17] . 2021. Deep structured learning for natural language processing. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 3 (2021), 53:1–53:14.Google Scholar
Digital Library
- [18] . 2021. Multi-label text classification via joint learning from label embedding and label correlation. Neurocomputing 460 (2021), 385–398.Google Scholar
Digital Library
- [19] . 2016. Recurrent neural network for text classification with multi-task learning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, (Ed.). 2873–2879.Google Scholar
- [20] . 2022. Combining context-relevant features with multi-stage attention network for short text classification. Comput. Speech Lang. 71 (2022), 101268.Google Scholar
Digital Library
- [21] . 2022. Hybrid embedding-based text representation for hierarchical multi-label text classification. Expert Syst. Appl. 187 (2022), 115905.Google Scholar
Cross Ref
- [22] . 2020. Text classification using label names only: A language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020. 9006–9017.Google Scholar
Cross Ref
- [23] . 2021. Deep learning-based text classification: A comprehensive review. ACM Comput. Surv. 54, 3 (2021), 62:1–62:40.Google Scholar
- [24] . 2017. Comparison of naive bayes, random forest, decision tree, support vector machines, and logistic regression classifiers for text reviews classification. Balt. J. Mod. Comput. 5, 2 (2017).Google Scholar
- [25] . 2021. Sentiment analysis of sinhala news comments. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 4 (2021), 59:1–59:23.Google Scholar
Digital Library
- [26] . 2004. Understanding inverse document frequency: On theoretical arguments for IDF. J. Documentation 60, 5 (2004), 503–520.Google Scholar
Cross Ref
- [27] . 2021. Deep interactive memory network for aspect-level sentiment analysis. ACM Trans. Asian Low Resour. Lang. Inf. Process. 20, 1 (2021), 3:1–3:12.Google Scholar
Digital Library
- [28] . 2021. Pattern-enhanced contrastive policy learning network for sequential recommendation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021. 1593–1599.Google Scholar
Cross Ref
- [29] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 5998–6008.Google Scholar
- [30] . 2018. Joint embedding of words and labels for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. 2321–2331.Google Scholar
Cross Ref
- [31] . 2021. A short text classification method based on convolutional neural network and semantic extension. Int. J. Comput. Intell. Syst. 14, 1 (2021), 367–375.Google Scholar
Cross Ref
- [32] . 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, July 8-14, 2012, Jeju Island, Korea - Volume 2: Short Papers. 90–94.Google Scholar
- [33] . 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 5754–5764.Google Scholar
- [34] . 2018. Multi-task label embedding for text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. 4545–4553.Google Scholar
Cross Ref
- [35] . 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, PACLIC 29, Shanghai, China, October 30 - November 1, 2015.Google Scholar
- [36] . 2017. The first evaluation of chinese human-computer dialogue technology. CoRR abs/1709.10217 (2017).
arXiv:1709.10217 http://arxiv.org/abs/1709.10217Google Scholar - [37] . Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers.Google Scholar
- [38] . 2020. A survey on text classification and its applications. Web Intell. 18, 3 (2020), 205–216.Google Scholar
Cross Ref
- [39] . 2016. Compositional recurrent neural networks for chinese short text classification. In 2016 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2016, Omaha, NE, USA, October 13-16, 2016. 137–144.Google Scholar
Cross Ref
Index Terms
Short Text Classification of Chinese with Label Information Assisting
Recommendations
Chinese Short Text Classification with Mutual-Attention Convolutional Neural Networks
The methods based on the combination of word-level and character-level features can effectively boost performance on Chinese short text classification. A lot of works concatenate two-level features with little processing, which leads to losing feature ...
Multiple weak supervision for short text classification
AbstractFor short text classification, insufficient labeled data, data sparsity, and imbalanced classification have become three major challenges. For this, we proposed multiple weak supervision, which can label unlabeled data automatically. Different ...
Semi-supervised multi-label classification using incomplete label information
Highlights- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
AbstractClassifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...






Comments