skip to main content
research-article

A Novel Classification Model SA-MPCNN for Power Equipment Defect Text

Authors Info & Claims
Published:12 August 2021Publication History
Skip Abstract Section

Abstract

The text classification of power equipment defect is of great significance to equipment health condition evaluation and power equipment maintenance decisions. Most of the existing classification methods do not sufficiently consider the semantic relation between words in the same sentence and cannot extract deep semantic features. To tackle those problems, this article proposes a novel classification method by combining the self-attention mechanism and multi-channel pyramid convolution neural networks. We utilize the bidirectional gated recurrent unit to model the text sequence and, on this basis, improve self-attention layer to dot multiplication on the forward and backward features to obtain the global attention score. Thereby, effective features are enhanced, invalid features are weakened, and important text representation vectors are obtained. To solve the problem that the shallow network structure cannot extract deep semantic features, we design a multi-channel pyramid convolution network, which first extracts deep text features from the channels of different windows and then fuses the text features of each channel. By comparing with the state-of-the-art methods, the model in this article has better performance in text classification of power equipment defects.

References

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473. Retrieved from https://arxiv.org/abs/1409.0473.Google ScholarGoogle Scholar
  2. Vishwanath Bijalwan, Vinay Kumar, Pinki Kumari, and Jordan Pascual. 2014. KNN based machine learning approach for text and document mining. Int. J. Datab. Theory Appl. 7, 1 (2014), 61–70.Google ScholarGoogle ScholarCross RefCross Ref
  3. Jindong Chen, Yizhou Hu, Jingping Liu, Yanghua Xiao, and Haiyun Jiang. 2019. Deep short text classification with knowledge powered attention. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6252–6259.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Yushi Chen, Jelena Ponocko, Nikola Milosevic, Goran Nenadic, and Jovica V. Milanovic. 2016. Towards application of text mining for enhanced power network data analytics—part ii: Offline analysis of textual data. (2016), 1–8.Google ScholarGoogle Scholar
  5. Xie Cheng, Guoping Zou, Huifang Wang, and Yongtao Jin. 2016. A new condition assessment method for distribution transformers based on operation data and record text mining technique. In Proceedings of the China International Conference on Electricity Distribution.Google ScholarGoogle Scholar
  6. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555. Retrieved from https://arxiv.org/abs/1412.3555.Google ScholarGoogle Scholar
  7. Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2015. Gated feedback recurrent neural networks. In Proceedings of the International Conference on Machine Learning. 2067–2075. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. (2018), 4171–4186.Google ScholarGoogle Scholar
  9. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 6645–6649.Google ScholarGoogle ScholarCross RefCross Ref
  10. Qipeng Guo, Xipeng Qiu, Pengfei Liu, Xiangyang Xue, and Zheng Zhang. 2019. Multi-scale self-attention for text classification. arXiv:1912.00544. Retrieved from https://arxiv.org/abs/1912.00544.Google ScholarGoogle Scholar
  11. Abdalraouf Hassan and Ausif Mahmood. 2017. Efficient deep learning model for text classification based on recurrent and convolutional layers. In Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA’17). IEEE, 1108–1113.Google ScholarGoogle ScholarCross RefCross Ref
  12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neur. Comput. 9, 8 (1997), 1735–1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Wei Jiang and Zhong Jin. 2017. Integrating bidirectional LSTM with inception for text classification. In Proceedings of the 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR’17). IEEE, 870–875.Google ScholarGoogle ScholarCross RefCross Ref
  14. Rie Johnson and Tong Zhang. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 562–570.Google ScholarGoogle ScholarCross RefCross Ref
  15. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv:1408.5882. Retrieved from https://arxiv.org/abs/1408.5882.Google ScholarGoogle Scholar
  16. Kamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, and Laura E. Barnes. 2017. HDLTex: Hierarchical deep learning for text classification. (2017), 1–25.Google ScholarGoogle Scholar
  17. Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Gaël Letarte, Frédérik Paradis, Philippe Giguère, and François Laviolette. 2018. Importance of self-attention for sentiment analysis. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. 267–275.Google ScholarGoogle ScholarCross RefCross Ref
  19. Yuancheng Li, Pan Zhang, and Rong Huang. 2019. Lightweight quantum encryption for secure transmission of power data in smart grid. IEEE Access 7 (2019), 36285–36293.Google ScholarGoogle ScholarCross RefCross Ref
  20. Ziquan Liu, Huifang Wang, Jing Cao, and Jian Qiu. 2018. A classification model of power equipment defect texts based on convolutional neural network. Power System Technology (2018), 475–487.Google ScholarGoogle Scholar
  21. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv:1508.04025. Retrieved from https://arxiv.org/abs/1508.04025.Google ScholarGoogle Scholar
  22. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781. Retrieved from https://arxiv.org/abs/1301.3781.Google ScholarGoogle Scholar
  23. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Nikolaos Pappas and Andrei Popescu-Belis. 2017. Multilingual hierarchical attention networks for document classification. arXiv:1707.00896. Retrieved from https://arxiv.org/abs/1707.00896.Google ScholarGoogle Scholar
  25. Tomas Pranckevičius and Virginijus Marcinkevičius. 2016. Application of logistic regression with part-of-the-speech tagging for multi-class text classification. In Proceedings of the 2016 IEEE 4th Workshop on Advances in Information, Electronic and Electrical Engineering (AIEEE’16). IEEE, 1–5.Google ScholarGoogle ScholarCross RefCross Ref
  26. Jian Qiu, Huifang Wang, Gaoliang Ying, Bo Zhang, Guoping Zou, and H. E. Benteng. 2016. Text mining technique and application of lifecycle condition assessment for circuit breaker. Autom. Electr. Power Syst. 40, 6 (2016), 107–112.Google ScholarGoogle Scholar
  27. C. Rudin, D. Waltz, R. N. Anderson, A. Boulanger, A. Salleb-Aouissi, M. Chow, H. Dutta, P. N. Gross, B. Huang, and S. Ierome. 2012. Machine learning for the new york city power grid. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2 (2012), 328–345. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Feng Shen, Xiong Luo, and Yi Chen. 2013. Text classification dimension reduction algorithm for chinese web page based on deep learning. (2013), 451–456.Google ScholarGoogle Scholar
  29. Weiwei Shi. 2015. Improving power grid monitoring data quality: An efficient machine learning framework for missing data prediction. In Proceedings of the IEEE International Conference on High Performance Computing and Communications (HPCC’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Mengjiao Song, Xingyu Zhao, Yong Liu, and Zhihong Zhao. 2018. Text sentiment analysis based on convolutional neural network and bidirectional LSTM model. In Proceedings of the International Conference of Pioneering Computer Scientists, Engineers and Educators. Springer, 55–68.Google ScholarGoogle ScholarCross RefCross Ref
  31. Haifeng Sun, Zhaoyu Wang, Jianhui Wang, Zhen Huang, NichelleLe Carrington, and Jianxin Liao. 2016. Data-driven power outage detection by social sensors. IEEE Trans. Smart Grid 7, 5 (2016), 2516–2524.Google ScholarGoogle ScholarCross RefCross Ref
  32. Haifeng Sun, Zhaoyu Wang, Jianhui Wang, Huang Zhen, Nichelle Le Carrington, and Jianxin Liao. 2016. Data-driven power outage detection by social sensors. IEEE Trans. Smart Grid 7, 5 (2016), 2516–2524.Google ScholarGoogle ScholarCross RefCross Ref
  33. Qinting Tang, Jiayu Chen, Hengtong Lu, Yu Du, Kehan Yang, et al. 2019. Full attention-based bi-GRU neural network for news text classification. In Proceedings of the 2019 IEEE 5th International Conference on Computer and Communications (ICCC’19). IEEE, 1970–1974.Google ScholarGoogle ScholarCross RefCross Ref
  34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Hui-fang Wang and Zi-quan Liu. 2019. An error recognition method for power equipment defect records based on knowledge graph technology. Front. Inf. Technol. Electr. Eng. 20, 11 (2019), 1564–1577.Google ScholarGoogle ScholarCross RefCross Ref
  36. Shiyao Wang, Minlie Huang, and Zhidong Deng. 2018. Densely connected CNN with multi-scale feature attention for text classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’18). 4468–4474. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Sida I. Wang and Christopher D. Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 90–94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Daqian Wei, Bo Wang, Gang Lin, Dichen Liu, Zhaoyang Dong, Hesen Liu, and Yilu Liu. 2017. Research on unstructured text data mining and fault classification based on RNN-LSTM with malfunction inspection report. Energies 10, 3 (2017), 406–.Google ScholarGoogle ScholarCross RefCross Ref
  39. Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv:1901.11196. Retrieved from https://arxiv.org/abs/1901.11196.Google ScholarGoogle Scholar
  40. Xin Wu, Yi Cai, Qing Li, Jingyun Xu, and Ho-fung Leung. 2018. Combining contextual information by self-attention mechanism in convolutional neural networks for text classification. In Proceedings of the International Conference on Web Information Systems Engineering. Springer, 453–467.Google ScholarGoogle ScholarCross RefCross Ref
  41. Zheng Xiao and PiJun Liang. 2016. Chinese sentiment analysis using bidirectional LSTM with word embedding. In Proceedings of the International Conference on Cloud Computing and Security. Springer, 601–610.Google ScholarGoogle ScholarCross RefCross Ref
  42. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1480–1489.Google ScholarGoogle ScholarCross RefCross Ref
  43. Wenpeng Yin, Katharina Kann, Mo Yu, and Hinrich Schütze. 2017. Comparative study of cnn and rnn for natural language processing. arXiv:1702.01923. Retrieved from https://arxiv.org/abs/1702.01923.Google ScholarGoogle Scholar
  44. Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria. 2018. Recent trends in deep learning based natural language processing. IEEE Comput. Intell. Mag. 13, 3 (2018), 55–75.Google ScholarGoogle ScholarCross RefCross Ref
  45. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. arXiv:1509.01626. Retrieved from https://arxiv.org/abs/1509.01626. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Novel Classification Model SA-MPCNN for Power Equipment Defect Text

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!