skip to main content
research-article
Open Access

Global Encoding for Long Chinese Text Summarization

Published:06 October 2020Publication History
Skip Abstract Section

Abstract

Text summarization is one of the significant tasks of natural language processing, which automatically converts text into a summary. Some summarization systems, for short/long English, and short Chinese text, benefit from advances in the neural encoder-decoder model because of the availability of large datasets. However, the long Chinese text summarization research has been limited to datasets of a couple of hundred instances. This article aims to explore the long Chinese text summarization task. To begin with, we construct a first large-scale, long Chinese text summarization corpus, the Long Chinese Summarization of Police Inquiry Record Text (LCSPIRT). Based on this corpus, we propose a sequence-to-sequence (Seq2Seq) model that incorporates a global encoding process with an attention mechanism. Our model achieves a competitive result on the LCSPIRT corpus compared with several benchmark methods.

References

  1. Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:arXiv preprint arXiv:1409.0473 (2014).Google ScholarGoogle Scholar
  2. Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 1662--1675.Google ScholarGoogle ScholarCross RefCross Ref
  3. Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Meeting of the Association for Computational Linguistics. 484--494.Google ScholarGoogle ScholarCross RefCross Ref
  4. Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder--decoder approaches. In Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST’14). 103--111.Google ScholarGoogle ScholarCross RefCross Ref
  5. Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 93--98.Google ScholarGoogle Scholar
  6. Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). 615--621.Google ScholarGoogle ScholarCross RefCross Ref
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171--4186.Google ScholarGoogle Scholar
  8. Yue Dong, Yikang Shen, Eric Crawford, Herke van Hoof, and Jackie Chi Kit Cheung. 2018. BanditSum: Extractive summarization as a contextual bandit. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 3739--3748.Google ScholarGoogle ScholarCross RefCross Ref
  9. Günes Erkan and Dragomir R. Radev. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. J. Arti. Intell. Res. 22 (2004), 457--479.Google ScholarGoogle ScholarCross RefCross Ref
  10. Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. 2019. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proceedings of the 57th Meeting of the Association for Computational Linguistics. 1074--1084.Google ScholarGoogle Scholar
  11. Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 4098--4109.Google ScholarGoogle ScholarCross RefCross Ref
  12. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th Meeting of the Association for Computational Linguistics. 1631--1640.Google ScholarGoogle Scholar
  13. Yongshuai Hou, Yang Xiang, Buzhou Tang, Qingcai Chen, Xiaolong Wang, and Fangze Zhu. 2017. Identifying high quality document--summary pairs through text matching. Information 8, 2 (2017), 64.Google ScholarGoogle ScholarCross RefCross Ref
  14. Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015. LCSTS: A large scale Chinese short text summarization dataset. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1967--1972.Google ScholarGoogle ScholarCross RefCross Ref
  15. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:arXiv preprint arXiv:1412.6980 (2014).Google ScholarGoogle Scholar
  16. Diederik P. Kingma and Jimmy Ba. 2019. Scoring sentence singletons and pairs for abstractive summarization. arXiv:arXiv preprint arXiv:1906.00077 (2019).Google ScholarGoogle Scholar
  17. Piji Li, Wai Lam, Lidong Bing, and Zihao Wang. 2017. Deep recurrent generative decoder for abstractive text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2091--2100.Google ScholarGoogle ScholarCross RefCross Ref
  18. Wei Li, Xinyan Xiao, Yajuan Lyu, and Yuanzhuo Wang. 2018. Improving neural abstractive document summarization with explicit information selection modeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1787--1796.Google ScholarGoogle ScholarCross RefCross Ref
  19. Zhixin Li, Zhi Peng, Suqin Tang, Canlong Zhang, and Huifang Ma. 2020. Text summarization method based on double attention pointer network. IEEE Access 8 (2020), 11279--11288.Google ScholarGoogle ScholarCross RefCross Ref
  20. Chin-Yew Lin and Eduard Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Junyang Lin, Xu Sun, Shuming Ma, and Qi Su. 2018. Global encoding for abstractive summarization. In Proceedings of the 56th Meeting of the Association for Computational Linguistics (Volume 2). 163--169.Google ScholarGoogle ScholarCross RefCross Ref
  22. Jeffrey Ling and Alexander Rush. 2017. Coarse-to-fine attention models for document summarization. In Proceedings of the Workshop on New Frontiers in Summarization. 33--42.Google ScholarGoogle ScholarCross RefCross Ref
  23. Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018. Generating Wikipedia by summarizing long sequences. arXiv:arXiv preprint arXiv:1801.10198 (2018).Google ScholarGoogle Scholar
  24. Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 3721--3731.Google ScholarGoogle ScholarCross RefCross Ref
  25. Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine translation. In Proceedings of the 53rd Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Vol. 1. 11--19.Google ScholarGoogle ScholarCross RefCross Ref
  26. Shuming Ma and Xu Sun. 2017. A semantic relevance based neural network for text summarization and text simplification. arXiv:arXiv preprint arXiv:1710.02318 (2017).Google ScholarGoogle Scholar
  27. Shuming Ma, Xu Sun, Junyang Lin, and Xuancheng Ren. 2018. A hierarchical end-to-end model for jointly improving text summarization and sentiment classification. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. AAAI Press, 4251--4257.Google ScholarGoogle ScholarCross RefCross Ref
  28. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.Google ScholarGoogle Scholar
  29. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 280--290.Google ScholarGoogle ScholarCross RefCross Ref
  30. Kishore Papineni, Salim Roukos, Todd Ward, and Weijing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Meeting of the Association for Computational Linguistics. 311--318.Google ScholarGoogle Scholar
  31. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 379--389.Google ScholarGoogle ScholarCross RefCross Ref
  32. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Meeting of the Association for Computational Linguistics. 1073--1083.Google ScholarGoogle Scholar
  33. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818--2826.Google ScholarGoogle ScholarCross RefCross Ref
  34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 5998--6008.Google ScholarGoogle Scholar
  35. Bingzhen Wei, Xuancheng Ren, Yi Zhang, Xiaoyan Cai, Qi Su, and Xu Sun. 2019. Regularizing output distribution of abstractive Chinese social media text summarization for improved semantic consistency. ACM Trans. Low-resour. Lang. Inf. Proc. 18, 3 (2019), 1--15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019. MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 563--578.Google ScholarGoogle ScholarCross RefCross Ref
  37. Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. In Proceedings of the 56th Meeting of the Association for Computational Linguistics. 654--663.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Global Encoding for Long Chinese Text Summarization

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 19, Issue 6
          November 2020
          277 pages
          ISSN:2375-4699
          EISSN:2375-4702
          DOI:10.1145/3426881
          Issue’s Table of Contents

          Copyright © 2020 Owner/Author

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 6 October 2020
          • Accepted: 1 June 2020
          • Revised: 1 April 2020
          • Received: 1 August 2019
          Published in tallip Volume 19, Issue 6

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!