skip to main content
research-article

Memorizing All for Implicit Discourse Relation Recognition

Authors Info & Claims
Published:13 December 2021Publication History
Skip Abstract Section

Abstract

Implicit discourse relation recognition is a challenging task due to the absence of the necessary informative clues from explicit connectives. An implicit discourse relation recognizer has to carefully tackle the semantic similarity of sentence pairs and the severe data sparsity issue. In this article, we learn token embeddings to encode the structure of a sentence from a dependency point of view in their representations and use them to initialize a baseline model to make it really strong. Then, we propose a novel memory component to tackle the data sparsity issue by allowing the model to master the entire training set, which helps in achieving further performance improvement. The memory mechanism adequately memorizes information by pairing representations and discourse relations of all training instances, thus filling the slot of the data-hungry issue in the current implicit discourse relation recognizer. The proposed memory component, if attached with any suitable baseline, can help in performance enhancement. The experiments show that our full model with memorizing the entire training data provides excellent results on PDTB and CDTB datasets, outperforming the baselines by a fair margin.

REFERENCES

  1. Bai Hongxiao and Zhao Hai. 2018. Deep enhanced representation for implicit discourse relation recognition. In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18). 571583.Google ScholarGoogle Scholar
  2. Braud Chloé and Denis Pascal. 2015. Comparing word representations for implicit discourse relation classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 22012211.Google ScholarGoogle ScholarCross RefCross Ref
  3. Cai Jiaxun, He Shexia, Li Zuchao, and Zhao Hai. 2018. A full end-to-end semantic role labeler, syntactic-agnostic over syntactic-aware? In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18). 27532765.Google ScholarGoogle Scholar
  4. Chen Jifan, Zhang Qi, Liu Pengfei, and Huang Xuanjing. 2016a. Discourse relations detection via a mixed generative-discriminative framework. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16). 29212927. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chen Jifan, Zhang Qi, Liu Pengfei, Qiu Xipeng, and Huang Xuanjing. 2016b. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 17261735.Google ScholarGoogle ScholarCross RefCross Ref
  6. Dai Zeyu and Huang Ruihong. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’18). 141151.Google ScholarGoogle ScholarCross RefCross Ref
  7. Dasigi Pradeep, Ammar Waleed, Dyer Chris, and Hovy Eduard. 2017. Ontology-aware token embeddings for prepositional phrase attachment. In Proceedings of the Association for Computational Linguistics (ACL’17).Google ScholarGoogle ScholarCross RefCross Ref
  8. Dauphin Yann N., Fan Angela, Auli Michael, and Grangier David. 2017. Language modeling with gated convolutional networks. In Proceedings of the International Conference on Machine Learning. PMLR, 933941. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Dozat Timothy and Manning Christopher D.. 2017. Deep biaffine attention for neural dependency parsing. In Proceedings of the International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  10. Gerani Shima, Mehdad Yashar, Carenini Giuseppe, Ng Raymond T., and Nejat Bita. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 16021613.Google ScholarGoogle ScholarCross RefCross Ref
  11. Guo Fengyu, He Ruifang, Jin Di, Dang Jianwu, Wang Longbiao, and Li Xiangang. 2018. Implicit discourse relation recognition using neural tensor network with interactive attention and sparse learning. In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18). 547558.Google ScholarGoogle Scholar
  12. He Shexia, Li Zuchao, Zhao Hai, and Bai Hongxiao. 2018. Syntax for semantic role labeling, to be, or not to be. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18). 20612071.Google ScholarGoogle ScholarCross RefCross Ref
  13. Honnibal Matthew and Johnson Mark. 2015. An improved non-monotonic transition system for dependency parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 13731378.Google ScholarGoogle ScholarCross RefCross Ref
  14. Jansen Peter, Surdeanu Mihai, and Clark Peter. 2014. Discourse complements lexical semantics for non-factoid answer reranking. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14). 977986.Google ScholarGoogle ScholarCross RefCross Ref
  15. Ji Yangfeng and Eisenstein Jacob. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Trans. Assoc. Comput. Linguist. 3 (2015), 329344.Google ScholarGoogle ScholarCross RefCross Ref
  16. Ji Yangfeng, Haffari Gholamreza, and Eisenstein Jacob. 2016. A latent variable recurrent neural network for discourse-driven language models. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’16). 332342.Google ScholarGoogle ScholarCross RefCross Ref
  17. Jia Yanyan, Ye Yuan, Feng Yansong, Lai Yuxuan, Yan Rui, and Zhao Dongyan. 2018. Modeling discourse cohesion for discourse parsing via memory network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18). 438443.Google ScholarGoogle ScholarCross RefCross Ref
  18. Kishimoto Yudai, Murawaki Yugo, and Kurohashi Sadao. 2018. A knowledge-augmented neural network model for implicit discourse relation classification. In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18). 584595.Google ScholarGoogle Scholar
  19. Kumar Ankit, Irsoy Ozan, Ondruska Peter, Iyyer Mohit, Bradbury James, Gulrajani Ishaan, Zhong Victor, Paulus Romain, and Socher Richard. 2016. Ask me anything: Dynamic memory networks for natural language processing. In Proceedings of the International Conference on Machine Learning (ICML’16). 13781387. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Lan Man, Wang Jianxiang, Wu Yuanbin, Niu Zheng-Yu, and Wang Haifeng. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’17). 12991308.Google ScholarGoogle ScholarCross RefCross Ref
  21. Lei Wenqiang, Wang Xuancong, Liu Meichun, Ilievski Ilija, He Xiangan, and Kan Min-Yen. 2017. SWIM: A simple word interaction model for implicit discourse relation recognition. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI’17). 40264032. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Lei Wenqiang, Xiang Yuanxin, Wang Yuwei, Zhong Qian, Liu Meichun, and Kan Min-Yen. 2018. Linguistic properties matter for implicit discourse relation recognition: Combining semantic interaction, topic continuity and attribution. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18). 48484855. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Li Zuchao, Guan Chaoyu, Zhao Hai, Wang Rui, Parnow Kevin, and Zhang Zhuosheng. 2020. Memory network for linguistic structure parsing. IEEE/ACM Trans. Audio, Speech, Lang. Process. 28 (2020), 27432755.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Li Zuchao, He Shexia, Zhao Hai, Zhang Yiqing, Zhang Zhuosheng, Zhou Xi, and Zhou Xiang. 2019. Dependency or span, end-to-end uniform semantic role labeling. In Proceedings of the 33rd Conference of the Association for the Advancement of Artificial Intelligence (AAAI’19), Vol. 33. 67306737. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Lin Ziheng, Kan Min-Yen, and Ng Hwee Tou. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’09). 343351. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Liu Qi, Zhang Yue, and Liu Jiangming. 2018. Learning domain representation for multi-domain sentiment classification. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’18). 541550.Google ScholarGoogle ScholarCross RefCross Ref
  27. Liu Yang and Li Sujian. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 12241233.Google ScholarGoogle ScholarCross RefCross Ref
  28. Liu Yang, Li Sujian, Zhang Xiaodong, and Sui Zhifang. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Luan Yi, Ji Yangfeng, Hajishirzi Hannaneh, and Li Boyang. 2016. Multiplicative representations for unsupervised semantic role induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 118123.Google ScholarGoogle ScholarCross RefCross Ref
  30. McCann Bryan, Bradbury James, Xiong Caiming, and Socher Richard. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems. MIT Press, 62946305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Miller Alexander, Fisch Adam, Dodge Jesse, Karimi Amir-Hossein, Bordes Antoine, and Weston Jason. 2016. Key-Value memory networks for directly reading documents. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 14001409.Google ScholarGoogle ScholarCross RefCross Ref
  32. Mrkšić Nikola, Séaghdha Diarmuid Ó., Thomson Blaise, Gašić Milica, Rojas-Barahona Lina M., Su Pei-Hao, Vandyke David, Wen Tsung-Hsien, and Young Steve. 2016a. Counter-fitting word vectors to linguistic constraints. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’16). 142148.Google ScholarGoogle Scholar
  33. Mrkšić Nikola, Séaghdha Diarmuid O., Thomson Blaise, Gašić Milica, Rojas-Barahona Lina, Su Pei-Hao, Vandyke David, Wen Tsung-Hsien, and Young Steve. 2016b. Counter-fitting word vectors to linguistic constraints. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT’16).Google ScholarGoogle Scholar
  34. Munir Kashif, Zhao Hai, and Li Zuchao. 2021. Learning context-aware convolutional filters for implicit discourse relation classification. IEEE/ACM Trans. Audio, Speech, Lang. Process. 29 (2021), 24212433.Google ScholarGoogle ScholarCross RefCross Ref
  35. Nie Allen, Bennett Erin, and Goodman Noah. 2019. DisSent: Learning sentence representations from explicit discourse relations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL’19). 44974510.Google ScholarGoogle ScholarCross RefCross Ref
  36. Peters Matthew, Ammar Waleed, Bhagavatula Chandra, and Power Russell. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17). 17561765.Google ScholarGoogle ScholarCross RefCross Ref
  37. Peters Matthew, Neumann Mark, Iyyer Mohit, Gardner Matt, Clark Christopher, Lee Kenton, and Zettlemoyer Luke. 2018. Deep contextualized word representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’18). 22272237.Google ScholarGoogle ScholarCross RefCross Ref
  38. Pitler Emily, Louis Annie, and Nenkova Ani. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of he Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 683691. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Popa Diana Nicoleta, Perez Julien, Henderson James, and Gaussier Eric. 2019. Implicit discourse relation classification with syntax-aware contextualized word representations. In Proceedings of the 32nd International Flairs Conference.Google ScholarGoogle Scholar
  40. Prasad Rashmi, Dinesh Nikhil, Lee Alan, Miltsakaki Eleni, Robaldo Livio, Joshi Aravind K., and Webber Bonnie L.. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the 6th conference on International Language Resources and Evaluation (LREC’08). 29612968.Google ScholarGoogle Scholar
  41. Qin Lianhui, Zhang Zhisong, and Zhao Hai. 2016a. Implicit discourse relation recognition with context-aware character-enhanced embeddings. In Proceedings of the 26th International Conference on Computational Linguistics (COLING’16). 19141924.Google ScholarGoogle Scholar
  42. Qin Lianhui, Zhang Zhisong, and Zhao Hai. 2016b. Shallow discourse parsing using convolutional neural network. In Proceedings of the 20th Conference on Computational Natural Language Learning (CoNLL’16). 7077.Google ScholarGoogle ScholarCross RefCross Ref
  43. Qin Lianhui, Zhang Zhisong, and Zhao Hai. 2016c. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 22632270.Google ScholarGoogle ScholarCross RefCross Ref
  44. Qin Lianhui, Zhang Zhisong, Zhao Hai, Hu Zhiting, and Xing Eric. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17). 10061017.Google ScholarGoogle ScholarCross RefCross Ref
  45. Rönnqvist Samuel, Schenk Niko, and Chiarcos Christian. 2017. A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17). 256262.Google ScholarGoogle ScholarCross RefCross Ref
  46. Rutherford Attapol, Demberg Vera, and Xue Nianwen. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17). 281291.Google ScholarGoogle ScholarCross RefCross Ref
  47. Rutherford Attapol and Xue Nianwen. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:HLT’15). 799808.Google ScholarGoogle ScholarCross RefCross Ref
  48. Rutherford Attapol and Xue Nianwen. 2016. Robust non-explicit neural discourse parser in English and Chinese. In Proceedings of the 20th Conference on Computational Natural Language Learning (CoNLL’16). 5559.Google ScholarGoogle ScholarCross RefCross Ref
  49. Salant Shimi and Berant Jonathan. 2018. Contextualized word representations for reading comprehension. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 554559.Google ScholarGoogle ScholarCross RefCross Ref
  50. Schenk Niko, Chiarcos Christian, Donandt Kathrin, Rönnqvist Samuel, Stepanov Evgeny, and Riccardi Giuseppe. 2016. Do we really need all those rich linguistic features? A neural network-based approach to implicit sense labeling. In Proceedings of the 20th Conference on Computational Natural Language Learning (CoNLL’16). 4149.Google ScholarGoogle ScholarCross RefCross Ref
  51. Sennrich Rico, Haddow Barry, and Birch Alexandra. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 17151725.Google ScholarGoogle ScholarCross RefCross Ref
  52. Shi Wei and Demberg Vera. 2019. Learning to explicitate connectives with Seq2Seq network for implicit discourse relation classification. In Proceedings of the 13th International Conference on Computational Semantics. 188199.Google ScholarGoogle ScholarCross RefCross Ref
  53. Srivastava Rupesh Kumar, Greff Klaus, and Schmidhuber Jürgen. 2015. Training very deep networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, 23772385. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Sukhbaatar Sainbayar, Szlam Arthur, Weston Jason, and Fergus Rob. 2015. End-To-End memory networks. In Advances in Neural Information Processing Systems, vol. 28. MIT Press, 24402448. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Tu Lifu, Gimpel Kevin, and Livescu Karen. 2017. Learning to embed words in context for syntactic tasks. In Proceedings of the 2nd Workshop on Representation Learning for NLP. 26275.Google ScholarGoogle ScholarCross RefCross Ref
  56. Ngo Linh Van, Than Khoat, Nguyen Thien Huu, et al. 2019. Employing the correspondence of relations and connectives to identify implicit discourse relations via label embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL’19). 42014207.Google ScholarGoogle Scholar
  57. Weston Jason, Chopra Sumit, and Bordes Antoine. 2015. Memory networks. In Proceedings of the International Conference on Learning Representations (ICLR’15).Google ScholarGoogle Scholar
  58. Xiong Caiming, Merity Stephen, and Socher Richard. 2016. Dynamic memory networks for visual and textual question answering. In Proceedings of the International Conference on Machine Learning (ICML’16). 23972406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Xu Yang, Hong Yu, Ruan Huibin, Yao Jianmin, Zhang Min, and Zhou Guodong. 2018. Using active learning to expand training data for implicit discourse relation recognition. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18). 725731.Google ScholarGoogle ScholarCross RefCross Ref
  60. Xue Nianwen, Ng Hwee Tou, Pradhan Sameer, Prasad Rashmi, Bryant Christopher, and Rutherford Attapol. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the 19th Conference on Computational Natural Language Learning (CoNLL’15). 116.Google ScholarGoogle ScholarCross RefCross Ref
  61. Xue Nianwen, Ng Hwee Tou, Pradhan Sameer, Rutherford Attapol, Webber Bonnie, Wang Chuan, and Wang Hongmin. 2016. CoNLL 2016 shared task on multilingual shallow discourse parsing. In Proceedings of the 20th Conference on Computational Natural Language Learning (CoNLL’16). 119.Google ScholarGoogle ScholarCross RefCross Ref
  62. Zhang Biao, Su Jinsong, Xiong Deyi, Lu Yaojie, Duan Hong, and Yao Junfeng. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 22302235.Google ScholarGoogle ScholarCross RefCross Ref
  63. Zhang Yingxue, Jian Ping, Meng Fandong, Geng Ruiying, Cheng Wei, and Zhou Jie. 2019. Semantic graph convolutional network for implicit discourse relation classification. Retrieved from https://arxiv.org/abs/1910.09183.Google ScholarGoogle Scholar
  64. Zhou Yuping and Xue Nianwen. 2012. PDTB-style discourse annotation of Chinese text. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL’12). 6977. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Memorizing All for Implicit Discourse Relation Recognition

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 3
      May 2022
      413 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3505182
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 13 December 2021
      • Accepted: 1 September 2021
      • Revised: 1 August 2021
      • Received: 1 May 2021
      Published in tallip Volume 21, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!