skip to main content
research-article

Conducting Natural Language Inference with Word-Pair-Dependency and Local Context

Authors Info & Claims
Published:20 February 2020Publication History
Skip Abstract Section

Abstract

This article proposes to conduct natural language inference with novel Enhanced-Relation-Head-Dependent triplets (RHD triplets), which are constructed via enhancing each word in the RHD triplet with its associated local context. Most previous approaches based on deep neural network (DNN) for this task either perform token alignment without considering syntactic dependency among words, or directly use tree- LSTM to generate passage representation with irrelevant information. To improve token alignment and inference judgment with word-pair-dependency, the RHD triplet structure is first proposed. To avoid incorporating irrelevant information, this proposed approach performs comparison directly on each triplet-pair of the given passage-pair (instead of comparing each triplet in a passage with the content merged from the whole opposite passage). Furthermore, to take local context into consideration while conducting token alignment and inference judgment, we also enhance the words of the triplets with their associated local context to improve the performance. Experimental results show that the proposed approach is better than most previous approaches that adopt tree structures, and its performance is comparable to other state-of-the-art approaches (however, our approach is more human comprehensible).

References

  1. Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464 (2018).Google ScholarGoogle Scholar
  2. Jorge Balazs, Edison Marrese-Taylor, Pablo Loyola, and Yutaka Matsuo. 2017. Refining raw sentence representations for textual entailment recognition via attention. In The Workshop on Evaluating Vector Space Representations for NLP. 51--55.Google ScholarGoogle ScholarCross RefCross Ref
  3. Pinaki Bhaskar, Somnath Banerjee, Partha Pakray, Samadrita Banerjee, Sivaji Bandyopadhyay, and Alexander Gelbukh. 2013. A hybrid question answering system for multiple choice question (MCQ). In QA4MRE at Conference and Labs of the Evaluation Forum.Google ScholarGoogle Scholar
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. Computer Science (2015).Google ScholarGoogle Scholar
  5. Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. 2016. A fast unified model for parsing and sentence understanding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1466--1477. DOI:https://doi.org/10.18653/v1/P16-1139Google ScholarGoogle ScholarCross RefCross Ref
  6. Daniel Cer, Michel Galley, Jurafsky Dan, and Christopher D. Manning. 2009. Measuring machine translation quality as semantic equivalence: A metric based on entailment features. Machine Translation 23, 2/3 (2009), 181--193.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2018. Neural natural language inference models enhanced with external knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 2406--2417.Google ScholarGoogle ScholarCross RefCross Ref
  8. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1657--1668. DOI:https://doi.org/10.18653/v1/P17-1152Google ScholarGoogle ScholarCross RefCross Ref
  9. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Recurrent neural network-based sentence encoder with gated attention for natural language inference. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, 36--40. DOI:https://doi.org/10.18653/v1/W17-5307Google ScholarGoogle ScholarCross RefCross Ref
  10. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  11. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Proceedings of the International Conference on Machine Learning Challenges: Evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entailment. 177--190.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google ScholarGoogle Scholar
  13. Qianlong Du, Chengqing Zong, and Keh-Yih Su. 2018. Adopting the word-pair-dependency-triplets with individual comparison for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, August 20-26, 2018. 414--425.Google ScholarGoogle Scholar
  14. Dedre Gentner. 1983. Structure-mapping: A theoretical framework for analogy. Cognitive Science 7, 2 (1983), 155--170.Google ScholarGoogle Scholar
  15. Dedre Gentner and Arthur B. Markman. 1997. Structure mapping in analogy and similarity. American Psychologist 52, 1 (1997), 45.Google ScholarGoogle ScholarCross RefCross Ref
  16. Yichen Gong, Heng Luo, and Jian Zhang. 2018. Natural language inference over interaction space. In International Conference on Learning Representations.Google ScholarGoogle Scholar
  17. Sanda Harabagiu and Andrew Hickl. 2006. Methods for using textual entailment in open-domain question answering. In Proceedings of the International Conference on Computational Linguistics and Meeting of the Association for Computational Linguistics, ACL 2006, Sydney, Australia, 17-21 July. 905--912.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sanda Harabagiu, Andrew Hickl, and Finley Lacatusu. 2007. Satisfying information needs with multi-document summaries. Information Processing 8 Management 43, 6 (2007), 1619--1642.Google ScholarGoogle Scholar
  19. Jun Hatori, Yusuke Miyao, and Jun’ichi Tsujii. 2009. On contribution of sense dependencies to word sense disambiguation. Information and Media Technologies 4, 4 (2009), 1129--1155.Google ScholarGoogle Scholar
  20. Michael Heilman and Noah A. Smith. 2010. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 1011--1019.Google ScholarGoogle Scholar
  21. Thorsten Joachims. 1998. Making Large-scale SVM Learning Practical. Technical Report. Technical Report, SFB 475: Komplexitätsreduktion in Multivariaten Datenstrukturen, Universität Dortmund.Google ScholarGoogle Scholar
  22. Milen Kouylekov and Bernardo Magnini. 2005. Recognizing textual entailment with tree edit distance algorithms. In Proceedings of the 1st Challenge Workshop Recognising Textual Entailment. 17--20.Google ScholarGoogle Scholar
  23. Alice Lai and Julia Hockenmaier. 2014. Illinois-LH: A denotational and distributional approach to semantics. In International Workshop on Semantic Evaluation. 329--334.Google ScholarGoogle ScholarCross RefCross Ref
  24. Omer Levy, Torsten Zesch, Ido Dagan, and Iryna Gurevych. 2013. Recognizing partial textual entailment. In Meeting of the Association for Computational Linguistics. 451--455.Google ScholarGoogle Scholar
  25. Haoran Li, Junnan Zhu, Jiajun Zhang, and Chengqing Zong. 2018. Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 1430--1441. http://aclweb.org/anthology/C18-1121Google ScholarGoogle Scholar
  26. Pengfei Liu, Xipeng Qiu, Jifan Chen, and Xuanjing Huang. 2016. Deep fusion LSTMs for text semantic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1034--1043.Google ScholarGoogle ScholarCross RefCross Ref
  27. Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. 2016. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605.09090 (2016).Google ScholarGoogle Scholar
  28. Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  29. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 130--136. DOI:https://doi.org/10.18653/v1/P16-2022Google ScholarGoogle ScholarCross RefCross Ref
  30. Tsendsuren Munkhdalai and Hong Yu. 2017. Neural tree indexers for text understanding. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, 11--21.Google ScholarGoogle ScholarCross RefCross Ref
  31. Yixin Nie and Mohit Bansal. 2017. Shortcut-stacked sentence encoders for multi-domain inference. In The Workshop on Evaluating Vector Space Representations for NLP. 41--45.Google ScholarGoogle ScholarCross RefCross Ref
  32. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2249--2255. DOI:https://doi.org/10.18653/v1/D16-1244Google ScholarGoogle ScholarCross RefCross Ref
  33. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1532--1543.Google ScholarGoogle ScholarCross RefCross Ref
  34. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Retrieved from https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).Google ScholarGoogle Scholar
  35. Lorenza Romano, Milen Kouylekov, Idan Szpektor, Ido Dagan, and Alberto Lavelli. 2006. Investigating a generic paraphrase-based approach for relation extraction. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics.Google ScholarGoogle Scholar
  36. Nidhi Sharma, Richa Sharma, and Kanad K. Biswas. 2015. Recognizing textual entailment using dependency analysis and machine learning. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. 147--153.Google ScholarGoogle Scholar
  37. Hideki Shima, Hiroshi Kanayama, Cheng-Wei Lee, Chuan-Jie Lin, Teruko Mitamura, Yusuke Miyao, Shuming Shi, and Koichi Takeda. 2011. Overview of NTCIR-9 RITE: Recognizing inference in TExt. In NTCIR.Google ScholarGoogle Scholar
  38. Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. A compare-propagate architecture with alignment factorization for natural language inference. CoRR, abs/1801.00102 (2018).Google ScholarGoogle Scholar
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.Google ScholarGoogle Scholar
  40. Shuohang Wang and Jing Jiang. 2016. Learning natural language inference with LSTM. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 1442--1451. DOI:https://doi.org/10.18653/v1/N16-1170Google ScholarGoogle ScholarCross RefCross Ref
  41. Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral multi-perspective matching for natural language sentences. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI-17. 4144--4150. DOI:https://doi.org/10.24963/ijcai.2017/579Google ScholarGoogle ScholarCross RefCross Ref
  42. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 1112--1122. DOI:https://doi.org/10.18653/v1/N18-1101Google ScholarGoogle ScholarCross RefCross Ref
  43. Min Xiao and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In Proceedings of the 18th Conference on Computational Natural Language Learning. 119--129.Google ScholarGoogle ScholarCross RefCross Ref
  44. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).Google ScholarGoogle Scholar
  45. Kai Zhao, Liang Huang, and Mingbo Ma. 2016. Textual entailment with structured attentions and composition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, 2248--2258.Google ScholarGoogle Scholar

Index Terms

  1. Conducting Natural Language Inference with Word-Pair-Dependency and Local Context

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!