Abstract
This article proposes to conduct natural language inference with novel Enhanced-Relation-Head-Dependent triplets (RHD triplets), which are constructed via enhancing each word in the RHD triplet with its associated local context. Most previous approaches based on deep neural network (DNN) for this task either perform token alignment without considering syntactic dependency among words, or directly use tree- LSTM to generate passage representation with irrelevant information. To improve token alignment and inference judgment with word-pair-dependency, the RHD triplet structure is first proposed. To avoid incorporating irrelevant information, this proposed approach performs comparison directly on each triplet-pair of the given passage-pair (instead of comparing each triplet in a passage with the content merged from the whole opposite passage). Furthermore, to take local context into consideration while conducting token alignment and inference judgment, we also enhance the words of the triplets with their associated local context to improve the performance. Experimental results show that the proposed approach is better than most previous approaches that adopt tree structures, and its performance is comparable to other state-of-the-art approaches (however, our approach is more human comprehensible).
- Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464 (2018).Google Scholar
- Jorge Balazs, Edison Marrese-Taylor, Pablo Loyola, and Yutaka Matsuo. 2017. Refining raw sentence representations for textual entailment recognition via attention. In The Workshop on Evaluating Vector Space Representations for NLP. 51--55.Google Scholar
Cross Ref
- Pinaki Bhaskar, Somnath Banerjee, Partha Pakray, Samadrita Banerjee, Sivaji Bandyopadhyay, and Alexander Gelbukh. 2013. A hybrid question answering system for multiple choice question (MCQ). In QA4MRE at Conference and Labs of the Evaluation Forum.Google Scholar
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. Computer Science (2015).Google Scholar
- Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. 2016. A fast unified model for parsing and sentence understanding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1466--1477. DOI:https://doi.org/10.18653/v1/P16-1139Google Scholar
Cross Ref
- Daniel Cer, Michel Galley, Jurafsky Dan, and Christopher D. Manning. 2009. Measuring machine translation quality as semantic equivalence: A metric based on entailment features. Machine Translation 23, 2/3 (2009), 181--193.Google Scholar
Digital Library
- Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2018. Neural natural language inference models enhanced with external knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 2406--2417.Google Scholar
Cross Ref
- Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1657--1668. DOI:https://doi.org/10.18653/v1/P17-1152Google Scholar
Cross Ref
- Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Recurrent neural network-based sentence encoder with gated attention for natural language inference. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, 36--40. DOI:https://doi.org/10.18653/v1/W17-5307Google Scholar
Cross Ref
- Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.Google Scholar
Cross Ref
- Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Proceedings of the International Conference on Machine Learning Challenges: Evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entailment. 177--190.Google Scholar
Digital Library
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google Scholar
- Qianlong Du, Chengqing Zong, and Keh-Yih Su. 2018. Adopting the word-pair-dependency-triplets with individual comparison for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, August 20-26, 2018. 414--425.Google Scholar
- Dedre Gentner. 1983. Structure-mapping: A theoretical framework for analogy. Cognitive Science 7, 2 (1983), 155--170.Google Scholar
- Dedre Gentner and Arthur B. Markman. 1997. Structure mapping in analogy and similarity. American Psychologist 52, 1 (1997), 45.Google Scholar
Cross Ref
- Yichen Gong, Heng Luo, and Jian Zhang. 2018. Natural language inference over interaction space. In International Conference on Learning Representations.Google Scholar
- Sanda Harabagiu and Andrew Hickl. 2006. Methods for using textual entailment in open-domain question answering. In Proceedings of the International Conference on Computational Linguistics and Meeting of the Association for Computational Linguistics, ACL 2006, Sydney, Australia, 17-21 July. 905--912.Google Scholar
Digital Library
- Sanda Harabagiu, Andrew Hickl, and Finley Lacatusu. 2007. Satisfying information needs with multi-document summaries. Information Processing 8 Management 43, 6 (2007), 1619--1642.Google Scholar
- Jun Hatori, Yusuke Miyao, and Jun’ichi Tsujii. 2009. On contribution of sense dependencies to word sense disambiguation. Information and Media Technologies 4, 4 (2009), 1129--1155.Google Scholar
- Michael Heilman and Noah A. Smith. 2010. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 1011--1019.Google Scholar
- Thorsten Joachims. 1998. Making Large-scale SVM Learning Practical. Technical Report. Technical Report, SFB 475: Komplexitätsreduktion in Multivariaten Datenstrukturen, Universität Dortmund.Google Scholar
- Milen Kouylekov and Bernardo Magnini. 2005. Recognizing textual entailment with tree edit distance algorithms. In Proceedings of the 1st Challenge Workshop Recognising Textual Entailment. 17--20.Google Scholar
- Alice Lai and Julia Hockenmaier. 2014. Illinois-LH: A denotational and distributional approach to semantics. In International Workshop on Semantic Evaluation. 329--334.Google Scholar
Cross Ref
- Omer Levy, Torsten Zesch, Ido Dagan, and Iryna Gurevych. 2013. Recognizing partial textual entailment. In Meeting of the Association for Computational Linguistics. 451--455.Google Scholar
- Haoran Li, Junnan Zhu, Jiajun Zhang, and Chengqing Zong. 2018. Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 1430--1441. http://aclweb.org/anthology/C18-1121Google Scholar
- Pengfei Liu, Xipeng Qiu, Jifan Chen, and Xuanjing Huang. 2016. Deep fusion LSTMs for text semantic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1034--1043.Google Scholar
Cross Ref
- Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. 2016. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605.09090 (2016).Google Scholar
- Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). 1--8.Google Scholar
Cross Ref
- Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 130--136. DOI:https://doi.org/10.18653/v1/P16-2022Google Scholar
Cross Ref
- Tsendsuren Munkhdalai and Hong Yu. 2017. Neural tree indexers for text understanding. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, 11--21.Google Scholar
Cross Ref
- Yixin Nie and Mohit Bansal. 2017. Shortcut-stacked sentence encoders for multi-domain inference. In The Workshop on Evaluating Vector Space Representations for NLP. 41--45.Google Scholar
Cross Ref
- Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2249--2255. DOI:https://doi.org/10.18653/v1/D16-1244Google Scholar
Cross Ref
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1532--1543.Google Scholar
Cross Ref
- Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Retrieved from https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).Google Scholar
- Lorenza Romano, Milen Kouylekov, Idan Szpektor, Ido Dagan, and Alberto Lavelli. 2006. Investigating a generic paraphrase-based approach for relation extraction. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics.Google Scholar
- Nidhi Sharma, Richa Sharma, and Kanad K. Biswas. 2015. Recognizing textual entailment using dependency analysis and machine learning. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. 147--153.Google Scholar
- Hideki Shima, Hiroshi Kanayama, Cheng-Wei Lee, Chuan-Jie Lin, Teruko Mitamura, Yusuke Miyao, Shuming Shi, and Koichi Takeda. 2011. Overview of NTCIR-9 RITE: Recognizing inference in TExt. In NTCIR.Google Scholar
- Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. A compare-propagate architecture with alignment factorization for natural language inference. CoRR, abs/1801.00102 (2018).Google Scholar
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.Google Scholar
- Shuohang Wang and Jing Jiang. 2016. Learning natural language inference with LSTM. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 1442--1451. DOI:https://doi.org/10.18653/v1/N16-1170Google Scholar
Cross Ref
- Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral multi-perspective matching for natural language sentences. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI-17. 4144--4150. DOI:https://doi.org/10.24963/ijcai.2017/579Google Scholar
Cross Ref
- Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 1112--1122. DOI:https://doi.org/10.18653/v1/N18-1101Google Scholar
Cross Ref
- Min Xiao and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In Proceedings of the 18th Conference on Computational Natural Language Learning. 119--129.Google Scholar
Cross Ref
- Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).Google Scholar
- Kai Zhao, Liang Huang, and Mingbo Ma. 2016. Textual entailment with structured attentions and composition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, 2248--2258.Google Scholar
Index Terms
Conducting Natural Language Inference with Word-Pair-Dependency and Local Context
Recommendations
Building a Vietnamese Dataset for Natural Language Inference Models
AbstractNatural language inference models are essential resources for many natural language understanding applications. These models are possibly built by training or fine-tuning using deep neural network architectures for state-of-the-art results. That ...
Natural Language Inference for Portuguese Using BERT and Multilingual Information
Computational Processing of the Portuguese LanguageAbstractRecognizing Textual Entailment, also known as inference recognition, aims to identify when the meaning of a piece of text contains the meaning of another fragment of text. In this work, we investigate multiples approaches for recognizing inference ...
Word n-gram attention models for sentence similarity and inference
Highlights- Bag-of-Words models boost their performance when making use of context.
- Context ...
AbstractSemantic Textual Similarity and Natural Language Inference are two popular natural language understanding tasks used to benchmark sentence representation models where two sentences are paired. In such tasks sentences are represented as ...






Comments