Abstract
Most existing multi-document machine reading comprehension models mainly focus on understanding the interactions between the input question and documents, but ignore the following two kinds of understandings. First, to understand the semantic meaning of words in the input question and documents from the perspective of each other. Second, to understand the supporting cues for a correct answer from the perspective of intra-document and inter-documents. Ignoring these two kinds of important understandings would make the models overlook some important information that may be helpful for finding correct answers. To overcome this deficiency, we propose a deep understanding based model for multi-document machine reading comprehension. It has three cascaded deep understanding modules which are designed to understand the accurate semantic meaning of words, the interactions between the input question and documents, and the supporting cues for the correct answer. We evaluate our model on two large scale benchmark datasets, namely TriviaQA Web and DuReader. Extensive experiments show that our model achieves state-of-the-art results on both datasets.
- . 2015. Neural machine translation by jointly learning to align and translate. In ICLR 2015: International Conference on Learning Representations 2015.Google Scholar
- . 2021. Self-supervised test-time learning for reading comprehension.arXiv preprint arXiv:2103.11263 (2021).Google Scholar
- . 2020. Multi-choice relational reasoning for machine reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics. 6448–6458.Google Scholar
Cross Ref
- . 2020. ForceReader: A BERT-based interactive machine reading comprehension model with attention separation. In Proceedings of the 28th International Conference on Computational Linguistics. 2676–2686.Google Scholar
Cross Ref
- . 2017. Smarnet: Teaching machines to read and comprehend like human.arXiv preprint arXiv:1710.02772 (2017).Google Scholar
- . 2018. Simple and effective multi-paragraph reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 845–855.Google Scholar
Cross Ref
- . 2017. Attention-over-attention neural networks for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 593–602.Google Scholar
Cross Ref
- . 2020. Multi-paragraph reading comprehension with token-level dynamic reader and hybrid verifier. In 2020 International Joint Conference on Neural Networks (IJCNN). 1–8.Google Scholar
Cross Ref
- . 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.Google Scholar
- . 2020a. Hierarchical LSTM with char-subword-word tree-structure representation for Chinese named entity recognition. Science in China Series F: Information Sciences 63, 10 (2020), 202102.Google Scholar
Cross Ref
- . 2020b. Recurrent chunking mechanisms for long-text machine reading comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6751–6761.Google Scholar
Cross Ref
- . 2020a. Incorporating syntax and frame semantics in neural network for machine reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics. 2635–2641.Google Scholar
Cross Ref
- . 2020b. A frame-based sentence representation for machine reading comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 891–896.Google Scholar
Cross Ref
- . 2018. DuReader: A Chinese machine reading comprehension dataset from real-world applications. In Proceedings of the Workshop on Machine Reading for Question Answering. 37–46.Google Scholar
Cross Ref
- . 2019. Read + verify: Machine reading comprehension with unanswerable questions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6529–6537.Google Scholar
Digital Library
- . 2020. NUT-RC: Noisy user-generated text-oriented reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics. 2687–2698.Google Scholar
Cross Ref
- . 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1601–1611.Google Scholar
Cross Ref
- . 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In ICLR 2020: Eighth International Conference on Learning Representations.Google Scholar
- . 2020b. Towards medical machine reading comprehension with structural knowledge and plain text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1427–1438.Google Scholar
Cross Ref
- . 2020a. MRC examples answerable by BERT without a question are less effective in MRC model training. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: Student Research Workshop. 146–152.Google Scholar
- . 2018. Stochastic answer networks for machine reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1694–1704.Google Scholar
Cross Ref
- . 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019).Google Scholar
- . 2020. Synonym knowledge enhanced reader for Chinese idiom reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics. 3684–3695.Google Scholar
Cross Ref
- . 2020. MaP: A matrix-based prediction approach to improve span extraction in machine reading comprehension. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing. 687–695.Google Scholar
- . 2020. Bridging information-seeking human gaze and machine reading comprehension. In Proceedings of the 24th Conference on Computational Natural Language Learning. 142–152.Google Scholar
Cross Ref
- . 2019. Discourse-aware semantic self-attention for narrative reading comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2541–2552.Google Scholar
Cross Ref
- . 2016. MS MARCO: A human generated MAchine reading COmprehension dataset. In [email protected].Google Scholar
- . 2019. Multi-style generative reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2273–2284.Google Scholar
Cross Ref
- . 2020. Bi-directional cognitive thinking network for machine reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics.Google Scholar
Cross Ref
- . 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1532–1543.Google Scholar
Cross Ref
- . 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Vol. 1. 2227–2237.Google Scholar
Cross Ref
- . 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vol. 2. 784–789.Google Scholar
Cross Ref
- . 2016. Bidirectional attention flow for machine comprehension. In ICLR (Poster).Google Scholar
- . 2019. Improving machine reading comprehension with general reading strategies. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2633–2643.Google Scholar
Cross Ref
- . 2020. Scene restoring for narrative machine reading comprehension. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 3063–3073.Google Scholar
Cross Ref
- . 2015. Pointer networks. In NIPS’15 Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, Vol. 28. 2692–2700.Google Scholar
- . 2017. Machine comprehension using match-LSTM and answer pointer. In ICLR (Poster). 1.Google Scholar
- . 2018c. Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1705–1714.Google Scholar
Cross Ref
- . 2017. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 189–198.Google Scholar
Cross Ref
- . 2018a. Multi-passage machine reading comprehension with cross-passage answer verification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1918–1927.Google Scholar
Cross Ref
- . 2020. TPLinker: Single-stage joint extraction of entities and relations through token pair linking. In Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain (Online), 1572–1582.Google Scholar
Cross Ref
- . 2018b. Joint training of candidate extraction and answer selection for reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1715–1724.Google Scholar
Cross Ref
- . 2018. Fast Reading Comprehension With ConvNets. arXiv preprint arXiv:1711.04352 (2018).Google Scholar
- . 2019. Multi-task learning with sample re-weighting for machine reading comprehension. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2644–2655.Google Scholar
Cross Ref
- . 2019. A deep cascade model for multi-document reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7354–7361.Google Scholar
Digital Library
- . 2019b. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2346–2357.Google Scholar
Cross Ref
- . 2019c. End-to-end open-domain question answering with BERTserini. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations). 72–77.Google Scholar
- . 2019a. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, Vol. 32. 5753–5763.Google Scholar
- . 2018. QANet: Combining local convolution with global self-attention for reading comprehension. In International Conference on Learning Representations.Google Scholar
- . 2021. ReadTwice: Reading very large documents with memories. arXiv preprint arXiv:2105.04241 (2021).Google Scholar
- . 2020c. Learn with noisy data via unsupervised loss correction for weakly supervised reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics. 2624–2634.Google Scholar
Cross Ref
- . 2020a. Semantics-aware BERT for language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9628–9635.Google Scholar
Cross Ref
- . 2020b. SG-Net: Syntax-guided machine reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9636–9643.Google Scholar
Cross Ref
- . 2021. Retrospective reader for machine reading comprehension. In AAAI 2021.Google Scholar
- . 2020. Document modeling with graph attention networks for multi-grained machine reading comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6708–6718.Google Scholar
Cross Ref
Index Terms
Deep Understanding Based Multi-Document Machine Reading Comprehension
Recommendations
An Understanding-oriented Robust Machine Reading Comprehension Model
Although existing machine reading comprehension models are making rapid progress on many datasets, they are far from robust. In this article, we propose an understanding-oriented machine reading comprehension model to address three kinds of robustness ...
Multi-passage extraction-based machine reading comprehension based on verification sorting
Highlights- This paper proposes a sequencing-based multi-passage reading comprehension model. Compared with the traditional machine reading model, the innovation of this ...
AbstractFor traditional single-passage machine reading comprehension, the text data of a single passage does not well reflect the complexity of practical application scenarios. Many researchers have shifted their research goals to study multi-...
Graphical abstractDisplay Omitted
Sentence Extraction-Based Machine Reading Comprehension for Vietnamese
Knowledge Science, Engineering and ManagementAbstractThe development of natural language processing (NLP) in general and machine reading comprehension in particular has attracted the great attention of the research community. In recent years, there are a few datasets for machine reading ...






Comments