Abstract
Machine reading comprehension (MRC) requires machines to read and answer questions about a given text. This can be achieved through either predicting answers or extracting them. Extracting answers from text involves predicting the first and last index of the answer span within the paragraph. Training machines to answer questions requires datasets that are created for such a purpose. The lack of availability of benchmarking datasets for the Arabic language has hindered research into machine reading comprehension from Arabic text. The aim of this article is to propose an Arabic Span-Extraction-based Reading Comprehension Benchmark (ASER) and complement it with neural baseline models for performance evaluations. Detailed steps are depicted for building and evaluating ASER, which is an Arabic dataset created manually for the task of machine reading comprehension. It contains 10,000 records from different domains and is divided into training and testing sets. The results of ASER evaluation led to the conclusion that it is a challenging benchmark since the answers have varying lengths and human performance resulted in an exact match of 42%. On the other hand, two main baseline models were the focus of ASER experimentation: the sequence-to-sequence (Seq2Seq) model with different neural networks and the bidirectional attention flow (BIDAF) model. These experiments were implemented using different embeddings, and the results showed an exact match with lower values than human performance.
- [1] . 2019. Feature Selection and Enhanced Krill Herd Algorithm for Text Document Clustering. Springer.Google Scholar
Cross Ref
- [2] 2018. A combination of objective functions and hybrid krill herd algorithm for text document clustering analysis. Engineering Applications of Artificial Intelligence 73 (2018), 111–125.Google Scholar
Cross Ref
- [3] . 2021. What does BERT learn from Arabic machine reading comprehension datasets? In Proceedings of the 6th Arabic Natural Language Processing Workshop.Google Scholar
- [4] . 2020. AraBERT: Transformer-based model for arabic language understanding. In Proceedings of the LREC 2020 Workshop Language Resources and Evaluation Conference.Google Scholar
- [5] . 2021. Arabic question answering systems: Gap analysis. IEEE Access 9 (2021), 63876–63904.Google Scholar
Cross Ref
- [6] . 2017. Towards building a frame-based ontology for the arabic language. In Proceedings of the International Arab Conference on Information Technology.Google Scholar
- [7] . 2020. Towards building an open-domain corpus for arabic reading comprehension. In Proceedings of the 35th IBlMA Conference.Google Scholar
- [8] . 2018. Neural Reading Comprehension and Beyond. Stanford University.Google Scholar
Digital Library
- [9] . 2016. A thorough examination of the CNN/Daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2358–2367.Google Scholar
Cross Ref
- [10] . 2014. Learning phrase representations using RNN Encoder–Decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.Google Scholar
Cross Ref
- [11] . 2022. Interactive gated decoder for machine reading comprehension. Transactions on Asian and Low-resource Language Information Processing 21, 4 (2022), 1–19.Google Scholar
Digital Library
- [12] . 2021. English machine reading comprehension datasets: A survey. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 8784–8804.Google Scholar
Cross Ref
- [13] . 2019. Towards machine comprehension of arabic text. In Proceedings of the KDIR.Google Scholar
Cross Ref
- [14] . 2020. A comparison of LSTM and BERT for small corpus. arXiv:2009.05451. Retrieved from https://arxiv.org/abs/2009.05451Google Scholar
- [15] . 2018. Learning word vectors for 157 languages. In Proceedings of the 11th International Conference on Language Resources and Evaluation.Google Scholar
- [16] . 2021. I3rab: A new arabic dependency treebank based on arabic grammatical theory. Transactions on Asian and Low-resource Language Information Processing 21, 2 (2021), 1–32.Google Scholar
- [17] . 2015. Teaching machines to read and comprehend. Advances in Neural Information Processing Systems 28 (2015).Google Scholar
- [18] . 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.Google Scholar
Digital Library
- [19] . 2008. (Eds.). Recurrent Neural Networks, Vol. 400. I-Tech Education and Publishing KG.Google Scholar
Cross Ref
- [20] . 2022. Biomedical question answering: A survey of approaches and challenges. ACM Computing Surveys 55, 2 (2022), 1–36.Google Scholar
- [21] . 2016. Learning recurrent span representations for extractive question answering. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [22] . 2013. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems 26 (2013).Google Scholar
- [23] . 2019. Neural arabic question answering. In Proceedings of the 4rth Arabic Natural Language Processing Workshop. 108–118.Google Scholar
Cross Ref
- [24] . 2016. MS MARCO: A human generated machine reading comprehension dataset. CoCo@ NIPS 2640 (2016), 660.Google Scholar
- [25] . 2022. Improving machine reading comprehension with multi-task learning and self-training. Mathematics 10, 3, (2022), 310.Google Scholar
Cross Ref
- [26] . 2017. MEMEN: Multi-layer embedding with memory networks for machine comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, the 30th innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence.Google Scholar
- [27] . 2016. SQuAD: 100,000+ Questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2383–2392.Google Scholar
Cross Ref
- [28] . 2016. Bi-directional attention flow for machine comprehension. International Conference on Learning Representations.Google Scholar
- [29] . 2017. AraVec: A set of arabic word embedding models for use in arabic NLP. Procedia Computer Science 117 (2017), 256–265.Google Scholar
Cross Ref
- [30] . 2014. Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 27 (2014).Google Scholar
- [31] . 2016. NewsQA: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP. 191–200.Google Scholar
- [32] . 2019. Lost in translation: Loss and decay of linguistic richness in machine translation. In Proceedings of the Machine Translation Summit XVII. European Association for Machine Translation.Google Scholar
- [33] . 2017. Machine comprehension using match-lstm and answer pointer. In Proceedings of the ICLR 2017: International Conference on Learning Representations. Toulon. 1–15.Google Scholar
- [34] . 2017. R-Net: machine reading comprehension with self-matching networks. Natural Language Computer Group, Microsoft Reserach.Google Scholar
- [35] . 2017. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Google Scholar
Cross Ref
- [36] . 2016. Multi-Perspective context matching for machine comprehension. arXiv:1612.04211. Retrieved from https://arxiv.org/abs/1612.04211Google Scholar
- [37] . 2017. Making neural QA as simple as possible but not simpler. In Proceedings of the 21st Conference on Computational Natural Language Learning. 271–280.Google Scholar
Cross Ref
- [38] . 2017. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text. 94–106.Google Scholar
Cross Ref
- [39] . 2017. A constituent-centric neural architecture for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Google Scholar
Cross Ref
Index Terms
Arabic Span Extraction-based Reading Comprehension Benchmark (ASER) and Neural Baseline Models
Recommendations
Arabic Reading Machine for Visually Impaired People Using TTS and OCR
ISMS '13: Proceedings of the 2013 4th International Conference on Intelligent Systems, Modelling and SimulationThis paper suggests a standalone Arabic Reading Machine using TTS (Text-to-speech) and OCR (Optical Character Recognition) software built in a user friendly way for Visually Impaired People. In the Arab world, the assistive reading technology for ...
Nateq Reading Arabic Text for Visually Impaired People
Universal Access in Human-Computer Interaction. Methods, Technologies, and UsersAbstractNateq is a system developed to aid visually impaired people in their daily life tasks. Nateq allows blind users to read text written on papers and labels using their mobile phones. It uses two sources to read text from, either from camera or photo ...






Comments