Abstract
Machine reading comprehension (MRC) is a crucial and challenging task in natural language processing (NLP). In order to equip machines with logical reasoning abilities, the challenging logical reasoning tasks are proposed. Existing approaches use graph-based neural models based on either sentence-level or entity-level graph construction methods which designed to capture a logical structure and enable inference over it. However, sentence-level methods result in a loss of fine-grained information and difficulty in capturing implicit relationships, while entity-level methods fail to capture the overall logical structure of the text. To address these issues, we propose a multi-grained graph-based mechanism for solving logical reasoning MRC. To combine the advantages of sentence-level and entity-level information, we mine elementary discourse units (EDUs) and entities from texts to construct graph, and learn the logical-aware features through a graph network for subsequent answer prediction. Furthermore, we implement a positional embedding mechanism to enforce the positional dependence, which facilitates logical reasoning. Our experimental results demonstrate that our approach provides significant and consistent improvements via multi-grained graphs, outperforming competitive baselines on both ReClor and LogiQA benchmarks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cai, L., Li, J., Wang, J., Ji, S.: Line Graph Neural Networks for Link Prediction (2020)
Cetto, M., Niklaus, C., Freitas, A., Handschuh, S.: Graphene: Semantically-Linked Propositions in Open Information Extraction (2018)
Chen, K., et al.: Question Directed Graph Attention Network for Numerical Reasoning over Text (2020)
Cho, K., et al.: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (2014)
Chowdhury, G.G.: Introduction to Modern Information Retrieval. Facet Publishing (2010)
Christopoulou, F., Miwa, M., Ananiadou, S.: Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs (2019)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 (2019)
Dua, D., et al.: DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019)
Fan, F., Feng, Y., Zhao, D.: Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, pp. 3433–3442. Association for Computational Linguistics (2018)
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, vol. 28. Curran Associates Inc. (2015)
Hirschman, L., Light, M., Breck, E., Burger, J.D.: Deep Read: a reading comprehension system. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. College Park, Maryland, USA, pp. 325–332. Association for Computational Linguistics (1999)
"History of programming languages–II | ACM Other Books. https://dl.acm.org/doi/abs/10.1145/234286
Huang, L., Bras, R.L., Bhagavatula, C., Choi, Y.: Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning (2019)
Huang, Y., Fang, M., Cao, Y., Wang, L., Liang, X.: DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021)
Jia, R., Cao, Y., Tang, H., Fang, F., Cao, C., Wang, S.: Neural extractive summarization with hierarchical attentive heterogeneous graph network. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3622–3631. Online: Association for Computational Linguistics (2020)
Jiao, F., Guo, Y., Song, X., Nie, L.: MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning (2022)
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., Roth, D.: Looking beyond the surface: a challenge set for reading comprehension over multiple sentences. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 252–262. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Li, X., Cheng, G., Chen, Z., Sun, Y., Qu, Y.: AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension (2022)
Liu, S., Zhang, X., Zhang, S., Wang, H., Zhang, W.: Neural machine reading comprehension: methods and trends. Appl. Surf. Sci. 471, 18–22 (2019)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach (2019)
Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., Zhang, Y.: LogiQA: a challenge dataset for machine reading comprehension with logical reasoning (2020)
McCarthy, J.: Artificial intelligence, logic and formalizing common sense. In: Thomason, R.H. (ed.) Philosophical Logic and Artificial Intelligence, pp. 161–190. Springer, Netherlands, Dordrecht (1989). https://doi.org/10.1007/978-94-009-2448-2_6
Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.: Can a suit of armor conduct electricity? A new dataset for open book question answering (2018)
Nguyen, T. et al.: MS MARCO: A Human Generated MAchine Reading COmprehension Dataset (2016)
Ouyang, S., Zhang, Z., Zhao, H.: Fact-driven Logical Reasoning (2021)
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text (2016)
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD (2018)
Ran, Q., Lin, Y., Li, P., Zhou, J., Liu, Z.: NumNet: machine reading comprehension with numerical reasoning (2019)
Riloff, E., Thelen, M.: A rule-based question answering system for reading comprehension tests. In: ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-Based Language Understanding Systems (2000)
Sahu, S.K., Christopoulou, F., Miwa, M., Ananiadou, S.: Inter-sentence relation extraction with document-level graph convolutional neural network (2019)
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009)
Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional Attention Flow for Machine Comprehension (2018)
Sugawara, S., Aizawa, A.: An analysis of prerequisite skills for reading comprehension. In: Proceedings of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, pp. 1–5. Association for Computational Linguistics, Austin, TX (2016)
Tandon, N., Mishra, B.D., Sakaguchi, K., Bosselut, A., Clark, P.: WIQA: a dataset for “What if..." reasoning over procedural text (2019)
Wang, S., Jiang, J.: Machine Comprehension Using Match-LSTM and Answer Pointer (2016)
Wang, S., et al.: Logic-driven context extension and data augmentation for logical reasoning of text (2021)
Wang, S., et al.: From LSAT: the progress and challenges of complex reasoning (2021)
Welbl, J., Stenetorp, P., Riedel, S.: Constructing datasets for multi-hop reading comprehension across documents. Trans. Assoc. Comput. Linguist. 6, 287–302 (2018)
Xu, K., Wu, L., Wang, Z., Yu, M., Chen, L., Sheinin, V.: Exploiting rich syntactic information for semantic parsing with graph-to-sequence model (2018)
Xu, M., Li, L., Wong, D.F., Liu, Q., Chao, L.S.: Document graph for neural machine translation (2021)
Xu, F., Liu, J., Lin, Q., Pan, Y., Zhang, L.: Logiformer: a two-branch graph transformer network for interpretable logical reasoning. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1055–1065 (2022)
Yang, Z., et al.: HotpotQA: a dataset for diverse, explainable multi-hop question answering (2018)
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. Proc. AAAI Conf. Artif. Intell. 33(01), 7370–7377 (2019)
Ying, C.T., et al.: Transformers really perform bad for graph representation?"
Yu, A.W., et al.: QANet: combining local convolution with global self-attention for reading comprehension (2018)
Yu, W., Jiang, Z., Dong, Y., Feng, J.: ReClor: a reading comprehension dataset requiring logical reasoning (2020)
Zeng, S., Xu, R., Chang, B., Li, L.: Double graph based reasoning for document-level relation extraction (2020)
Zhang, Z., Zhao, H., Wang, R.: Machine reading comprehension: the role of contextualized language models and beyond. arXiv:2005.06249 (2020)
Zheng, B., et al.: Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020)
Zhou, B., Khashabi, D., Ning, Q., Roth, D.: Going on a vacation takes longer than “Going for a walk”: a study of temporal commonsense understanding (2019)
Acknowledgements
The authors would like to thank the Associate Editor and anonymous reviewers for their valuable comments and suggestions. This work is funded in part by the National Natural Science Foundation of China under Grants No.62176029, and in part by the graduate research and innovation foundation of Chongqing, China under Grants No.CYB21063. This work also is supported in part by the Chongqing Technology Innovation and Application Development Special under Grants CSTB2022TIAD-KPX0206.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, J. et al. (2023). Multi-grained Logical Graph Network for Reasoning-Based Machine Reading Comprehension. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14179. Springer, Cham. https://doi.org/10.1007/978-3-031-46674-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-46674-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46673-1
Online ISBN: 978-3-031-46674-8
eBook Packages: Computer ScienceComputer Science (R0)