Abstract
Question answering, serving as one of important tasks in natural language processing, enables machines to understand questions in natural language and answer the questions concisely. From web search to expert systems, question answering systems are widely applied to various domains in assisting information seeking. Deep learning methods have boosted various tasks of question answering and have demonstrated dramatic effects in performance improvement for essential steps of question answering. Thus, leveraging deep learning methods for question answering has drawn much attention from both academia and industry in recent years. This paper provides a systematic review of the recent development of deep learning methods for question answering. The survey covers the scope including methods, datasets, and applications. The methods are discussed in terms of network structure characteristics, methodology innovations, and their effectiveness. The survey is expected to be a contribution to the summarization of recent research progress and future directions of deep learning methods for question answering.

Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adlouni YE, Rodríguez H, Meknassi M, El Alaoui SO, En-nahnahi N (2019) A multi-approach to community question answering. Expert Sys Appl 137:432–442
Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, Hasan M, van Essen BC, Awwal AAS, Asari VK (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292
Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: ICLR
Banerjee S, Naskar S, Rosso P, Bandyopadhyay S (2018) Code mixed cross script factoid question classification - a deep learning approach. J Intell & Fuzzy Sys 34(5):2959–2969
Bast H, Haussmann E (2015) More accurate question answering on freebase. In: CIKM’15, pp 1431–1440
Ben Abacha A, Demner-Fushman D (2019) A question-entailment approach to question answering. BMC Bioinfo 20(1):e33
Bengio Y (2009) Learning deep architectures for AI. Found Trends in Machine Learn 2(1):1–127
Berant J, Chou A, Roy F, Liang P (2013) Semantic parsing on freebase from question-answer pairs. In: EMNLP, pp 1533–1544
Bi M, Zhang Q, Zuo M, Xu Z, Jin Q (2019) Bi-directional lstm model with symptoms-frequency position attention for question answering system in medical domain. Neural Process Lett 51(5):570
Bisk Y, Reddy S, Blitzer J, Hockenmaier J, Steedman M (2016) Evaluating induced ccg parsers on grounded semantic parsing. In: EMNLP, pp 2022–2027
Cai L, Zhou S, Yan X (2019) Yuan R (2019) A stacked bilstm neural network based on coattention mechanism for question answering. Computat Intell Neurosci 9:1–12
Cai LQ, Wei M, Zhou ST, Yan X (2020) Intelligent question answering in restricted domains using deep learning and question pair matching. IEEE Access 8:32922–32934
Chen Z, Zhang C, Zhao Z, Yao C, Cai D (2018) Question retrieval for community-based question answering via heterogeneous social influential network. Neurocomputing 285:117–124
Chen ZY, Chang CH, Chen YP, Nayak J, Ku LW (2019) Uhop: An unrestricted-hop relation extraction framework for knowledge-based question answering. In: NAACL-HLT, pp 345–356
Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. In: EMNLP, pp 1724–1734
Cortes E, Woloszyn V, Binder A, Himmelsbach T, Barone D, Möller S (2020) An empirical comparison of question classification methods for question answering systems. In: LREC, pp 5408–5416
Croce D, Filice S, Basili R (2019) Making sense of kernel spaces in neural learning. Computer Speech & Language 58:51–75
Dargan S, Kumar M, Ayyagari MR, Kumar G (2019) A survey of deep learning and its applications: A new paradigm to machine learning. Archi Computat Method Eng 85(4):114
Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp 4171–4186
Dimitrakis E, Sgontzos K, Tzitzikas Y (2019) A survey on question answering systems over linked data and documents. J Intell Info Sys 51(5):570
Dong L, Mallinson J, Reddy S, Lapata M (2017) Learning to paraphrase for question answering. In: EMNLP, pp 875–886
Du X, Shao J, Cardie C (2017) Learning to ask: Neural question generation for reading comprehension. In: ACL, pp 1342–1352
Dubey M, Banerjee D, Abdelkawi A, Lehmann J (2019) Lc-quad 2.0: A large dataset for complex question answering over wikidata and dbpedia. SEMWEB 11779:69–78
Elman JL (1990) Finding structure in time. Cognitive Sci 14(2):179–211
Elsahar H, Gravier C, Laforest F (2018) Zero-shot question generation from knowledge graphs for unseen predicates and entity types. In: NAACL-HLT, pp 218–228
Fukushima K (1988) Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Networks 1(2):119–130
Garg S, Vu T, Moschitti A (2020) Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. AAAI 34:7780–7788
Goldberg Y (2016) A primer on neural network models for natural language processing. J Artif Intell Res 57(1):345–420
Green BF, Wolf AK, Chomsky C, Laughery K (1961) Baseball: an automatic question-answerer. In: IRE-AIEE-ACM ’61 (Western), pp 219–224
Gulcehre C, Ahn S, Nallapati R, Zhou B, Bengio Y (2016) Pointing the unknown words. In: ACL, pp 140–149
Hao Z, Wu B, Wen W, Cai R (2019) A subgraph-representation-based method for answering complex questions over knowledge bases. Neural Networks 119:57–65
He J, Fu M, Tu M (2019) Applying deep matching networks to chinese medical question answering: a study and a dataset. BMC Med Info Decision Making 19(S2):1
Hirschman L, Gaizauskas R (2001) Natural language question answering: the view from here. Nat Lang Eng 7(4):275–300
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Computat 9(8):1735–1780
Huang H, Wei X, Nie L, Mao X, Xu XS (2019) From question to text: Question-oriented feature attention for answer selection. ACM Trans Info Sys 37(1):1–33
Huang W, Qu Q, Yang M (2020) Interactive knowledge-enhanced attention network for answer selection. Neural Comput Appl 32(15):11343–11359
Indurthi SR, Raghu D, Khapra MM, Joshi S (2017) Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In: EACL, pp 376–385
Jiang B, Tan L, Ren Y, Li F (2019) Intelligent interaction with virtual geographical environments based on geographic knowledge graph. ISPRS Int J Geo-Info 8(10):428
Jing L, Gulcehre C, Peurifoy J, Shen Y, Tegmark M, Soljacic M, Bengio Y (2019) Gated orthogonal recurrent units: on learning to forget. Neural Computat 31(4):765–783
Khalifa M, Shaalan K (2019) Character convolutions for arabic named entity recognition with long short-term memory networks. Comp Speech & Language 58:335–346
Kim S, Park D, Choi Y, Lee K, Kim B, Jeon M, Kim J, Tan AC, Kang J (2018) A pilot study of biomedical text comprehension using an attention-based deep neural reader: design and experimental analysis. JMIR Med Info 6(1):e2
Kim Y, Lee H, Shin J, Jung K (2019) Improving neural question generation using answer separation. AAAI 33:6602–6609
Kolomiyets O, Moens MF (2011) A survey on question answering technology from an information retrieval perspective. Info Sci 181(24):5412–5434
Kumar A, Irsoy O, Ondruska P, Iyyer M, Bradbury J, Gulrajani I, Zhong V, Paulus R, Socher R (2016) Ask me anything: Dynamic memory networks for natural language processing. In: ICML, pp 1378–1387
Kumar V, Hua Y, Ramakrishnan G, Qi G, Gao L, Li YF (2019) Difficulty-controllable multi-hop question generation from knowledge graphs. SEMWEB 11778:382–398
Lan Y, Jiang J (2020) Query graph generation for answering multi-hop complex questions from knowledge bases. In: ACL, pp 969–974
Lan Y, Wang S, Jiang J (2019) Knowledge base question answering with a matching-aggregation model and question-specific contextual relations. IEEE/ACM Trans Audio, Speech, and Language Process 27(10):1629–1638
Lan Y, Wang S, Jiang J (2019) Multi-hop knowledge base question answering with an iterative sequence matching model. In: ICDM, pp 359–368
Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2020) Albert: A lite bert for self-supervised learning of language representations. In: ICLR
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceed of the IEEE 86:2278–2324
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Lee CH, Lee HY, Wu SL, Liu CL, Fang W, Hsu JY, Tseng BH (2019) Machine comprehension of spoken content: Toefl listening test and spoken squad. IEEE/ACM Trans on Audio, Speech, and Language Process 27(9):1469–1480
Li J, Sun A, Han J, Li C (2022) A survey on deep learning for named entity recognition. IEEE Transact Knowledge & Data Eng 34:50–70
Li X, Zhang S, Wang B, Gao Z, Fang L, Xu H (2019) A hybrid framework for problem solving of comparative questions. IEEE Access 7:185961–185976
Lin T, Goyal P, Girshick R, He K, Dollár P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Machine Intell 42(2):318–327
Liu D, Niu Z, Zhang C, Zhang J (2019) Multi-scale deformable cnn for answer selection. IEEE Access 7:164986–164995
Liu H, Liu Y, Wong LP, Lee LK, Hao T (2020) A hybrid neural network bert-cap based on pre-trained language model and capsule network for user intent classification. Complexity 2020:1–11
Luo K, Lin F, Luo X, Zhu K (2018) Knowledge base question answering via encoding of complex query graphs. In: EMNLP, pp 2185–2194
Mahmoud A, Zrigui M (2019) Sentence embedding and convolutional neural network for semantic textual similarity detection in arabic language. Arab J Sci Eng 44(11):9263–9274
Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao J (2021) Deep learning-based text classification: A comprehensive review. ACM Comput Surv 54(3):62:1–62:40
Ojokoh B, Adebisi E (2019) A review of question answering systems. J Web Eng 17(8):717–758
Otter DW, Medina JR, Kalita JK (2021) A survey of the usages of deep learning in natural language processing. IEEE Trans Neural Network Learn Sys 32:604–624
Pan L, Lei W, Chua TS, Kan MY (2019) Recent advances in neural question generation. ArXiv abs/1905.08949
Parshakova T, Rameau F, Serdega A, Kweon IS, Kim DS (2019) Latent question interpretation through variational adaptation. IEEE/ACM Trans Audio, Speech and Language Process 27(11):1713–1724
Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: NAACL-HLT, pp 2227–2237
Qu Y, Liu J, Kang L, Shi Q, Ye D (2018) Question answering over freebase via attentive rnn with similarity matrix based cnn. arXiv: abs/1804.03317
Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. In: EMNLP, pp 2383–2392
Ren Q, Cheng X, Su S (2020) Multi-task learning with generative adversarial training for multi-passage machine reading comprehension. AAAI 34:8705–8712
Roy PK, Singh JP (2019) Predicting closed questions on community question answering sites using convolutional neural network. Neural Comput Appl 19(5):53
Sanh V, Debut L, Chaumond J, Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv: abs/1910.01108
Sawant U, Garg S, Chakrabarti S, Ramakrishnan G (2019) Neural architecture for question answering using a knowledge graph and web corpus. Info Retr J 22(3–4):324–349
Shah AA, Ravana SD, Hamid S, Ismail MA (2018) Accuracy evaluation of methods and techniques in web-based question answering systems: a survey. Knowl Info Sys 58(03):611–650
Shao T, Guo Y, Chen H, Hao Z (2019) Transformer-based neural network for answer selection in question answering. IEEE Access 7:26146–26156
Shao T, Kui X, Zhang P, Chen H (2019) Collaborative learning for answer selection in question answering. IEEE Access 7:7337–7347
Shuang K, Liu Y, Zhang W, Zhang Z (2018) Summarization filter: Consider more about the whole query in machine comprehension. IEEE Access 6:58702–58709
Song L, Wang Z, Hamza W, Zhang Y, Gildea D (2018) Leveraging context information for natural question generation. In: NAACL-HLT, New Orleans, Louisiana, pp 569–574
Song Y, Hu QV, He L (2019) P-cnn: Enhancing text matching with positional convolutional neural network. Knowledge-Based Sys 169:67–79
Subramanian S, Wang T, Yuan X, Zhang S, Trischler A, Bengio Y (2018) Neural models for key phrase extraction and question generation. In: QA@ACL, pp 78–88
Sukhbaatar S, Szlam A, Weston J, Fergus R (2015) End-to-end memory networks. In: NIPS, p 2440-2448
Sun Y, Xia T (2019) A hybrid network model for tibetan question answering. IEEE Access 7:52769–52777
Talmor A, Berant J (2018) Repartitioning of the complexwebquestions dataset. arXiv: abs/1807.09623
Talmor A, Berant J (2018) The web as a knowledge-base for answering complex questions. In: NAACL-HLT, pp 641–651
Tan C, Wei F, Zhou Q, Yang N, Du B, Lv W, Zhou M (2018) Context-aware answer sentence selection with hierarchical gated recurrent neural networks. IEEE/ACM Trans Audio, Speech and Language Process 26(3):540–549
Tay Y, Tuan LA, Hui SC (2018) Hyperbolic representation learning for fast and efficient neural question answering. In: WSDM, pp 583–591
Tixier AJP (2018) Notes on deep learning for nlp. arXiv: abs/1808.09772
Tolias K, Chatzis SP (2019) \(t\)-exponential memory networks for question-answering machines. IEEE Trans Neural Networks Learn Sys 30(8):2463–2477
Wang M, A Smith N, Mitamura T (2007) What is the jeopardy model? a quasi-synchronous grammar for qa. In: EMNLP-CoNLL, pp 22–32
Wang S, Zhou W, Jiang C (2020) A survey of word embeddings based on deep learning. Computing 102(3):717–740
Wang Z, Liu J, Xiao X, Lyu Y, Wu T (2018) Joint training of candidate extraction and answer selection for reading comprehension. In: ACL, pp 1715–1724
Wen J, Tu H, Cheng X, Xie R, Yin W (2019) Joint modeling of users, questions and answers for answer selection in cqa. Expert Sys Appl 118:563–572
Weston J, Bordes A, Chopra S, Rush AM, van Merriënboer B, Joulin A, Mikolov T (2016) Towards ai-complete question answering: A set of prerequisite toy tasks. In: ICLR (Poster)
Wu Y, Wu W, Li Z, Zhou M (2018) Knowledge enhanced hybrid neural network for text matching. In: AAAI, pp 5586–5593
Wulamu A, Sun Z, Xie Y, Xu C, Yang A (2019) An improved end-to-end memory network for qa tasks. Computers, Materials & Continua 60(3):1283–1295
Xia C, Zhang C, Yan X, Chang Y, Yu P (2018) Zero-shot user intent detection via capsule neural networks. In: EMNLP, pp 3090–3099
Xin J, Lin Y, Liu Z, Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: AAAI, pp 5997–6004
Yang B, Mitchell T (2017) Leveraging knowledge bases in lstms for improving machine reading. In: ACL, pp 1436–1446
Yang M, Tu W, Qu Q, Zhou W, Liu Q, Zhu J (2019) Advanced community question answering by leveraging external knowledge and multi-task learning. Knowledge-Based Sys 171:106–119
Yang X, Fan P (2019) Convolutional end-to-end memory networks for multi-hop reasoning. IEEE Access 7:135268–135276
Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le QV (2019) Xlnet: generalized autoregressive pretraining for language understanding. In: NeurIPS, pp 5754–5764
Yao X (2014) Feature-driven question answering with natural language alignment. John Hopkins University (PhD thesis)
Yih Wt, Richardson M, Meek C, Chang MW, Suh J (2016) The value of semantic parse labeling for knowledge base question answering. In: ACL, pp 201–206
Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75
Yuan X, Wang T, Gulcehre C, Sordoni A, Bachman P, Zhang S, Subramanian S, Trischler A (2017) Machine comprehension by text-to-text neural question generation. In: Rep4NLP@ACL, pp 15–25
Yue C, Cao H, Xiong K, Cui A, Qin H, Li M (2017) Enhanced question understanding with dynamic memory networks for textual question answering. Expert Sys Appl 80:39–45
Zhang L, Winn J, Tomioka R (2016) Gaussian attention model and its application to knowledge base embedding and question answering. arXiv: abs/1611.02266
Zhang S, Zhang X, Wang H, Cheng J, Li P, Ding Z (2017) Chinese medical question answer matching using end-to-end character-level multi-scale cnns. Appl Sci 7(8):767
Zhang S, Zhang X, Wang H, Guo L, Liu S (2018) Multi-scale attentive interaction networks for chinese medical question answer selection. IEEE Access 6:74061–74071
Zhang S, Zhang W, Niu J (2019) Improving short-text representation in convolutional networks by dependency parsing. Knowledge and Information Systems 61(1):463–484
Zhang X, Lu W, Li F, Peng X, Zhang R (2019) Deep feature fusion model for sentence semantic matching. Comput, Mater & Continua 61(2):601–616
Zhang Y, Dai H, Kozareva Z, Smola AJ, Le Song (2018) Variational reasoning for question answering with knowledge graph. In: AAAI, pp 6069–6076
Zhao Y, Ni X, Ding Y, Ke Q (2018) Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In: EMNLP, pp 3901–3910
Zhou M, Huang M, Zhu X (2018) An interpretable reasoning network for multi-relation question answering. In: COLING, pp 2010–2022
Zhou Q, Yang N, Wei F, Tan C, Bao H, Zhou M (2017) Neural question generation from text: A preliminary study. NLPCC 10619:662–671
Zhu S, Cheng X, Su S (2020) Knowledge-based question answering by tree-to-sequence learning. Neurocomputing 372:64–72
Acknowledgements
This work is supported by grants from National Natural Science Foundation of China (No. 61772146), The Science and Technology Plan of Guangzhou (No. 201804010296), and Natural Science Foundation of Guangdong Province, China (No. 2018A030310051).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hao, T., Li, X., He, Y. et al. Recent progress in leveraging deep learning methods for question answering. Neural Comput & Applic 34, 2765–2783 (2022). https://doi.org/10.1007/s00521-021-06748-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-021-06748-3