Skip to main content

Advertisement

Log in

Recent progress in leveraging deep learning methods for question answering

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Question answering, serving as one of important tasks in natural language processing, enables machines to understand questions in natural language and answer the questions concisely. From web search to expert systems, question answering systems are widely applied to various domains in assisting information seeking. Deep learning methods have boosted various tasks of question answering and have demonstrated dramatic effects in performance improvement for essential steps of question answering. Thus, leveraging deep learning methods for question answering has drawn much attention from both academia and industry in recent years. This paper provides a systematic review of the recent development of deep learning methods for question answering. The survey covers the scope including methods, datasets, and applications. The methods are discussed in terms of network structure characteristics, methodology innovations, and their effectiveness. The survey is expected to be a contribution to the summarization of recent research progress and future directions of deep learning methods for question answering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Adlouni YE, Rodríguez H, Meknassi M, El Alaoui SO, En-nahnahi N (2019) A multi-approach to community question answering. Expert Sys Appl 137:432–442

    Article  Google Scholar 

  2. Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, Hasan M, van Essen BC, Awwal AAS, Asari VK (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292

    Article  Google Scholar 

  3. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: ICLR

  4. Banerjee S, Naskar S, Rosso P, Bandyopadhyay S (2018) Code mixed cross script factoid question classification - a deep learning approach. J Intell & Fuzzy Sys 34(5):2959–2969

    Article  Google Scholar 

  5. Bast H, Haussmann E (2015) More accurate question answering on freebase. In: CIKM’15, pp 1431–1440

  6. Ben Abacha A, Demner-Fushman D (2019) A question-entailment approach to question answering. BMC Bioinfo 20(1):e33

    Article  Google Scholar 

  7. Bengio Y (2009) Learning deep architectures for AI. Found Trends in Machine Learn 2(1):1–127

    Article  MathSciNet  MATH  Google Scholar 

  8. Berant J, Chou A, Roy F, Liang P (2013) Semantic parsing on freebase from question-answer pairs. In: EMNLP, pp 1533–1544

  9. Bi M, Zhang Q, Zuo M, Xu Z, Jin Q (2019) Bi-directional lstm model with symptoms-frequency position attention for question answering system in medical domain. Neural Process Lett 51(5):570

    Google Scholar 

  10. Bisk Y, Reddy S, Blitzer J, Hockenmaier J, Steedman M (2016) Evaluating induced ccg parsers on grounded semantic parsing. In: EMNLP, pp 2022–2027

  11. Cai L, Zhou S, Yan X (2019) Yuan R (2019) A stacked bilstm neural network based on coattention mechanism for question answering. Computat Intell Neurosci 9:1–12

    Google Scholar 

  12. Cai LQ, Wei M, Zhou ST, Yan X (2020) Intelligent question answering in restricted domains using deep learning and question pair matching. IEEE Access 8:32922–32934

    Article  Google Scholar 

  13. Chen Z, Zhang C, Zhao Z, Yao C, Cai D (2018) Question retrieval for community-based question answering via heterogeneous social influential network. Neurocomputing 285:117–124

    Article  Google Scholar 

  14. Chen ZY, Chang CH, Chen YP, Nayak J, Ku LW (2019) Uhop: An unrestricted-hop relation extraction framework for knowledge-based question answering. In: NAACL-HLT, pp 345–356

  15. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. In: EMNLP, pp 1724–1734

  16. Cortes E, Woloszyn V, Binder A, Himmelsbach T, Barone D, Möller S (2020) An empirical comparison of question classification methods for question answering systems. In: LREC, pp 5408–5416

  17. Croce D, Filice S, Basili R (2019) Making sense of kernel spaces in neural learning. Computer Speech & Language 58:51–75

    Article  Google Scholar 

  18. Dargan S, Kumar M, Ayyagari MR, Kumar G (2019) A survey of deep learning and its applications: A new paradigm to machine learning. Archi Computat Method Eng 85(4):114

    MathSciNet  Google Scholar 

  19. Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp 4171–4186

  20. Dimitrakis E, Sgontzos K, Tzitzikas Y (2019) A survey on question answering systems over linked data and documents. J Intell Info Sys 51(5):570

    Google Scholar 

  21. Dong L, Mallinson J, Reddy S, Lapata M (2017) Learning to paraphrase for question answering. In: EMNLP, pp 875–886

  22. Du X, Shao J, Cardie C (2017) Learning to ask: Neural question generation for reading comprehension. In: ACL, pp 1342–1352

  23. Dubey M, Banerjee D, Abdelkawi A, Lehmann J (2019) Lc-quad 2.0: A large dataset for complex question answering over wikidata and dbpedia. SEMWEB 11779:69–78

    Google Scholar 

  24. Elman JL (1990) Finding structure in time. Cognitive Sci 14(2):179–211

    Article  Google Scholar 

  25. Elsahar H, Gravier C, Laforest F (2018) Zero-shot question generation from knowledge graphs for unseen predicates and entity types. In: NAACL-HLT, pp 218–228

  26. Fukushima K (1988) Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Networks 1(2):119–130

    Article  Google Scholar 

  27. Garg S, Vu T, Moschitti A (2020) Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. AAAI 34:7780–7788

    Article  Google Scholar 

  28. Goldberg Y (2016) A primer on neural network models for natural language processing. J Artif Intell Res 57(1):345–420

    Article  MathSciNet  MATH  Google Scholar 

  29. Green BF, Wolf AK, Chomsky C, Laughery K (1961) Baseball: an automatic question-answerer. In: IRE-AIEE-ACM ’61 (Western), pp 219–224

  30. Gulcehre C, Ahn S, Nallapati R, Zhou B, Bengio Y (2016) Pointing the unknown words. In: ACL, pp 140–149

  31. Hao Z, Wu B, Wen W, Cai R (2019) A subgraph-representation-based method for answering complex questions over knowledge bases. Neural Networks 119:57–65

    Article  Google Scholar 

  32. He J, Fu M, Tu M (2019) Applying deep matching networks to chinese medical question answering: a study and a dataset. BMC Med Info Decision Making 19(S2):1

    Google Scholar 

  33. Hirschman L, Gaizauskas R (2001) Natural language question answering: the view from here. Nat Lang Eng 7(4):275–300

    Article  Google Scholar 

  34. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Computat 9(8):1735–1780

    Article  Google Scholar 

  35. Huang H, Wei X, Nie L, Mao X, Xu XS (2019) From question to text: Question-oriented feature attention for answer selection. ACM Trans Info Sys 37(1):1–33

    Article  Google Scholar 

  36. Huang W, Qu Q, Yang M (2020) Interactive knowledge-enhanced attention network for answer selection. Neural Comput Appl 32(15):11343–11359

    Article  Google Scholar 

  37. Indurthi SR, Raghu D, Khapra MM, Joshi S (2017) Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In: EACL, pp 376–385

  38. Jiang B, Tan L, Ren Y, Li F (2019) Intelligent interaction with virtual geographical environments based on geographic knowledge graph. ISPRS Int J Geo-Info 8(10):428

    Article  Google Scholar 

  39. Jing L, Gulcehre C, Peurifoy J, Shen Y, Tegmark M, Soljacic M, Bengio Y (2019) Gated orthogonal recurrent units: on learning to forget. Neural Computat 31(4):765–783

    Article  MathSciNet  MATH  Google Scholar 

  40. Khalifa M, Shaalan K (2019) Character convolutions for arabic named entity recognition with long short-term memory networks. Comp Speech & Language 58:335–346

    Article  Google Scholar 

  41. Kim S, Park D, Choi Y, Lee K, Kim B, Jeon M, Kim J, Tan AC, Kang J (2018) A pilot study of biomedical text comprehension using an attention-based deep neural reader: design and experimental analysis. JMIR Med Info 6(1):e2

    Article  Google Scholar 

  42. Kim Y, Lee H, Shin J, Jung K (2019) Improving neural question generation using answer separation. AAAI 33:6602–6609

    Article  Google Scholar 

  43. Kolomiyets O, Moens MF (2011) A survey on question answering technology from an information retrieval perspective. Info Sci 181(24):5412–5434

    Article  MathSciNet  Google Scholar 

  44. Kumar A, Irsoy O, Ondruska P, Iyyer M, Bradbury J, Gulrajani I, Zhong V, Paulus R, Socher R (2016) Ask me anything: Dynamic memory networks for natural language processing. In: ICML, pp 1378–1387

  45. Kumar V, Hua Y, Ramakrishnan G, Qi G, Gao L, Li YF (2019) Difficulty-controllable multi-hop question generation from knowledge graphs. SEMWEB 11778:382–398

    Google Scholar 

  46. Lan Y, Jiang J (2020) Query graph generation for answering multi-hop complex questions from knowledge bases. In: ACL, pp 969–974

  47. Lan Y, Wang S, Jiang J (2019) Knowledge base question answering with a matching-aggregation model and question-specific contextual relations. IEEE/ACM Trans Audio, Speech, and Language Process 27(10):1629–1638

    Article  Google Scholar 

  48. Lan Y, Wang S, Jiang J (2019) Multi-hop knowledge base question answering with an iterative sequence matching model. In: ICDM, pp 359–368

  49. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2020) Albert: A lite bert for self-supervised learning of language representations. In: ICLR

  50. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceed of the IEEE 86:2278–2324

    Article  Google Scholar 

  51. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  52. Lee CH, Lee HY, Wu SL, Liu CL, Fang W, Hsu JY, Tseng BH (2019) Machine comprehension of spoken content: Toefl listening test and spoken squad. IEEE/ACM Trans on Audio, Speech, and Language Process 27(9):1469–1480

    Article  Google Scholar 

  53. Li J, Sun A, Han J, Li C (2022) A survey on deep learning for named entity recognition. IEEE Transact Knowledge & Data Eng 34:50–70

  54. Li X, Zhang S, Wang B, Gao Z, Fang L, Xu H (2019) A hybrid framework for problem solving of comparative questions. IEEE Access 7:185961–185976

    Article  Google Scholar 

  55. Lin T, Goyal P, Girshick R, He K, Dollár P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Machine Intell 42(2):318–327

    Article  Google Scholar 

  56. Liu D, Niu Z, Zhang C, Zhang J (2019) Multi-scale deformable cnn for answer selection. IEEE Access 7:164986–164995

    Article  Google Scholar 

  57. Liu H, Liu Y, Wong LP, Lee LK, Hao T (2020) A hybrid neural network bert-cap based on pre-trained language model and capsule network for user intent classification. Complexity 2020:1–11

    Google Scholar 

  58. Luo K, Lin F, Luo X, Zhu K (2018) Knowledge base question answering via encoding of complex query graphs. In: EMNLP, pp 2185–2194

  59. Mahmoud A, Zrigui M (2019) Sentence embedding and convolutional neural network for semantic textual similarity detection in arabic language. Arab J Sci Eng 44(11):9263–9274

    Article  Google Scholar 

  60. Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao J (2021) Deep learning-based text classification: A comprehensive review. ACM Comput Surv 54(3):62:1–62:40

  61. Ojokoh B, Adebisi E (2019) A review of question answering systems. J Web Eng 17(8):717–758

    Article  Google Scholar 

  62. Otter DW, Medina JR, Kalita JK (2021) A survey of the usages of deep learning in natural language processing. IEEE Trans Neural Network Learn Sys 32:604–624

    Article  Google Scholar 

  63. Pan L, Lei W, Chua TS, Kan MY (2019) Recent advances in neural question generation. ArXiv abs/1905.08949

  64. Parshakova T, Rameau F, Serdega A, Kweon IS, Kim DS (2019) Latent question interpretation through variational adaptation. IEEE/ACM Trans Audio, Speech and Language Process 27(11):1713–1724

    Article  Google Scholar 

  65. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: NAACL-HLT, pp 2227–2237

  66. Qu Y, Liu J, Kang L, Shi Q, Ye D (2018) Question answering over freebase via attentive rnn with similarity matrix based cnn. arXiv: abs/1804.03317

  67. Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. In: EMNLP, pp 2383–2392

  68. Ren Q, Cheng X, Su S (2020) Multi-task learning with generative adversarial training for multi-passage machine reading comprehension. AAAI 34:8705–8712

    Article  Google Scholar 

  69. Roy PK, Singh JP (2019) Predicting closed questions on community question answering sites using convolutional neural network. Neural Comput Appl 19(5):53

    Google Scholar 

  70. Sanh V, Debut L, Chaumond J, Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv: abs/1910.01108

  71. Sawant U, Garg S, Chakrabarti S, Ramakrishnan G (2019) Neural architecture for question answering using a knowledge graph and web corpus. Info Retr J 22(3–4):324–349

    Article  Google Scholar 

  72. Shah AA, Ravana SD, Hamid S, Ismail MA (2018) Accuracy evaluation of methods and techniques in web-based question answering systems: a survey. Knowl Info Sys 58(03):611–650

    Article  Google Scholar 

  73. Shao T, Guo Y, Chen H, Hao Z (2019) Transformer-based neural network for answer selection in question answering. IEEE Access 7:26146–26156

    Article  Google Scholar 

  74. Shao T, Kui X, Zhang P, Chen H (2019) Collaborative learning for answer selection in question answering. IEEE Access 7:7337–7347

    Article  Google Scholar 

  75. Shuang K, Liu Y, Zhang W, Zhang Z (2018) Summarization filter: Consider more about the whole query in machine comprehension. IEEE Access 6:58702–58709

    Article  Google Scholar 

  76. Song L, Wang Z, Hamza W, Zhang Y, Gildea D (2018) Leveraging context information for natural question generation. In: NAACL-HLT, New Orleans, Louisiana, pp 569–574

  77. Song Y, Hu QV, He L (2019) P-cnn: Enhancing text matching with positional convolutional neural network. Knowledge-Based Sys 169:67–79

    Article  Google Scholar 

  78. Subramanian S, Wang T, Yuan X, Zhang S, Trischler A, Bengio Y (2018) Neural models for key phrase extraction and question generation. In: QA@ACL, pp 78–88

  79. Sukhbaatar S, Szlam A, Weston J, Fergus R (2015) End-to-end memory networks. In: NIPS, p 2440-2448

  80. Sun Y, Xia T (2019) A hybrid network model for tibetan question answering. IEEE Access 7:52769–52777

    Article  Google Scholar 

  81. Talmor A, Berant J (2018) Repartitioning of the complexwebquestions dataset. arXiv: abs/1807.09623

  82. Talmor A, Berant J (2018) The web as a knowledge-base for answering complex questions. In: NAACL-HLT, pp 641–651

  83. Tan C, Wei F, Zhou Q, Yang N, Du B, Lv W, Zhou M (2018) Context-aware answer sentence selection with hierarchical gated recurrent neural networks. IEEE/ACM Trans Audio, Speech and Language Process 26(3):540–549

    Article  Google Scholar 

  84. Tay Y, Tuan LA, Hui SC (2018) Hyperbolic representation learning for fast and efficient neural question answering. In: WSDM, pp 583–591

  85. Tixier AJP (2018) Notes on deep learning for nlp. arXiv: abs/1808.09772

  86. Tolias K, Chatzis SP (2019) \(t\)-exponential memory networks for question-answering machines. IEEE Trans Neural Networks Learn Sys 30(8):2463–2477

    Article  MathSciNet  Google Scholar 

  87. Wang M, A Smith N, Mitamura T (2007) What is the jeopardy model? a quasi-synchronous grammar for qa. In: EMNLP-CoNLL, pp 22–32

  88. Wang S, Zhou W, Jiang C (2020) A survey of word embeddings based on deep learning. Computing 102(3):717–740

    Article  MathSciNet  MATH  Google Scholar 

  89. Wang Z, Liu J, Xiao X, Lyu Y, Wu T (2018) Joint training of candidate extraction and answer selection for reading comprehension. In: ACL, pp 1715–1724

  90. Wen J, Tu H, Cheng X, Xie R, Yin W (2019) Joint modeling of users, questions and answers for answer selection in cqa. Expert Sys Appl 118:563–572

    Article  Google Scholar 

  91. Weston J, Bordes A, Chopra S, Rush AM, van Merriënboer B, Joulin A, Mikolov T (2016) Towards ai-complete question answering: A set of prerequisite toy tasks. In: ICLR (Poster)

  92. Wu Y, Wu W, Li Z, Zhou M (2018) Knowledge enhanced hybrid neural network for text matching. In: AAAI, pp 5586–5593

  93. Wulamu A, Sun Z, Xie Y, Xu C, Yang A (2019) An improved end-to-end memory network for qa tasks. Computers, Materials & Continua 60(3):1283–1295

    Article  Google Scholar 

  94. Xia C, Zhang C, Yan X, Chang Y, Yu P (2018) Zero-shot user intent detection via capsule neural networks. In: EMNLP, pp 3090–3099

  95. Xin J, Lin Y, Liu Z, Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: AAAI, pp 5997–6004

  96. Yang B, Mitchell T (2017) Leveraging knowledge bases in lstms for improving machine reading. In: ACL, pp 1436–1446

  97. Yang M, Tu W, Qu Q, Zhou W, Liu Q, Zhu J (2019) Advanced community question answering by leveraging external knowledge and multi-task learning. Knowledge-Based Sys 171:106–119

    Article  Google Scholar 

  98. Yang X, Fan P (2019) Convolutional end-to-end memory networks for multi-hop reasoning. IEEE Access 7:135268–135276

    Article  Google Scholar 

  99. Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le QV (2019) Xlnet: generalized autoregressive pretraining for language understanding. In: NeurIPS, pp 5754–5764

  100. Yao X (2014) Feature-driven question answering with natural language alignment. John Hopkins University (PhD thesis)

  101. Yih Wt, Richardson M, Meek C, Chang MW, Suh J (2016) The value of semantic parse labeling for knowledge base question answering. In: ACL, pp 201–206

  102. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75

    Article  Google Scholar 

  103. Yuan X, Wang T, Gulcehre C, Sordoni A, Bachman P, Zhang S, Subramanian S, Trischler A (2017) Machine comprehension by text-to-text neural question generation. In: Rep4NLP@ACL, pp 15–25

  104. Yue C, Cao H, Xiong K, Cui A, Qin H, Li M (2017) Enhanced question understanding with dynamic memory networks for textual question answering. Expert Sys Appl 80:39–45

    Article  Google Scholar 

  105. Zhang L, Winn J, Tomioka R (2016) Gaussian attention model and its application to knowledge base embedding and question answering. arXiv: abs/1611.02266

  106. Zhang S, Zhang X, Wang H, Cheng J, Li P, Ding Z (2017) Chinese medical question answer matching using end-to-end character-level multi-scale cnns. Appl Sci 7(8):767

    Article  Google Scholar 

  107. Zhang S, Zhang X, Wang H, Guo L, Liu S (2018) Multi-scale attentive interaction networks for chinese medical question answer selection. IEEE Access 6:74061–74071

    Article  Google Scholar 

  108. Zhang S, Zhang W, Niu J (2019) Improving short-text representation in convolutional networks by dependency parsing. Knowledge and Information Systems 61(1):463–484

    Article  Google Scholar 

  109. Zhang X, Lu W, Li F, Peng X, Zhang R (2019) Deep feature fusion model for sentence semantic matching. Comput, Mater & Continua 61(2):601–616

    Article  Google Scholar 

  110. Zhang Y, Dai H, Kozareva Z, Smola AJ, Le Song (2018) Variational reasoning for question answering with knowledge graph. In: AAAI, pp 6069–6076

  111. Zhao Y, Ni X, Ding Y, Ke Q (2018) Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In: EMNLP, pp 3901–3910

  112. Zhou M, Huang M, Zhu X (2018) An interpretable reasoning network for multi-relation question answering. In: COLING, pp 2010–2022

  113. Zhou Q, Yang N, Wei F, Tan C, Bao H, Zhou M (2017) Neural question generation from text: A preliminary study. NLPCC 10619:662–671

    Google Scholar 

  114. Zhu S, Cheng X, Su S (2020) Knowledge-based question answering by tree-to-sequence learning. Neurocomputing 372:64–72

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by grants from National Natural Science Foundation of China (No. 61772146), The Science and Technology Plan of Guangzhou (No. 201804010296), and Natural Science Foundation of Guangdong Province, China (No. 2018A030310051).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yingying Qu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, T., Li, X., He, Y. et al. Recent progress in leveraging deep learning methods for question answering. Neural Comput & Applic 34, 2765–2783 (2022). https://doi.org/10.1007/s00521-021-06748-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06748-3

Keywords

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy