Skip to main content

Advertisement

Log in

TD-LSTM: a time distributed and deep-learning-based architecture for classification of motor imagery and execution in EEG signals

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

One of the critical challenges in brain-computer interfaces is the classification of brain activities through the analysis of EEG signals. This paper seeks to improve the efficacy of deep learning-based rehabilitation systems, aiming to deliver superior services for individuals with physical disabilities. The research introduces the time distributed long short-term memory (TD-LSTM) framework, which incorporates an LSTM and a time distributed approach to classify brain activities. Learning in TD-LSTM is achieved by uncovering time-dependent semantic dependencies within EEG signals over time. By extracting all discriminative and relevant spatiotemporal dependencies via TD-LSTM, valuable information on different time steps in each sequence has been obtained. Time distributed approach shortens the input time series, making learning from long time series sequences easier, and the learning process of complex temporal and spatial dependencies in time series multi-channel EEG signals becomes more efficient. The main contributions in this paper can be outlined as follows: (1) implementation of brain activity binary classification of motor imagery/execution tasks using time distributed approach via RNN network for the first time, (2) evaluation of the performance and generalizability of the proposed method on four benchmark datasets, (3) dual-purpose classification which providing an efficient ways for classifying both types of motor imagery/execution brain activity. The experimental results show that the proposed method performs well compared to several baseline research works.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Algorithm 7
Algorithm 8
Algorithm 9
Fig. 5
Algorithm 10
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

This research also uses well-known data from public repositories that can be shared based on request.

Notes

  1. http://www.bbci.de/competition/iv/#dataset2a.

  2. https://github.com/jingwang2020/ECML-PKDD_MMCNN..

  3. https://physionet.org/content/eegmmidb/1.0.0/.

  4. https://github.com/rootskar/EEGMotorImagery.

  5. https://github.com/MortezaKarimian/TD-LSTM.

References

  1. Graimann B, Allison BZ, Pfurtscheller G (2010) Brain-computer interfaces: revolutionizing human-computer interaction. Springer Science and Business Media

  2. Meng H-J et al (2018) Differences between motor execution and motor imagery of grasping movements in the motor cortical excitatory circuit. PeerJ 6:e5588–e5588

    Article  Google Scholar 

  3. Tayeb et al. (2019) Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors 19(1)

  4. Zhang G, Davoodnia V, Sepas-Moghaddam A, Zhang Y, Etemad A (2020) Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sens J 20(6):3113–3122

    Article  Google Scholar 

  5. Tabar YR, Halici U (2017) A novel deep learning approach for classification of EEG motor imagery signals. J Neural Eng 14(1):16003

    Article  Google Scholar 

  6. Wang Z, Cao L, Zhang Z, Gong X, Sun Y, Wang H (2018) Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition. Concurr Comput Pract Exp 30(23):e4413

    Article  Google Scholar 

  7. Roots K, Muhammad Y, Muhammad N (2020) Fusion convolutional neural network for cross-subject EEG motor imagery classification. Computers 9(3):72

    Article  Google Scholar 

  8. Riyad M, Khalil M, Adib A (2021) A novel multi-scale convolutional neural network for motor imagery classification. Biomed Signal Process Control 68:102747

    Article  Google Scholar 

  9. Hou Y, Zhou L, Jia S, Lun X (2020) A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 17(1):16048

    Article  Google Scholar 

  10. Zhang C, Kim Y-K, Eskandarian A (2021) EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification. J Neural Eng 18(4):46014

    Article  Google Scholar 

  11. Zhang X, Yao L, Huang C, Sheng QZ, Wang X (2017) Intent recognition in smart living through deep recurrent neural networks. In: International conference on neural information processing, pp 748–758

  12. Chen W et al. (2018) EEG-based motion intention recognition via multi-task RNNs. In: Proceedings of the 2018 SIAM International conference on data mining, society for industrial and applied mathematics, pp 279–287

  13. Chen W, Yue L, Li B, Wang C, Sheng QZ (2019) DAMTRNN: a delta attention-based multi-task RNN for intention recognition BT—advanced data mining and applications, pp 373–388

  14. Ma X, Qiu S, Du C, Xing J, He H (2018) Improving EEG-based motor imagery classification via spatial and temporal recurrent neural networks. In: 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 1903–1906

  15. Wang P, Jiang A, Liu X, Shang J, Zhang L (2018) LSTM-based EEG classification in motor imagery tasks. IEEE Trans Neural Syst Rehabil Eng 26(11):2086–2095

    Article  Google Scholar 

  16. Luo T, Zhou C, Chao F (2018) Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinform 19(1):344

    Article  Google Scholar 

  17. Idowu OP, Ilesanmi AE, Li X, Samuel OW, Fang P, Li G (2021) An integrated deep learning model for motor intention recognition of multi-class EEG signals in upper limb amputees. Comput Methods Programs Biomed 206:106121

    Article  Google Scholar 

  18. Zhang D et al. (2018) Cascade and parallel convolutional recurrent neural networks on EEG-based intention recognition for brain computer interface. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  19. Zhang K et al (2020) Data augmentation for motor imagery signal classification based on a hybrid neural network. Sensors 20(16):4485

    Article  Google Scholar 

  20. BCI competition IV (2008) Available: http://www.bbci.de/competition/iv/. Accessed 05 Apr 2021

  21. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51(6):1034–1043

    Article  Google Scholar 

  22. Van Houdt G, Mosquera C, Nápoles G (2020) A review on the long short-term memory model. Artif Intell Rev 53(8):5929–5955

    Article  Google Scholar 

  23. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  24. Krause B, Lu L, Murray I, Renals S (2016) Multiplicative LSTM for sequence modeling, arXiv Prepr. arXiv1609.07959

  25. TimeDistributed layer (2021) Available: https://keras.io/api/layers/recurrent_layers/time_distributed/. (Accessed 21 Apr 2021)

  26. Qiao H, Wang T, Wang P, Qiao S, Zhang L (2018) A time-distributed spatiotemporal feature learning method for machine health monitoring with multi-sensor time series. Sensors 18(9):2931

    Article  Google Scholar 

  27. Jason Brownlee (2017) Gentle introduction to models for sequence prediction with RNNs [Online]. Available: https://machinelearningmastery.com/models-sequence-prediction-recurrent-neural-networks/. Accessed 05 Oct 2021

  28. Ferlet P (2019) How to work with time distributed data in a neural network, https://medium.com/, 2019. [Online]. Available: https://medium.com/smileinnovation/how-to-work-with-time-distributed-data-in-a-neural-network-b8b39aa4ce00#:~:text=TimeDistributed layer is very useful,the data in “time”. Accessed 05 Apr 2021

  29. Al-Saegh A, Dawwd SA, Abdul-Jabbar JM (2021) Deep learning for motor imagery EEG-based classification: a review. Biomed Signal Process Control 63:102172

    Article  Google Scholar 

  30. Kumar S, Sharma A, Tsunoda T (2019) Brain wave classification using long short-term memory network based OPTICAL predictor. Sci Rep 9(1):9153

    Article  Google Scholar 

  31. Sethuraman P (2020) A comparison of DNN, CNN and LSTM using TF/Keras [Online]. Available: https://towardsdatascience.com/a-comparison-of-dnn-cnn-and-lstm-using-tf-keras-2191f8c77bbe. Accessed 25 May 2021

  32. So G (2019) Should we abandon LSTM for CNN? AI/ML at Symantec [Online]. Available: https://medium.com/ai-ml-at-symantec/should-we-abandon-lstm-for-cnn-83accaeb93d6#:~:text=An LSTM is designed to,well on images and speech. Accessed 25 May 2021

  33. Jingwei L, Yin C, Weidong Z (2015) Deep learning EEG response representation for brain computer interface. In: 2015 34th Chinese control conference (CCC), pp 3518–3523

  34. Ma X, Qiu S, He H (2022) Time-distributed attention network for EEG-based motor imagery decoding from the same limb. IEEE Trans Neural Syst Rehabil Eng 30:496–508

    Article  Google Scholar 

  35. Boloukian B, Safi-Esfahani F (2020) Recognition of words from brain-generated signals of speech-impaired people: application of autoencoders as a neural Turing machine controller in deep neural networks. Neural Netw 121:186–207

    Article  Google Scholar 

  36. Malekmohamadi Faradonbe S, Safi-Esfahani F, Karimian-kelishadrokhi M (2020) A review on neural turing machine (NTM). SN Comput Sci 1(6):333

    Article  Google Scholar 

  37. Caterini A (2017) A novel mathematical framework for the analysis of neural networks. UWSpace

  38. Python 3.6.0. [Online]. Available: https://www.python.org/downloads/release/python-360/. Accessed 02 Oct 2021

  39. TensorFlow [Online]. Available: https://www.tensorflow.org/. Accessed 02 Oct 2021

  40. Keras [Online]. Available: https://keras.io/. Accessed 02 Oct 2021

  41. NVIDIA cuDNN [Online]. Available: https://developer.nvidia.com/cudnn. Accessed 12 Aug 2021

  42. www.tensorflow.org [Online]. Available: https://www.tensorflow.org/api_docs/python/tf/compat/v1/keras/layers/CuDNNLSTM. Accessed 12 Aug 2021

  43. Jia Z, Lin Y, Wang J, Yang K, Liu T, Zhang X (2021) MMCNN: a multi-branch multi-scale convolutional neural network for motor imagery classification BT—machine learning and knowledge discovery in databases, pp 736–751

  44. Sleight J, Pillai PJ, Mohan S (2009) Classification of executed and imagined motor movement EEG signals

  45. Recurrent dropout -CuDNNLSTM.” [Online]. Available: https://github.com/keras-team/keras/issues/8935#issuecomment-365733367. Accessed 03 Oct 2021

  46. Abdulkader SN, Atia A, Mostafa M-SM (2015) Brain computer interfacing: applications and challenges. Egypt Informatics J 16(2):213–230

    Article  Google Scholar 

  47. Asaei-Moamam ZS, Safi-Esfahani F, Mirjalili S, Mohammadpour R, Nadimi-Shahraki MH (2023) DAerosol-NTM: applying deep learning and neural Turing machine in aerosol prediction. Neural Comput Appl 35(34):24123–24159. https://doi.org/10.1007/s00521-023-08868-4

    Article  Google Scholar 

  48. Asaei-Moamam ZS, Safi-Esfahani F, Mirjalili S, Mohammadpour R, Nadimi-Shahraki MH (2023) Air quality particulate-pollution prediction applying GAN network and the neural turing machine. Appl Soft Comput 147:110723. https://doi.org/10.1016/j.asoc.2023.110723

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Faramarz Safi-Esfahani.

Ethics declarations

Conflict of interest

There is no conflict of interest for this submission entitled “TD-LSTM: A Time Distributed and Deep-Learning-Based Architecture for Classification of Motor Imagery and Execution in EEG Signals.”

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karimian-Kelishadrokhi, M., Safi-Esfahani, F. TD-LSTM: a time distributed and deep-learning-based architecture for classification of motor imagery and execution in EEG signals. Neural Comput & Applic 36, 15843–15868 (2024). https://doi.org/10.1007/s00521-024-09731-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09731-w

Keywords

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy