Abstract
One of the critical challenges in brain-computer interfaces is the classification of brain activities through the analysis of EEG signals. This paper seeks to improve the efficacy of deep learning-based rehabilitation systems, aiming to deliver superior services for individuals with physical disabilities. The research introduces the time distributed long short-term memory (TD-LSTM) framework, which incorporates an LSTM and a time distributed approach to classify brain activities. Learning in TD-LSTM is achieved by uncovering time-dependent semantic dependencies within EEG signals over time. By extracting all discriminative and relevant spatiotemporal dependencies via TD-LSTM, valuable information on different time steps in each sequence has been obtained. Time distributed approach shortens the input time series, making learning from long time series sequences easier, and the learning process of complex temporal and spatial dependencies in time series multi-channel EEG signals becomes more efficient. The main contributions in this paper can be outlined as follows: (1) implementation of brain activity binary classification of motor imagery/execution tasks using time distributed approach via RNN network for the first time, (2) evaluation of the performance and generalizability of the proposed method on four benchmark datasets, (3) dual-purpose classification which providing an efficient ways for classifying both types of motor imagery/execution brain activity. The experimental results show that the proposed method performs well compared to several baseline research works.






















Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
This research also uses well-known data from public repositories that can be shared based on request.
References
Graimann B, Allison BZ, Pfurtscheller G (2010) Brain-computer interfaces: revolutionizing human-computer interaction. Springer Science and Business Media
Meng H-J et al (2018) Differences between motor execution and motor imagery of grasping movements in the motor cortical excitatory circuit. PeerJ 6:e5588–e5588
Tayeb et al. (2019) Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors 19(1)
Zhang G, Davoodnia V, Sepas-Moghaddam A, Zhang Y, Etemad A (2020) Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sens J 20(6):3113–3122
Tabar YR, Halici U (2017) A novel deep learning approach for classification of EEG motor imagery signals. J Neural Eng 14(1):16003
Wang Z, Cao L, Zhang Z, Gong X, Sun Y, Wang H (2018) Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition. Concurr Comput Pract Exp 30(23):e4413
Roots K, Muhammad Y, Muhammad N (2020) Fusion convolutional neural network for cross-subject EEG motor imagery classification. Computers 9(3):72
Riyad M, Khalil M, Adib A (2021) A novel multi-scale convolutional neural network for motor imagery classification. Biomed Signal Process Control 68:102747
Hou Y, Zhou L, Jia S, Lun X (2020) A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 17(1):16048
Zhang C, Kim Y-K, Eskandarian A (2021) EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification. J Neural Eng 18(4):46014
Zhang X, Yao L, Huang C, Sheng QZ, Wang X (2017) Intent recognition in smart living through deep recurrent neural networks. In: International conference on neural information processing, pp 748–758
Chen W et al. (2018) EEG-based motion intention recognition via multi-task RNNs. In: Proceedings of the 2018 SIAM International conference on data mining, society for industrial and applied mathematics, pp 279–287
Chen W, Yue L, Li B, Wang C, Sheng QZ (2019) DAMTRNN: a delta attention-based multi-task RNN for intention recognition BT—advanced data mining and applications, pp 373–388
Ma X, Qiu S, Du C, Xing J, He H (2018) Improving EEG-based motor imagery classification via spatial and temporal recurrent neural networks. In: 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 1903–1906
Wang P, Jiang A, Liu X, Shang J, Zhang L (2018) LSTM-based EEG classification in motor imagery tasks. IEEE Trans Neural Syst Rehabil Eng 26(11):2086–2095
Luo T, Zhou C, Chao F (2018) Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinform 19(1):344
Idowu OP, Ilesanmi AE, Li X, Samuel OW, Fang P, Li G (2021) An integrated deep learning model for motor intention recognition of multi-class EEG signals in upper limb amputees. Comput Methods Programs Biomed 206:106121
Zhang D et al. (2018) Cascade and parallel convolutional recurrent neural networks on EEG-based intention recognition for brain computer interface. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Zhang K et al (2020) Data augmentation for motor imagery signal classification based on a hybrid neural network. Sensors 20(16):4485
BCI competition IV (2008) Available: http://www.bbci.de/competition/iv/. Accessed 05 Apr 2021
Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51(6):1034–1043
Van Houdt G, Mosquera C, Nápoles G (2020) A review on the long short-term memory model. Artif Intell Rev 53(8):5929–5955
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Krause B, Lu L, Murray I, Renals S (2016) Multiplicative LSTM for sequence modeling, arXiv Prepr. arXiv1609.07959
TimeDistributed layer (2021) Available: https://keras.io/api/layers/recurrent_layers/time_distributed/. (Accessed 21 Apr 2021)
Qiao H, Wang T, Wang P, Qiao S, Zhang L (2018) A time-distributed spatiotemporal feature learning method for machine health monitoring with multi-sensor time series. Sensors 18(9):2931
Jason Brownlee (2017) Gentle introduction to models for sequence prediction with RNNs [Online]. Available: https://machinelearningmastery.com/models-sequence-prediction-recurrent-neural-networks/. Accessed 05 Oct 2021
Ferlet P (2019) How to work with time distributed data in a neural network, https://medium.com/, 2019. [Online]. Available: https://medium.com/smileinnovation/how-to-work-with-time-distributed-data-in-a-neural-network-b8b39aa4ce00#:~:text=TimeDistributed layer is very useful,the data in “time”. Accessed 05 Apr 2021
Al-Saegh A, Dawwd SA, Abdul-Jabbar JM (2021) Deep learning for motor imagery EEG-based classification: a review. Biomed Signal Process Control 63:102172
Kumar S, Sharma A, Tsunoda T (2019) Brain wave classification using long short-term memory network based OPTICAL predictor. Sci Rep 9(1):9153
Sethuraman P (2020) A comparison of DNN, CNN and LSTM using TF/Keras [Online]. Available: https://towardsdatascience.com/a-comparison-of-dnn-cnn-and-lstm-using-tf-keras-2191f8c77bbe. Accessed 25 May 2021
So G (2019) Should we abandon LSTM for CNN? AI/ML at Symantec [Online]. Available: https://medium.com/ai-ml-at-symantec/should-we-abandon-lstm-for-cnn-83accaeb93d6#:~:text=An LSTM is designed to,well on images and speech. Accessed 25 May 2021
Jingwei L, Yin C, Weidong Z (2015) Deep learning EEG response representation for brain computer interface. In: 2015 34th Chinese control conference (CCC), pp 3518–3523
Ma X, Qiu S, He H (2022) Time-distributed attention network for EEG-based motor imagery decoding from the same limb. IEEE Trans Neural Syst Rehabil Eng 30:496–508
Boloukian B, Safi-Esfahani F (2020) Recognition of words from brain-generated signals of speech-impaired people: application of autoencoders as a neural Turing machine controller in deep neural networks. Neural Netw 121:186–207
Malekmohamadi Faradonbe S, Safi-Esfahani F, Karimian-kelishadrokhi M (2020) A review on neural turing machine (NTM). SN Comput Sci 1(6):333
Caterini A (2017) A novel mathematical framework for the analysis of neural networks. UWSpace
Python 3.6.0. [Online]. Available: https://www.python.org/downloads/release/python-360/. Accessed 02 Oct 2021
TensorFlow [Online]. Available: https://www.tensorflow.org/. Accessed 02 Oct 2021
Keras [Online]. Available: https://keras.io/. Accessed 02 Oct 2021
NVIDIA cuDNN [Online]. Available: https://developer.nvidia.com/cudnn. Accessed 12 Aug 2021
www.tensorflow.org [Online]. Available: https://www.tensorflow.org/api_docs/python/tf/compat/v1/keras/layers/CuDNNLSTM. Accessed 12 Aug 2021
Jia Z, Lin Y, Wang J, Yang K, Liu T, Zhang X (2021) MMCNN: a multi-branch multi-scale convolutional neural network for motor imagery classification BT—machine learning and knowledge discovery in databases, pp 736–751
Sleight J, Pillai PJ, Mohan S (2009) Classification of executed and imagined motor movement EEG signals
Recurrent dropout -CuDNNLSTM.” [Online]. Available: https://github.com/keras-team/keras/issues/8935#issuecomment-365733367. Accessed 03 Oct 2021
Abdulkader SN, Atia A, Mostafa M-SM (2015) Brain computer interfacing: applications and challenges. Egypt Informatics J 16(2):213–230
Asaei-Moamam ZS, Safi-Esfahani F, Mirjalili S, Mohammadpour R, Nadimi-Shahraki MH (2023) DAerosol-NTM: applying deep learning and neural Turing machine in aerosol prediction. Neural Comput Appl 35(34):24123–24159. https://doi.org/10.1007/s00521-023-08868-4
Asaei-Moamam ZS, Safi-Esfahani F, Mirjalili S, Mohammadpour R, Nadimi-Shahraki MH (2023) Air quality particulate-pollution prediction applying GAN network and the neural turing machine. Appl Soft Comput 147:110723. https://doi.org/10.1016/j.asoc.2023.110723
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
There is no conflict of interest for this submission entitled “TD-LSTM: A Time Distributed and Deep-Learning-Based Architecture for Classification of Motor Imagery and Execution in EEG Signals.”
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Karimian-Kelishadrokhi, M., Safi-Esfahani, F. TD-LSTM: a time distributed and deep-learning-based architecture for classification of motor imagery and execution in EEG signals. Neural Comput & Applic 36, 15843–15868 (2024). https://doi.org/10.1007/s00521-024-09731-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-024-09731-w