Abstract
Neuromorphic models take inspiration from the human brain by adopting bio-plausible neuron models to build alternatives to traditional Machine Learning (ML) and Deep Learning (DL) solutions. The scarce availability of dedicated hardware able to actualize the emulation of brain-inspired computation, which is otherwise only simulated, yet still hinders the wide adoption of neuromorphic computing for edge devices and embedded systems. With this premise, we adopt the perspective of neuromorphic computing for conventional hardware and we present the L\(^2\)MU, a natively neuromorphic Legendre Memory Unit (LMU) which entirely relies on Leaky Integrate-and-Fire (LIF) neurons. Specifically, the original recurrent architecture of LMU has been redesigned by modelling every constituent element with neural populations made of LIF or Current-Based (CuBa) LIF neurons. To couple neuromorphic computing and off-the-shelf edge devices, we equipped the L\(^2\)MU with an input module for the conversion of real values into spikes, which makes it an encoding-free implementation of a Recurrent Spiking Neural Network (RSNN) able to directly work with raw sensor signals on non-dedicated hardware. As a use case to validate our network, we selected the task of Human Activity Recognition (HAR). We benchmarked our L\(^2\)MU on smartwatch signals from hand-oriented activities, deploying it on three different commercial edge devices in compressed versions too. The reported results remark the possibility of considering neuromorphic models not only in an exclusive relationship with dedicated hardware but also as a suitable choice to work with common sensors and devices.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allahbakhshi, H., Conrow, L., Naimi, B., Weibel, R.: Using accelerometer and GPS data for real-life physical activity type detection. Sensors (Switzerland) 20 (2020). 10.3390/s20030588
Arsalan, M., Santra, A., Issakov, V.: Low power radar-based air-writing system using genetic algorithm-assisted spiking legendre memory unit. In: 20th European Radar Conference (EuRAD) (2023)
Bartlett, M.E., Stewart, T.C., Thill, S.: Estimating levels of engagement for social human-robot interaction using legendre memory units. In: ACM/IEEE International Conference on Human-Robot Interaction (2021)
Bekolay, T., et al.: Nengo: a Python tool for building large-scale functional brain models. Front. Neuroinform. 7 (2014). 10.3389/fninf.2013.00048
Bos, H., Muir, D.: Sub-mw neuromorphic snn audio processing applications with rockpool and xylo. In: Embedded Artificial Intelligence. River Publishers (2023)
Capela, N.A., Lemaire, E.D., Baddour, N.: Feature Selection for Wearable Smartphone-Based Human Activity Recognition with Able bodied, Elderly, and Stroke Patients. PLOS ONE 10 (2015). https://doi.org/10.1371/journal.pone.0124414
Ceolini, E., et al.: Hand-gesture recognition based on emg and event-based camera sensor fusion: a benchmark in neuromorphic computing. Front. Neurosci. 14 (2020). https://doi.org/10.3389/fnins.2020.00637
Dami, S., Yahaghizadeh, M.: Predicting cardiovascular events with deep learning approach in the context of the internet of things. Neural Comput. Appl. 33 (2021). https://doi.org/10.1007/s00521-020-05542-x
Davies, M., et al.: Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38 (2018). https://doi.org/10.1109/MM.2018.112130359
Demrozi, F., Pravadelli, G., Bihorac, A., Rashidi, P.: Human activity recognition using inertial, physiological and environmental sensors: a comprehensive survey. IEEE Access 8 (2020). https://doi.org/10.1109/ACCESS.2020.3037715
Eshraghian, J.K., Ward, M., Neftci, E., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D.S., Lu, W.D.: Training spiking neural networks using lessons from deep learning. arXiv preprint arXiv:2109.12894 (2024)
Ferrari, A., Micucci, D., Mobilio, M., Napoletano, P.: Trends in human activity recognition using smartphones. J. Reliable Intell. Environ. 7 (2021). https://doi.org/10.1007/s40860-021-00147-0
Fra, V., Forno, E., Pignari, R., Stewart, T.C., Macii, E., Urgese, G.: Human activity recognition: suitability of a neuromorphic approach for on-edge AIoT applications. Neuromorphic Comput. Eng. 2 (2022). https://doi.org/10.1088/2634-4386/ac4c38
Frank, A.E., Kubota, A., Riek, L.D.: Wearable activity recognition for robust human-robot teaming in safety-critical environments via hybrid neural networks. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2019). https://doi.org/10.1109/IROS40897.2019.8968615
Gaurav, R., Stewart, T.C., Yi, Y.: Reservoir based spiking models for univariate time series classification. Frontiers in Computational Neuroscience (2023)
Gomaa, W., Khamis, M.A.: A perspective on human activity recognition from inertial motion data. Neural Computing and Applications (2023)
Gupta, G., Kshirsagar, M., Zhong, M., Gholami, S., Ferres, J.L.: Comparing recurrent convolutional neural networks for large scale bird species classification. Sci. Rep. (2021)
Izhikevich, E.M.: Dynamical systems in neuroscience. MIT press (2007)
Khan, N.S., Ghani, M.S.: A survey of deep learning based models for human activity recognition. Wireless Pers. Commun. (2021). https://doi.org/10.1007/s11277-021-08525-w
Kulsoom, F., Narejo, S., Mehmood, Z., Chaudhry, H.N., Butt, A., Bashir, A.K.: A review of machine learning-based human activity recognition for diverse applications. Neural Comput. Appl. 34 (2022)
Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter 12 (2011). https://doi.org/10.1145/1964897.1964918
Lara, O.D., Labrador, M.A.: A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutorials 15 (2013). https://doi.org/10.1109/SURV.2012.110112.00192
Liu, Z., Datta, G., Li, A., Beerel, P.A.: Lmuformer: low complexity yet powerful spiking model with legendre memory units. arXiv preprint arXiv:2402.04882 (2024)
Maass, W.: Networks of spiking neurons: The third generation of neural network models. Neural Networks 10 (1997).https://doi.org/10.1016/S0893-6080(97)00011-7
Mayr, C., Hoeppner, S., Furber, S.: Spinnaker 2: 10 million core processor system for brain simulation and machine learning. arXiv preprint arXiv:1911.02385 (2019)
Mekruksavanich, S., Jitpattanakul, A.: Deep convolutional neural network with RNNs for complex activity recognition using wrist-worn wearable sensor data. Electronics 10 (2021). https://doi.org/10.3390/electronics10141685
Miller, R.B.: Response time in man-computer conversational transactions. In: Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I (1968)
Müller-Cleve, S.F., et al.: Braille letter reading: a benchmark for spatio-temporal pattern recognition on neuromorphic hardware. Front. Neurosci. 16 (2022). https://doi.org/10.3389/fnins.2022.951164
Nweke, H.F., Teh, Y.W., Al-garadi, M.A., Alo, U.R.: Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Syst. Appl. 105 (2018). https://doi.org/10.1016/j.eswa.2018.03.056
Orchard, G., et al.: Efficient neuromorphic signal processing with Loihi 2. In: IEEE Workshop on Signal Processing Systems (SiPS), vol. 2021-Octob (2021). https://doi.org/10.1109/SiPS52927.2021.00053
Pedersen, J.E., et al.: Neuromorphic intermediate representation: a unified instruction set for interoperable brain-inspired computing. arXiv preprint arXiv:2311.14641 (2023)
Pelikan, H., Hofstetter, E.: Managing delays in human-robot interaction. ACM Trans. Comput.-Hum. Interact. (2023)
Popovski, P., et al.: A perspective on time toward wireless 6g. Proc. IEEE (2022)
Ramanujam, E., Perumal, T., Padmavathi, S.: Human activity recognition with smartphone and wearable sensors using deep learning techniques: a review. IEEE Sensors J. 21 (2021). https://doi.org/10.1109/JSEN.2021.3069927
Roy, K., Jaiswal, A., Panda, P.: Towards spike-based machine intelligence with neuromorphic computing. Nature 575 (2019). https://doi.org/10.1038/s41586-019-1677-2
Slim, S.O., Atia, A., M.A., M., M.Mostafa, M.S.: Survey on human activity recognition based on acceleration data. Int. J. Adv. Comput. Sci. Appl. 10 (2019). https://doi.org/10.14569/IJACSA.2019.0100311
Voelker, A., Kajić, I., Eliasmith, C.: Legendre memory units: continuous-time representation in recurrent neural networks. Advances in neural information processing systems 32 (2019)
Voelker, A.R., Eliasmith, C.: Programming neuromorphics using the neural engineering framework. In: Handbook of Neuroengineering (2020)
Weiss, G.M.: WISDM Smartphone and Smartwatch Activity and Biometrics Dataset. UCI Machine Learning Repository: WISDM Smartphone and Smartwatch Activity and Biometrics Dataset Data Set 7 (2019)
Weiss, G.M., Yoneda, K., Hayajneh, T.: Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access 7 (2019). https://doi.org/10.1109/ACCESS.2019.2940729
Yik, J., et al.: Neurobench: a framework for benchmarking neuromorphic computing algorithms and systems. arXiv preprint arXiv:2304.04640 (2024)
Acknowledgements
This research is funded by the European Union - NextGenerationEU Project 3A-ITALY MICS (PE0000004, CUP E13C22001900001, Spoke 6) and the Fluently project with Grant Agreement No. 101058680. We acknowledge a contribution from the Italian National Recovery and Resilience Plan (NRRP), M4C2, funded by the European Union - NextGenerationEU (Project IR0000011, CUP B51E22000150006, “EBRAINS-Italy”).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fra, V., Leto, B., Pignata, A., Macii, E., Urgese, G. (2024). Natively Neuromorphic LMU Architecture for Encoding-Free SNN-Based HAR on Commercial Edge Devices. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15025. Springer, Cham. https://doi.org/10.1007/978-3-031-72359-9_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-72359-9_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72358-2
Online ISBN: 978-3-031-72359-9
eBook Packages: Computer ScienceComputer Science (R0)