Abstract
Facial expression analysis is a widespread technology applied in various research areas, including sports science. In the last few decades, facial expression analysis has become a key technology for monitoring physical exercise. In this paper, a deep neural network is proposed to recognize seven basic emotions and their corresponding probability values (scores). The score of the neutral emotion was tracked throughout the exercise and related with heart rate and power generation by a stationary bicycle. It was found that in a certain power range, a participant changes his/her expression drastically. Twelve university students participated in the sub-maximal physical exercise in stationary bicycles. A facial video, heart rate,and power generation were recorded throughout the exercise. All the experiments, including the facial expression analysis, were carried out offline. The score of the neutral emotion and its derivative was plotted against maxHR% and maxPower%. The threshold point was determined by calculating the local minima, with the threshold power for all the participants being within 80% to 90% of its maximum value. From the results, it is concluded that the facial expression was different from one individual to another, but it was more consistant with power generation. The threshold point can be a useful cue for various purposes, such as: physiological parameter prediction and automatic load control in the exercise equipment, such as treadmill and stationary bicycle.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Reed, J.L., Pipe, A.L.: Practical approaches to prescribing physical activity and monitoring exercise intensity. Can. J. Cardiol. 32(4), 514–522 (2016)
Schuch, F.B., et al.: Exercise as a treatment for depression: a meta-analysis adjusting for publication bias, pp. 1879–1379 Electronic (2016)
Das, P., Horton, R.: Rethinking our approach to physical activity. Lancet (London, England) 380(9838), 189–190 (2012)
Bayles, M.P., Swank, A.M.: ACSM’s exercise testing and prescription. American College of Sports, Medicine (2018)
Hunt, K.J., Fankhauser, S.E.: Heart rate control during treadmill exercise using input-sensitivity shaping for disturbance rejection of very-low-frequency heart rate variability. Biomed. Sign. Process. Control 30, 31–42 (2016)
Xie, K., et al.: Non-contact heart rate monitoring for intensive exercise based on singular spectrum analysis. In: 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) (2019)
Aboagye, E., et al.: Individual preferences for physical exercise as secondary prevention for non-specific low back pain: a discrete choice experiment. PLoS ONE 12(12), e0187709–e0187709 (2017)
Uchida, M.C., et al.: Identification of muscle fatigue by tracking facial expressions. PLoS ONE 13(12), e0208834 (2018)
Khanal, S.R., et al.: Classification of physical exercise intensity by using facial expression analysis. In: 2018 Second International Conference on Computing Methodologies and Communication (ICCMC) (2018). https://doi.org/10.1109/ICCMC.2018.8488080
Ekman, P.: Basic emotions. Handb. Cogn. Emot. 98(45–60), 16 (1999)
Khanal, S.R., Sampaio, J., Barroso, J., Filipe, V.: Classification of physical exercise intensity based on facial expression using deep neural network. In: Antona, M., Stephanidis, C. (eds.) HCII 2019. LNCS, vol. 11573, pp. 455–467. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23563-5_36
Miles, K.H., et al.: Facial feature tracking: a psychophysiological measure to assess exercise intensity? 1466–447X Electronic
Khanal, S., et al.: Using emotion recognition in intelligent interface design for elderly care. In: Trends and Advances in Information Systems and Technologies. Cham: Springer International Publishing (2018). https://doi.org/10.1007/978-3-319-77712-2_23
Khanal, S.R., et al.: Physical exercise intensity monitoring through eye-blink and mouth’s shape analysis. In: 2018 2nd International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW) (2018). https://doi.org/10.1109/TISHW.2018.8559556
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 (2001)
Orieux, F., Giovannelli, J.F., Rodet, T.: Bayesian estimation of regularization and point spread function parameters for Wiener-Hunt deconvolution. J. Opt. Soc. Am. 27(7), 1593–1607 (2010)
Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: International Conference on Neural Information Processing. Springer (2013)
Jonsdottir, J., et al.: Intensive multimodal training to improve gait resistance, mobility, balance and cognitive function in persons with multiple sclerosis: a pilot randomized controlled trial. Front. Neurol. 9, 800 (2018)
Chowdhury, A.K., et al.: Prediction of relative physical activity intensity using multimodal sensing of physiological data. Sens. (Basel, Switzerland) 19(20), 4509 (2019)
Vieluf, S., et al.: Exercise-induced changes of multimodal interactions within the autonomic nervous network. Front. Physiol. 10, 240 (2019)
Jang, E.H., et al.: Analysis of physiological signals for recognition of boredom, pain, and surprise emotions. J. Physiol. Anthropol. 34(1), 25–25 (2015)
Hwang, R.J., et al.: Physical activity and neural correlates of sad facial expressions in premenstrual syndrome. J. Gynecol. Obstet. 6(3), 56–66 (2018)
Critchley, H.D., et al.: Activity in the human brain predicting differential heart rate responses to emotional facial expressions. NeuroImage 24(3), 751–762 (2005)
Pezzulo, G., et al.: Increased heart rate after exercise facilitates the processing of fearful but not disgusted faces. Sci. Rep. 8(1), 398 (2018)
Acknowledgements
This work was supported by Portuguese FCT – Foundation for Science and Technology - project UID0445/2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Khanal, S.R., Sampaio, J., Barroso, J., Filipe, V. (2020). Individual’s Neutral Emotional Expression Tracking for Physical Exercise Monitoring. In: Stephanidis, C., Kurosu, M., Degen, H., Reinerman-Jones, L. (eds) HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence. HCII 2020. Lecture Notes in Computer Science(), vol 12424. Springer, Cham. https://doi.org/10.1007/978-3-030-60117-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-60117-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60116-4
Online ISBN: 978-3-030-60117-1
eBook Packages: Computer ScienceComputer Science (R0)