Abstract
Learning music has been demonstrated to provide many benefits for children. However, music students, especially beginners, often suffer from lack of motivation and even can be frustrated if their musical skills do not improve as they practice over and over. In such situations, they usually end up dropping out of music school. To face this challenge, in this work a novel approach based on mixed reality and gamification is proposed to motivate music students. This approach has been validated thanks to HoloMusic XP, a multimedia tool that helps students learn music and piano. The devised architecture that supports HoloMusic XP has been designed and developed to scale when new music concepts must be addressed. Thanks to the use of mixed reality, the usually steep learning curve for beginner students can be mitigated and complex music concepts can be simplified due to the use of visual metaphors. The system has been evaluated in a real environment by teachers and students to measure its effectiveness and usability. After conducting the experiments, an increase in the students’ motivation and a general understanding of the multimedia representation have been achieved.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akçayır M, Akçayır G (2017) Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ Res Rev 20:1–11
Cai M, Amrizal MA, Abe T, Suganuma T (2019) Design and implementation of ar-supported system for piano learning. In: 2019 IEEE 8th global conference on consumer electronics (GCCE). IEEE, pp 49–50
Chou YK (2013) Octalysis: complete gamification framework. Yu-Kai Chou & Gamification
Chow J, Feng H, Amor R, Wünsche BC (2013) Music education using augmented reality with a head mounted display. In: Proceedings of the fourteenth Australasian user interface conference, vol 139. Australian Computer Society, Inc., pp 73–79
Comport AI, Marchand E, Pressigout M, Chaumette F (2006) Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans Visual Comput Graph 12(4):615–628
Costa-Giomi E, Flowers PJ, Sasaki W (2005) Piano lessons of beginning students who persist or drop out: teacher behavior, student behavior, and lesson progress. J Res Music Educ 53(3):234–247
Costanza P, Russell T (2017) Methodologies in music education. In: Critical essays in music education. Routledge, pp 255–266
Dannenberg RB, Sanchez M, Joseph A, Capell P, Joseph R, Saul R (1990) An expert system for teaching piano to novices. In: ICMC
Davis FD (1993) User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. Int J Man-Machine Stud 38 (3):475–487
Degli Innocenti E, Geronazzo M, Vescovi D, Nordahl R, Serafin S, Ludovico LA, Avanzini F (2019) Mobile virtual reality for musical genre learning in primary education. Comput Educ 139:102–117
Fletcher C, Hulusic V, Amelidis P (2019) Virtual reality ear training system: a study on spatialised audio in interval recognition. In: 2019 11th International conference on virtual worlds and games for serious applications (VS-Games). IEEE, pp 1–4
Hackl D, Anthes C (2017) Holokeys-an augmented reality application for learning the piano. In: Forum media technology, pp 140–144
Huang F, Zhou Y, Yu Y, Wang Z, Du S (2011) Piano ar: a markerless augmented reality based piano teaching system. In: 2011 Third international conference on intelligent human-machine systems and cybernetics, vol 2. IEEE, pp 47–52
Kalantari M, Rauschnabel P (2018) Exploring the early adopters of augmented reality smart glasses: the case of microsoft hololens. In: Augmented reality and virtual reality. Springer, pp 229–245
Martin-Gutierrez J, Del Rio Guerra MS, Lopez-Chao V, Soto Gastelum RH, Valenzuela Bojórquez JF (2020) Augmented reality to facilitate learning of the acoustic guitar. Appl Sci 10(7):2425
McPherson GE, O’Neill SA (2010) Students’ motivation to study music as compared to other school subjects: a comparison of eight countries. Res Stud Music Educ 32(2):101–137
Molloy W, Huang E, Wünsche BC (2019) Mixed reality piano tutor: a gamified piano practice environment. In: 2019 International conference on electronics, information, and communication (ICEIC). IEEE, pp 1–7
Moog RA (1986) Midi: musical instrument digital interface. J Audio Eng Soc 34(5):394–404
Overy K (1998) Can music really “improve” the mind? Psychol Music 26(1):97–99
Pan Z, Cheok AD, Yang H, Zhu J, Shi J (2006) Virtual reality and mixed reality for virtual learning environments. Comput Graph 30(1):20–28
Pesek M, Vučko ž, Šavli P, Kavčič A, Marolt M (2020) Troubadour: a gamified e-learning platform for ear training. IEEE Access 8:97090–97102
Rio-Guerra D, Sylvia M, Martin-Gutierrez J, Lopez-Chao VA, Flores Parra R, Ramirez Sosa MA (2019) Ar graphic representation of musical notes for self-learning on guitar. Appl Sci 9(21):4527
Rusiñol M, Chazalon J, Diaz-Chito K (2018) Augmented songbook: an augmented reality educational application for raising music awareness. Multimed Tools Appl 77(11):13773–13798
Smoliar SW, Waterworth JA, Kellock PR (1995) Pianoforte: a system for piano education beyond notation literacy. In: ACM multimedia, vol 95
Walsh A (2009) Quick response codes and libraries. Library Hi Tech News 26(5/6):7–9
Yin J, Wang Y, Hsu D (2005) Digital violin tutor: an integrated system for beginning violin learners. In: Proceedings of the 13th annual ACM international conference on multimedia. ACM, pp 976–985
Zeng H, He X, Pan H (2019) Funpianoar: a novel ar application for piano learning considering paired play based on multi-marker tracking. J Phys: Conf Ser 1229:012072. IOP Publishing
Acknowledgements
This research was partially funded by the Department of Technologies and Information Systems (grant number 00421372) and by the University of Castilla-La Mancha (AIR Research Group, grant number 2020-GRIN-28886).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Molero, D., Schez-Sobrino, S., Vallejo, D. et al. A novel approach to learning music and piano based on mixed reality and gamification. Multimed Tools Appl 80, 165–186 (2021). https://doi.org/10.1007/s11042-020-09678-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09678-9