skip to main content
10.1145/3411764.3445352acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Facilitating Text Entry on Smartphones with QWERTY Keyboard for Users with Parkinson’s Disease

Published: 07 May 2021 Publication History

Abstract

QWERTY is the primary smartphone text input keyboard configuration. However, insertion and substitution errors caused by hand tremors, often experienced by users with Parkinson’s disease, can severely affect typing efficiency and user experience. In this paper, we investigated Parkinson’s users’ typing behavior on smartphones. In particular, we identified and compared the typing characteristics generated by users with and without Parkinson’s symptoms. We then proposed an elastic probabilistic model for input prediction. By incorporating both spatial and temporal features, this model generalized the classical statistical decoding algorithm to correct insertion, substitution and omission errors, while maintaining direct physical interpretation. User study results confirmed that the proposed algorithm outperformed baseline techniques: users reached 22.8 WPM typing speed with a significantly lower error rate and higher user-perceived performance and preference. We concluded that our method could effectively improve the text entry experience on smartphones for users with Parkinson’s disease.

Supplementary Material

VTT File (3411764.3445352_videofigurecaptions.vtt)
VTT File (3411764.3445352_videopreviewcaptions.vtt)
MP4 File (3411764.3445352_videofigure.mp4)
Supplemental video
MP4 File (3411764.3445352_videopreview.mp4)
Preview video

References

[1]
2016. American National Corpus. http://www.anc.org/.
[2]
Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. MobileHCI’12 - Proceedings of the 14th International Conference on Human Computer Interaction with Mobile Devices and Services (2012), 251–260. https://doi.org/10.1145/2371574.2371612
[3]
Mohammed Belatar and Franck Poirier. 2008. Text Entry for Mobile Devices and Users with Severe Motor Impairments: Handiglyph, a Primitive Shapes Based Onscreen Keyboard. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility(Assets ’08). Association for Computing Machinery, New York, NY, USA, 209–216. https://doi.org/10.1145/1414471.1414510
[4]
Leah Findlater and Jacob O. Wobbrock. 2012. Personalized input: Improving ten-finger touchscreen typing through automatic adaptation. In Conference on Human Factors in Computing Systems - Proceedings. 815–824. https://doi.org/10.1145/2207676.2208520
[5]
Mayank Goel, Leah Findlater, and Jacob Wobbrock. 2012. WalkType: Using Accelerometer Data to Accomodate Situational Impairments in Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 2687–2696. https://doi.org/10.1145/2207676.2208662
[6]
Mayank Goel, Alex Jansen, Travis Mandel, Shwetak N. Patel, and Jacob O. Wobbrock. 2013. ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 2795–2798. https://doi.org/10.1145/2470654.2481386
[7]
Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language Modeling for Soft Keyboards. In Proceedings of the 7th International Conference on Intelligent User Interfaces (San Francisco, California, USA) (IUI ’02). Association for Computing Machinery, New York, NY, USA, 194–195. https://doi.org/10.1145/502716.502753
[8]
Toshiyuki Hagiya, Toshiharu Horiuchi, and Tomonori Yazaki. 2016. Typing tutor: Individualized tutoring in text entry for older adults based on input stumble detection. Conference on Human Factors in Computing Systems - Proceedings (2016), 733–744. https://doi.org/10.1145/2858036.2858455
[9]
Toshiyuki Hagiya, Toshiharu Horiuchi, Tomonori Yazaki, Tsuneo Kato, and Tatsuya Kawahara. 2017. Assistive typing application for older adults based on input stumble detection. Journal of Information Processing 25, June (2017), 417–425. https://doi.org/10.2197/ipsjjip.25.417
[10]
Christian Holz and Patrick Baudisch. 2011. Understanding Touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 2501–2510. https://doi.org/10.1145/1978942.1979308
[11]
Farzana Jabeen, Linmi Tao, Lin Tong, and Shanshan Mei. 2019. Modeling Chinese Input Interaction for Patients with Cloud Based Learning. (2019), 1–8. https://doi.org/10.1109/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00206
[12]
Shaun K. Kane, Jacob O. Wobbrock, Mark Harniss, and Kurt L. Johnson. 2008. TrueKeys: Identifying and correcting typing errors for people with motor impairments. In International Conference on Intelligent User Interfaces, Proceedings IUI. 349–352. https://doi.org/10.1145/1378773.1378827
[13]
Masatomo Kobayashi, Atsushi Hiyama, Takahiro Miura, Chieko Asakawa, Michitaka Hirose, and Tohru Ifukube. 2011. Elderly User Evaluation of Mobile Touchscreen Interactions. In Human-Computer Interaction – INTERACT 2011, Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer Berlin Heidelberg, 83–99.
[14]
Per-Ola Kristensson and Shumin Zhai. 2004. SHARK 2: a large vocabulary shorthand writing system for pen-based computers. In Proc. UIST’04. ACM, 43–52.
[15]
Per Ola Kristensson and Shumin Zhai. 2005. Relaxing stylus typing precision by geometric pattern matching. International Conference on Intelligent User Interfaces, Proceedings IUI (2005), 151–158. https://doi.org/10.1145/1040830.1040867
[16]
Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. 707–710.
[17]
Yu Hao Lin, Suwen Zhu, Yu Jung Ko, Wenzhe Cui, and Xiaojun Bi. 2018. Why is gesture typing promising for older adults? comparing gesture and tap typing behavior of older with young adults. In ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. 271–281. https://doi.org/10.1145/3234695.3236350
[18]
[18] I. Scott MacKenzie.2015. https://www.yorku.ca/mack/RN-TextEntrySpeed.html
[19]
I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In Conference on Human Factors in Computing Systems - Proceedings(CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 754–755. https://doi.org/10.1145/765891.765971
[20]
Kyle Montague, Hugo Nicolau, and Vicki L Hanson. 2014. Motor-Impaired Touchscreen Interactions in the Wild. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility(ASSETS ’14). Association for Computing Machinery, New York, NY, USA, 123–130. https://doi.org/10.1145/2661334.2661362
[21]
Lilian Genaro Motti, Nadine Vigouroux, and Philippe Gorce. 2013. Interaction techniques for older adults using touchscreen devices: A literature review. IHM 2013 - Actes de la 25ieme Conference Francophone sur l’Interaction Homme-Machine (2013), 125–134. https://doi.org/10.1145/2534903.2534920
[22]
Maia Naftali and Leah Findlater. 2014. Accessibility in Context: Understanding the Truly Mobile Experience of Smartphone Users with Motor Impairments. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility(ASSETS ’14). Association for Computing Machinery, New York, NY, USA, 209–216. https://doi.org/10.1145/2661334.2661372
[23]
Hugo Nicolau and Joaquim Jorge. 2012. Elderly text-entry performance on touchscreens. In ASSETS’12 - Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility. 127–134. https://doi.org/10.1145/2384916.2384939
[24]
Francisco Nunes, Paula Alexandra Silva, João Cevada, Ana Correia Barros, and Luís Teixeira. 2016. User interface design guidelines for smartphone applications for people with Parkinson’s disease. Universal Access in the Information Society 15, 4 (2016), 659–679. https://doi.org/10.1007/s10209-015-0440-1
[25]
Katrin Plaumann, Milos Babic, Tobias Drey, Witali Hepting, Daniel Stooss, and Enrico Rukzio. 2018. Improving Input Accuracy on Smartphones for Persons who are Affected by Tremor using Motion Sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1–30. https://doi.org/10.1145/3161169
[26]
Ondrej Polacek, Adam J Sporka, and Pavel Slavik. 2017. Text input for motor-impaired people. Universal Access in the Information Society 16, 1 (2017), 51–72. https://doi.org/10.1007/s10209-015-0433-0
[27]
Élvio Rodrigues, Micael Carreira, and Daniel Gonçalves. 2014. Improving text-entry experience for older adults on tablets. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8515 LNCS, PART 3(2014), 167–178. https://doi.org/10.1007/978-3-319-07446-7_17
[28]
Élvio Rodrigues, Micael Carreira, and Daniel Gonçalves. 2016. Enhancing typing performance of older adults on tablets. Universal Access in the Information Society 15, 3 (2016), 393–418. https://doi.org/10.1007/s10209-014-0394-8
[29]
Sayan Sarcar, Jussi Jokinen, Antti Oulasvirta, Xiangshi Ren, Chaklam Silpasuwanchai, and Zhenxin Wang. 2017. Ability-Based Optimization: Designing Smartphone Text Entry Interface for Older Adults, Regina Bernhaupt, Girish Dalvi, Anirudha Joshi, Devanuj K. Balkrishan, Jacki O’Neill, and Marco Winckler (Eds.). Springer International Publishing, Cham, 326–331.
[30]
S Sarcar, J P P Jokinen, A Oulasvirta, Z Wang, C Silpasuwanchai, and X Ren. 2018. Ability-Based Optimization of Touchscreen Interactions. IEEE Pervasive Computing 17, 1 (2018), 15–26. https://doi.org/10.1109/MPRV.2018.011591058
[31]
Weinan Shi, Chun Yu, Shuyi Fan, Feng Wang, Tong Wang, Xin Yi, Xiaojun Bi, and Yuanchun Shi. 2019. Vipboard: Improving screen-reader keyboard for visually impaired people with character-level auto correction. In Conference on Human Factors in Computing Systems - Proceedings. 1–12. https://doi.org/10.1145/3290605.3300747
[32]
Weinan Shi, Chun Yu, Xin Yi, Zhen Li, and Yuanchun Shi. 2018. TOAST: Ten-Finger Eyes-Free Typing on Touchable Surfaces. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1–23. https://doi.org/10.1145/3191765
[33]
R. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for text entry research: An evaluation of MSD and KSPC, and a new unified error metric. In Conference on Human Factors in Computing Systems - Proceedings. 113–120.
[34]
Shari Trewin. 2003. Automating Accessibility: The Dynamic Keyboard. SIGACCESS Access. Comput.77–78 (2003), 71–78. https://doi.org/10.1145/1029014.1028644
[35]
Shari Trewin, Cal Swart, and Donna Pettick. 2013. Physical Accessibility of Touchscreen Smartphones. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’13). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2513383.2513446
[36]
Radu-Daniel Vatavu and Ovidiu-Ciprian Ungurean. 2019. Stroke-Gesture Input for People with Motor Impairments: Empirical Results & Research Roadmap. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300445
[37]
Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating Fast Mobile Text Entry Using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 659–668. https://doi.org/10.1145/2702123.2702135
[38]
Chat Wacharamanotham, Jan Hurtmanns, Alexander Mertens, Martin Kronenbuerger, Christopher Schlick, and Jan Borchers. 2011. Evaluating Swabbing: A Touchscreen Input Method for Elderly Users with Tremor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). Association for Computing Machinery, New York, NY, USA, 623–626. https://doi.org/10.1145/1978942.1979031
[39]
Yuntao Wang, Chun Yu, Jie Liu, and Yuanchun Shi. 2013. Understanding Performance of Eyes-Free, Absolute Position Control on Touchable Mobile Phones. In Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (Munich, Germany) (MobileHCI ’13). Association for Computing Machinery, New York, NY, USA, 79–88. https://doi.org/10.1145/2493190.2493215
[40]
Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain text entry on mobile devices. In Conference on Human Factors in Computing Systems - Proceedings. 2307–2316. https://doi.org/10.1145/2556288.2557412
[41]
Daryl Weir, Simon Rogers, Roderick Murray-Smith, and Markus Löchtefeld. 2012. A user-specific Machine Learning approach for improving touch accuracy on mobile devices. In UIST’12 - Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. 465–475. https://doi.org/10.1145/2380116.2380175
[42]
Wikipedia contributors. 2020. Parkinson’s disease — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Parkinson%27s_disease&oldid=970177846 [Online; accessed 31-July-2020].
[43]
Jacob O Wobbrock, Brad A Myers, and John A Kembel. 2003. EdgeWrite: A Stylus-Based Text Entry Method Designed for High Accuracy and Stability of Motion. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology(UIST ’03). Association for Computing Machinery, New York, NY, USA, 61–70. https://doi.org/10.1145/964696.964703
[44]
Xin Yi, Chun Yu, Weinan Shi, and Yuanchun Shi. 2017. Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards. International Journal of Human Computer Studies 106, April(2017), 44–62. https://doi.org/10.1016/j.ijhcs.2017.05.001
[45]
Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: Reimagining keyboard interaction. Commun. ACM 55, 9 (2012), 91–101. https://doi.org/10.1145/2330667.2330689
[46]
Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: reimagining keyboard interaction. Commun. ACM 55, 9 (2012), 91–101.
[47]
Yu Zhong, Astrid Weber, Casey Burkhardt, Phil Weaver, and Jeffrey P. Bigham. 2015. Enhancing android accessibility for users with hand tremor by reducing fine pointing and steady tapping. In W4A 2015 - 12th Web for All Conference. 1–10. https://doi.org/10.1145/2745555.2747277
[48]
Suwen Zhu, Tianyao Luo, Xiaojun Bi, and Shumin Zhai. 2018. Typing on an invisible keyboard. In Conference on Human Factors in Computing Systems - Proceedings(CHI ’18, Vol. 2018-April). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174013

Cited By

View all
  • (2024)CRTypist: Simulating Touchscreen Typing Behavior via Computational RationalityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642918(1-17)Online publication date: 11-May-2024
  • (2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.232385840:8(1799-1831)Online publication date: 22-Mar-2024
  • (2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
  • Show More Cited By

Index Terms

  1. Facilitating Text Entry on Smartphones with QWERTY Keyboard for Users with Parkinson’s Disease
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
    May 2021
    10862 pages
    ISBN:9781450380966
    DOI:10.1145/3411764
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 May 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Parkinson’s disease
    2. QWERTY keyboard
    3. statistical decoding
    4. text entry
    5. touch model

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • Natural Science Foundation of China
    • National Key R&D Program of China

    Conference

    CHI '21
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)98
    • Downloads (Last 6 weeks)8
    Reflects downloads up to 21 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)CRTypist: Simulating Touchscreen Typing Behavior via Computational RationalityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642918(1-17)Online publication date: 11-May-2024
    • (2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.232385840:8(1799-1831)Online publication date: 22-Mar-2024
    • (2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
    • (2023)Imbalanced ensemble learning in determining Parkinson’s disease using Keystroke dynamicsExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.119522217:COnline publication date: 1-May-2023
    • (2022)Diagnostic accuracy of keystroke dynamics as digital biomarkers for fine motor decline in neuropsychiatric disorders: a systematic review and meta-analysisScientific Reports10.1038/s41598-022-11865-712:1Online publication date: 11-May-2022
    • (2022)Using Keytyping as a Biomarker for Cognitive Decline Diagnostics: The Convolutional Neural Network Based ApproachPattern Recognition and Artificial Intelligence10.1007/978-3-031-04112-9_28(367-381)Online publication date: 13-Apr-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy