Abstract
In this paper, the factors of positive user experiences when using AI systems are investigated. For this purpose, a two-stage qualitative usability study was conducted for the OMA-ML platform as an example. OMA-ML is an AutoML platform that automates complex tasks in machine learning (ML) and generates ML pipelines. The usability of OMA-ML was measured against the ISO 9241-110:2020 standard in an expert evaluation. The vulnerabilities with the greatest impact on the application were prioritised and tested in a qualitative usability test. The results of the usability test are presented along with recommendations in a usability evaluation. This study aims to contribute to the understanding of the usability of AI systems and their impact on the experience of the different user groups. It found that special attention needs to be paid to those interaction principles that serve to build user trust towards the AI system. For this purpose, the interaction principles with the main design dimensions for interaction with AI systems were derived.
This work was partially funded by the German federal ministry of education and research (BMBF) in the program Zukunft der Wertschöpfung (funding code 02L19C157), supported by Projektträger Karlsruhe (PTKA), and by hessian.AI Connectom for project “Innovative UX für User-Centered AI Systeme”. The responsibility for the content of this publication lies with the authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In this paper, we use the term AI systems as a general term for AI-based IT systems, including AI applications for end users (e.g., medical doctors) as well as AI platforms for AI specialists (e.g., ML experts developing end-user AI applications).
- 2.
The participants were required to sign a declaration of consent to have their data collected. The signed consent forms can be obtained on request. All participants were given the right to revoke the consent form and withdraw from the data collection.
References
Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 4th edn. Pearson Series in Artificial Intelligence. Pearson, Upper Saddle River (2021). ISBN 9780134610993
Chapman, S., Fry, A., Deschenes, A., McDonald, C.G.: Strategies to improve the user experience. Serials Review (2016). https://doi.org/10.1080/00987913.2016.1140614. Accessed 8 Mar 2023
Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction. Morgan Kaufmann, Burlington (2017)
Humm, B.G., Zender, A.: An ontology-based concept for meta AutoML. In: Maglogiannis, I., Macintyre, J., Iliadis, L. (eds.) Artificial Intelligence Applications and Innovations, ser. Springer eBook Collection, vol. 627, pp. 117–128. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79150-6_10
Zender, A., Humm, B.G.: Ontology-based Meta AutoML. Integr. Comput. Aided Eng. 29(4), 351–366 (2022)
Lundberg, S., Lee, S.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30, pp. 4765–4774. Curran Associates, Inc. (2017)
Alcácer, V., Cruz-Machado, V.: Scanning the Industry 4.0: a literature review on technologies for manufacturing systems. Eng. Sci. Technol. Int. J. 22(3), 899–919 (2019). https://doi.org/10.1016/j.jestch.2019.01.006
Bonaccorso, G.: Machine Learning Algorithms. Packt Publishing Ltd., Berkeley (2017)
Lu, Y.: Industry 4.0: a survey on technologies, applications and open research issues. J. Ind. Inf. Integr. 6, 1–10 (2017). https://doi.org/10.1016/j.jii.2017.04.005
Moustakis, V.S., Herrmann, J.: Where do machine learning and human-computer interaction meet? Appl. Artif. Intell. 11(7–8), 595–609 (1997). https://doi.org/10.1080/088395197117948
Chapman, S., Fry, A., Deschenes, A., McDonald, C.G.: Strategies to improve the user experience. Serials Review (2016). https://doi.org/10.1080/00987913.2016.1140614
Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., Kankanhalli, M.: Trends and trajectories for explainable, accountable and intelligible systems: an HCI research agenda. In: Conference on Human Factors in Computing Systems - Proceedings (2018). https://doi.org/10.1145/3173574.3174156
Wang, D., Yang, Q., Abdul, A., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: Conference on Human Factors in Computing Systems – Proceedings (2019). https://doi.org/10.1145/3290605.3300831
Chromik, M., Lachner, F., Butz, A.: Ml for UX? An inventory and predictions on the use of machine learning techniques for UX research. In: Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, pp. 1–11 (2020)
Fucs, A., Ferreira, J.J., Segura, V., De Paulo, B., de Paula, R., Cerqueira, R.: Sketch-based video; Storytelling for UX validation in AI design for applied research. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–8 (2020)
Li, F., Lu, Y.: Engaging end users in an AI-enabled smart service design-the application of the smart service blueprint scape (SSBS) framework. Proc. Des. Soc. 1, 1363–1372 (2021)
Kurdziolek, M.: Explaining the Unexplainable: Explainable AI (XAI) for UX. User Experience - The Magazine of the User Experience Professionals Association. https://uxpamagazine.org/explaining-the-unexplainable-explainable-ai-xai-for-ux. Accessed 8 Mar 2023
Nielsen, J.: Usability 101: Introduction to usability. Nielsen Norman Group (2012). https://www.nngroup.com/articles/usability-101-introduction-to-usability/. Accessed 2022
International Organization for Standardization (ISO): ISO 9241-210: Ergonomics of Human–System Interaction - Human-Centred Design for Interactive Systems. International Organization for Standardization, Geneva (2019)
Oliver, R.L.: Effect of expectation and disconfirmation on postexposure product evaluations: an alternative interpretation. J. Appl. Psychol. 62, 480–486 (1977). https://doi.org/10.1037/0021-9010.62.4.480
Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology: toward a unified view. MIS Q. 27(3), 425–478 (2003). https://doi.org/10.2307/30036540
Shneiderman, B.: Human-centered artificial intelligence: reliable, safe and trustworthy. Int. J. Hum.-Comput. Interact. 36(6), 495–504 (2020). https://doi.org/10.1080/10447318.2020.1741118
Bittner, K., Spence, I.: Use Case Modeling. Addison-Wesley Professional, Boston (2003)
International Organization for Standardization (ISO) (2020). ISO 9241-110:2020 Ergonomics of human-system interaction — Part 110: Interaction principles. https://www.iso.org/obp/ui/#iso:std:iso:9241:-110:ed-2:v1:en
Merriam, S.B.: Qualitative Research and Case Study Applications in Education. Revised and Expanded from “Case Study Research in Education”, p. 94104. Jossey-Bass Publishers, San Francisco (1998)
Dutke, S.: Mentale Modelle - Konstrukte des Wissens und Verstehens: Kognitionspschologische Grundlagen für die Software-Ergonomie (1994)
Creswell, J.W., Creswell, J.D.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications, New York (2017)
Moustakas, C.: Phenomenological Research Methods. Sage Publications, New York (1994)
Richter, M., Flückiger, M.D., Richter, M., Flückiger, M.D.: Die 7 ± 2 wichtigsten usability-methoden. Usability Engineering kompakt: Benutzbare Software gezielt entwickeln, pp. 21–76 (2010)
Oppong, S.H.: The problem of sampling in qualitative research. Asian J. Manage. Sci. Educ. 2(2), 202–210 (2013)
Sedgwick, P., Greenwood, N.: Understanding the Hawthorne effect. BMJ 351, h4672 (2015)
Nagappan, R.: Dealing with biases in qualitative research: a balancing act for researchers (2001)
Stiles, W.B.: Quality control in qualitative research. Clin. Psychol. Rev. 13(6), 593–618 (1993)
Harley. UX Expert Reviews. Nielsen Norman Group (2018). https://www.nngroup.com/articles/ux-expert-reviews/
Nielsen, J.: Usability Engineering. Morgan Kaufmann, New York (1994)
Nielsen, J.: Ten usability heuristics (2005). http://www.nngroup.com/articles/ten-usability-heuristics/
Nielsen, J.: Severity Ratings for Usability Problems: Article by Jakob Nielsen. Nielsen Norman Group (1994). https://www.nngroup.com/articles/ten-usability-heuristics
Botsman, R.: Who Can You Trust? How Technology Brought Us Together - and Why It Could Drive Us Apart, p. 179. Penguin Books Ltd. Kindle Version (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Brand, L., Humm, B.G., Krajewski, A., Zender, A. (2023). Towards Improved User Experience for Artificial Intelligence Systems. In: Iliadis, L., Maglogiannis, I., Alonso, S., Jayne, C., Pimenidis, E. (eds) Engineering Applications of Neural Networks. EANN 2023. Communications in Computer and Information Science, vol 1826. Springer, Cham. https://doi.org/10.1007/978-3-031-34204-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-34204-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34203-5
Online ISBN: 978-3-031-34204-2
eBook Packages: Computer ScienceComputer Science (R0)