Skip to main content
Springer Nature Link
Log in

PyFlowML: A Visual Language Framework to Foster Participation in ML-Based Decision Making

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14734))

Included in the following conference series:

  • 1006 Accesses

Abstract

Nowadays, Artificial Intelligence (AI) has become ubiquitous, with machine learning (ML) systems – rooted in algorithms and statistical models – playing a pivotal role in societal advancement. Nevertheless, given their complexity, less-experienced computing practitioners depend on skilled professionals. This paper addresses the challenge of democratizing AI, emphasizing the need for inclusivity in ML-based system design. We introduce the PyFlowML prototype, which extends the PyFlow framework, to investigate how Visual Programming Languages (VPLs) and no-code platforms can enhance user engagement in ML. PyFlowML tailors visual scripting capabilities to meet the specific needs and complexities inherent in ML analysis. This preliminary study is based on a heuristic evaluation of PyFlowML’s usability: we analyzed expert interactions with the tool, exploring its features to design trustworthy ML-based prototypes using Explainable AI (XAI) techniques. We employed the cognitive walk-through method, wherein experts engaged in a series of activities with PyFlowML while sharing their thoughts in real time during the session. While initial findings are promising, they also indicate that to effectively lower the entry barrier to ML-based system design and encourage broader participation, it is crucial to implement strategies that reduce the inherent complexity of ML analysis. This research sets the groundwork for future exploration into how VPL-based tools can transform the design of ML-based systems, aiming for more inclusive and collaborative AI development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Article Open access 28 November 2023

Notes

  1. 1.

    https://dynamobim.org.

  2. 2.

    https://github.com/google/blockly.

  3. 3.

    https://knowledge.dataiku.com/latest/ml-analytics/index.html.

  4. 4.

    https://www.sas.com/en_us/home.html.

  5. 5.

    https://enso.org.

  6. 6.

    https://rivet.ironcladapp.com.

  7. 7.

    https://www.knime.com.

  8. 8.

    https://rapidminer.com.

  9. 9.

    https://www.ni.com/it-it/shop/product/labview-analytics-and-machine-learning-toolkit.html.

  10. 10.

    https://teachablemachine.withgoogle.com/train/image.

  11. 11.

    https://pyflow.readthedocs.io/en/latest/intro.html.

  12. 12.

    https://www.kaggle.com/datasets/pavansubhasht/ibm-hr-analytics-attrition-dataset.

References

  1. Makridakis, S.: The forthcoming artificial intelligence (ai) revolution: its impact on society and firms. In: Futures, vol. 90, pp. 46–60 (2017), https://www.sciencedirect.com/science/article/pii/S0016328717300046

  2. Brynjolfsson, E., McAfee, A.: The second machine age: work, progress, and prosperity in a time of brilliant technologies. Book published by WW Norton & Company (2014)

    Google Scholar 

  3. Baidoo-Anu, D., Owusu Ansah, L.: Education in the era of generative artificial intelligence (ai): understanding the potential benefits of chatgpt in promoting teaching and learning. J. AI 7(1), 52–62 (2023). https://doi.org/10.61969/jai.1337500

  4. Mansoury, M., Abdollahpouri, H., Pechenizkiy, M., Mobasher, B., Burke, R.: Feedback loop and bias amplification in recommender systems. In: arXiv (2020). https://doi.org/10.48550/ARXIV.2007.13019

  5. Milano, S., Taddeo, M., Floridi, L.: Recommender systems and their ethical challenges. AI & Soc. 35(4), 957–967 (2020). https://doi.org/10.1007/s00146-020-00950-y

  6. Makhortykh, M., Urman, A., Ulloa, R.: Detecting race and gender bias in visual representation of AI on web search engines. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds.) BIAS 2021. CCIS, vol. 1418, pp. 36–50. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78818-6_5

    Chapter  Google Scholar 

  7. Shneiderman, B., Plaisant, C., Cohen, M., Jacobs, S., Elmqvist, N., Diakopoulos, N.: Grand challenges for hci researchers. Interactions 23(5), 24–25 (2016). https://doi.org/10.1145/2977645

  8. Harari, Y.N.: Why technology favors tyranny. The Atlantic 322(3), 64–73 (2018)

    Google Scholar 

  9. Council, N.R.: Beyond productivity: Information technology, innovation, and creativity. Book published by National Academies Press (2003)

    Google Scholar 

  10. Shneiderman, B.: Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy human-centered ai systems. ACM Trans. Interactive Intell. Syst. (TiiS) 10(4), 1–31 (2020)

    Google Scholar 

  11. Fischer, G.: End-user development: empowering stakeholders with artificial intelligence, meta-design, and cultures of participation. In: Fogli, D., Tetteroo, D., Barricelli, B.R., Borsci, S., Markopoulos, P., Papadopoulos, G.A. (eds.) IS-EUD 2021. LNCS, vol. 12724, pp. 3–16. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79840-6_1

    Chapter  Google Scholar 

  12. Lieberman, H., Paternò, F., Klann, M., Wulf, V.: End-user development: an emerging paradigm. In: End User Development, pp. 1–8. Springer (2006)

    Google Scholar 

  13. Cabitza, F., Fogli, D., Lanzilotti, R., Piccinno, A.: Rule-based tools for the configuration of ambient intelligence systems: a comparative user study. In: Multimedia Tools and Applications, vol. 76, pp. 5221–5241, 2017 (2017)

    Google Scholar 

  14. Gallo, S., Paterno, F., Malizia, A.: Conversational interfaces in iot ecosystems: where we are, what is still missing. In: Proceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia, MUM 2023, pp. 279–293, Vienna, Austria (2023). https://doi.org/10.1145/3626705.3627775

  15. Danado, J., Paternò, F.: Puzzle: a visual-based environment for end user development in touch-based mobile phones. In: Winckler, M., Forbrig, P., Bernhaupt, R. (eds.) HCSE 2012. LNCS, vol. 7623, pp. 199–216. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34347-6_12

    Chapter  Google Scholar 

  16. Fischer, G.: Understanding, fostering, and supporting cultures of participation. Interactions 18(3), 42–53 (2011)

    Google Scholar 

  17. Fischer, G., Fogli, D., Mørch, A., Piccinno, A., Valtolina, S.: Design trade-offs in cultures of participation: Empowering end users to improve their quality of life. In: Behav. Inf. Technol. 39(1), 1–4 (2020). https://doi.org/10.1080/0144929X.2020.1691346

  18. Paternò, F., Wulf, V.: New perspectives in end-user development. Book published by Springer (2017)

    Google Scholar 

  19. Halfaker, A., Geiger, R.S.: Ores: lowering barriers with participatory machine learning in wikipedia. In: Proc. ACM Hum.-Comput. Interact., Volume 4, CSCW2, Article 148, pp. 1–37, October 2020. https://doi.org/10.1145/3415219

  20. Shneiderman, B.: Human-centered ai. Book published by Oxford University Press (2022)

    Google Scholar 

  21. Dasgupta, S., Hill, B.M.: Scratch community blocks: supporting children as data scientists. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI 2017, pp. 3620–3631, Denver, Colorado, USA (2017). https://doi.org/10.1145/3025453.3025847

  22. Green, T.R.G., Petre, M.: Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. J. Vis. Lang. Comput. 7(2), 131–174 (1996). https://www.sciencedirect.com/science/article/pii/S1045926X96900099

  23. Mason, D., Dave, K.: Block-based versus flow-based programming for naive programmers. In: 2017 IEEE Blocks and Beyond Workshop, pp. 25–28 (2017)

    Google Scholar 

  24. Burnett, M.M., McIntyre, D.W.: Visual programming. In: Computer-Los Alamitos, vol. 28, p. 14. IEEE Institute of Electrical and Electronics, 1995 (1995)

    Google Scholar 

  25. Paternò, F.: End user development: survey of an emerging field for empowering people. In: International Scholarly Research Notices, vol. 2013, Hindawi (2013)

    Google Scholar 

  26. Kuhail, M.A., Farooq, S., Hammad, R., Bahja, M.: Characterizing visual programming approaches for end-user developers: a systematic review. IEEE Access 9, 14181–14202 (2021)

    Google Scholar 

  27. Myers, B.A.: Taxonomies of visual programming and program visualization. J. Vis. Lang. Comput. 1(1), 97–123 (1990)

    Google Scholar 

  28. Burnett, M.M., Baker, M.J.: A classification system for visual programming languages. J. Vis. Lang. Comput. 5(3), 287–300 (1994)

    Google Scholar 

  29. Turchi, T., Malizia, A.: Fostering computational thinking skills with a tangible blocks programming environment. In: 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pp. 232–233 (2016)

    Google Scholar 

  30. Turchi, T., Malizia, A., Dix, A.: Tapas: a tangible end-user development tool supporting the repurposing of pervasive displays. J. Vis. Lang. Comput. 39, 66–77 (2017)

    Google Scholar 

  31. Ardito, C., et al.: User-driven visual composition of service-based interactive spaces. J. Vis. Lang. Comput. 25(4), 278–296 (2014). https://www.sciencedirect.com/science/article/pii/S1045926X14000299

  32. Burnett, M.M., Ambler, A.L.: Interactive visual data abstraction in a declarative visual programming language. J. Visual Lang. Comput. 5(1), pp. 29–60 (1994). https://www.sciencedirect.com/science/article/pii/S1045926X84710032

  33. Hils, D.D.: Visual languages and computing survey: Data flow visual programming languages. J. Visual Lang. Comput. 3(1), 69–101 (1992). https://www.sciencedirect.com/science/article/pii/1045926X9290034J

  34. Alexandrova, S., Tatlock, Z., Cakmak, M.: Roboflow: A flow-based visual programming language for mobile manipulation tasks. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), Pages 5537-5544 (2015)

    Google Scholar 

  35. McNeel, B., Davidson, S.: Grasshopper. Online resource available at http://www.grasshopper3d.com/

  36. Schlenger, J.: Implementation of an ifc file creator and modifier using visual programming. Ph.D. thesis, Technical University of Munich (2022)

    Google Scholar 

  37. Williams, R., Moskal, M., De Halleux, P.: Ml blocks: a block-based, graphical user interface for creating tinyml models. In: 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pp. 1–5 (2022)

    Google Scholar 

  38. Demšar, J., et al.: Orange: data mining toolbox in python. J. Mach. Learn. Res. 14(1), 2349–2353 (2013)

    Google Scholar 

  39. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell (2000)

    Book  Google Scholar 

  40. Lazar, J., Feng, J., Hochheiser, H.: Research methods in human-computer interaction. Morgan Kaufmann, Publisher (2017)

    Google Scholar 

  41. Green, T.R.G., Blandford, A.E., Church, L., Roast, C.R., Clarke, S.: Cognitive dimensions: Achievements, new directions, and open questions. J. Vis. Lang. Comput. 17(4), 328–365 (2006). https://www.sciencedirect.com/science/article/pii/S1045926X06000280, ten Years of Cognitive Dimensions

  42. Blackwell, A., Green, T.: Notational systems the cognitive dimensions of notations framework. HCI models, theories, and frameworks: toward an interdisciplinary science. Morgan Kaufmann, vol. 234 (2003)

    Google Scholar 

  43. Blackwell, A., Green, T.: A cognitive dimensions questionnaire optimised for users. PPIG, vol. 13 (2000)

    Google Scholar 

Download references

Acknowledgement

Research partly funded by PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - “FAIR - Future Artificial Intelligence Research” - Spoke 1 “Human- centered AI”, funded by the European Commission under the NextGeneration EU programme.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Pisa, Pisa, Italy

    Serena Versino, Tommaso Turchi & Alessio Malizia

  2. Molde University College, Molde, Norway

    Alessio Malizia

Authors
  1. Serena Versino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tommaso Turchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessio Malizia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serena Versino .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Ethics declarations

Disclosure of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Versino, S., Turchi, T., Malizia, A. (2024). PyFlowML: A Visual Language Framework to Foster Participation in ML-Based Decision Making. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2024. Lecture Notes in Computer Science(), vol 14734. Springer, Cham. https://doi.org/10.1007/978-3-031-60606-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-60606-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-60605-2

  • Online ISBN: 978-3-031-60606-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy