sustainability

Article
Fostering AI Literacy in Elementary Science, Technology,
Engineering, Art, and Mathematics (STEAM) Education in the
Age of Generative AI
Stefanus Christian Relmasira 1,2, * , Yiu Chi Lai 1 and Jonan Phillip Donaldson 3

1 Department of Mathematics and Information Technology, The Education University of Hong Kong,
Hong Kong, China; yiuchi@eduhk.hk
2 Primary School Teacher Education, Satya Wacana Christian University, Salatiga 50711, Indonesia
3 Department of Curriculum and Instruction, University of Alabama at Birmingham,
Birmingham, AL 35294, USA
* Correspondence: s1131878@s.eduhk.hk or srelmasira@uksw.edu

Abstract: The advancement of generative AI technologies underscores the need for AI literacy, par-
ticularly in Southeast Asia’s elementary Science, Technology, Engineering, Art, and Mathematics
(STEAM) education. This study explores the development of AI literacy principles for elementary stu-
dents. Utilizing existing AI literacy models, a three-session classroom intervention was implemented
in an Indonesian school, grounded in constructivist, constructionist, and transformative learning
theories. Through design-based research (DBR) and network analysis of reflection papers (n = 77), the
intervention was evaluated and redesigned. Findings revealed clusters of interdependent elements
of learner experiences, categorized into successes, struggles, and alignments with learning theories.
These were translated into design moves for future intervention iterations, forming design principles
for AI literacy development. The study contributes insights into optimizing the positive effects and
minimizing the negative impacts of AI in education.

Keywords: AI literacy; generative AI; STEAM education; elementary school; design-based research;
network analysis
Citation: Relmasira, S.C.; Lai, Y.C.;
Donaldson, J.P. Fostering AI Literacy
in Elementary Science, Technology,
Engineering, Art, and Mathematics
1. Introduction and Background
(STEAM) Education in the Age of
Generative AI. Sustainability 2023, 15,
The potential adoption of artificial intelligence (AI) in various sectors in Southeast Asia
13595. https://doi.org/10.3390/ is promising, with significant implications for financial services, healthcare, the high-tech
su151813595 and telecommunication sector, and education. In ASEAN education, AI-based intelligent
tutor systems have been adopted and studied by Singapore and Malaysia to provide per-
Academic Editor: Assunta Di Vaio
sonalized teaching [1]. The ubiquity of smartphones and internet connectivity has made AI
Received: 5 August 2023 an integral part of children’s lives in many countries. For instance, AI-powered applications
Revised: 2 September 2023 such as language learning apps, personalized tutoring systems, and educational games are
Accepted: 3 September 2023 increasingly being used by children for learning purposes [2].
Published: 12 September 2023 Moreover, children are also interacting with AI through voice assistants like Siri, Alexa,
and Google Assistant, which are commonly found in smartphones and other smart devices.
These AI assistants can answer questions, set reminders, play music, and perform a variety
of other tasks, thereby familiarizing children with AI technology from an early age [3,4].
Copyright: © 2023 by the authors.
In the context of STEAM (Science, Technology, Engineering, Art, and Mathematics)
Licensee MDPI, Basel, Switzerland.
education, numerous research studies have employed artificial intelligence (AI) to enhance
This article is an open access article
learning. For instance, Hsu et al. [5] explored the use of AI image recognition in conjunction
distributed under the terms and
with a microcomputer control board. Kim and Kim [6] investigated teachers’ perspectives
conditions of the Creative Commons
on the application of AI in scientific writing within STEM disciplines. A unique approach
Attribution (CC BY) license (https://
was adopted by Narahara and Kobayashi [7], who trained a toy robot car using TensorFlow,
creativecommons.org/licenses/by/
4.0/).
an open-source machine learning framework pre-loaded on the Raspberry Pi, to infuse

Sustainability 2023, 15, 13595. https://doi.org/10.3390/su151813595 https://www.mdpi.com/journal/sustainability

Sustainability 2023, 15, 13595 2 of 25

more AI elements into robotics workshops for STEAM education. Additionally, Leonard [8]
briefly discussed the potential of AI in art education, providing examples of Google AI
tools such as Quick Draw and Auto Draw.
Despite the numerous benefits of integrating AI across various sectors, including
education, ethical issues pose significant challenges to its adoption [1]. It is crucial to
recognize that while experiences exposing children to AI are valuable, education alone
does not equip them with the necessary knowledge and skills to understand, use, and
evaluate AI technologies in an ethically appropriate manner. This highlights the importance
of incorporating AI literacy into the formal school curriculum, a step that many Asian
countries should consider.
To address the urgent need for AI literacy among children, this study adopted a design-
based research approach. It applied key learning principles derived from the literature on
AI literacy in an intervention grounded in cognitive constructivism, social constructivism,
constructionism, and transformative learning theories. The intervention was implemented
with grade five primary school students in Salatiga, Central Java, Indonesia. Consequently,
the research question we sought to answer was: how do elementary students experience
the application of learning theories and AI literacy principles in an intervention designed
to facilitate their understanding of AI, their engagement in the learning process, and their
ability to articulate the ethical or societal impacts of AI?
The study’s analysis drew on data from student reflective papers (n = 77) in the fourth
design iteration of the AI literacy development intervention. This data enabled us to
construct a network map using cluster analysis, which identified patterns in correlations
among various aspects of their experience of the intervention. These nodes reflected the
successes and struggles in students’ learning experiences, as well as the alignment of their
experiences with principles from learning theories. Based on these findings, we developed
design moves for the next iteration, providing solutions to the issues students faced. Further
details regarding the interventions, the design moves, and design principles are elaborated
upon in this study.
This study implemented a design-based research approach explained in the Method-
ology section to develop and iteratively refine an AI literacy intervention for elementary
students in Indonesia. Grounded in learning theories described in the Literature Review
section, the intervention (also described in the Methodology section) utilized student re-
flections and network analysis to derive design principles for AI literacy education suited
to young learners. In the Findings section, we highlight the importance of social learning,
hands-on activities, and appropriate scaffolding to make AI concepts accessible and mean-
ingful. The Discussion section suggests that this research provides valuable direction for
integrating AI literacy into elementary STEAM education through pedagogically sound
methods tailored to children’s learning needs. Details of the study design, methods, and
outcomes are presented in the sections that follow.

2. Literature Review
2.1. Definition of AI and Generative AI
Artificial intelligence (AI) is a branch of computer science focused on developing smart
machines capable of mimicking human thought and behavior. It involves the creation of
algorithms that enable machines to learn from data, make informed decisions, and solve
problems. AI can be divided into two primary categories: narrow AI and general AI [9].
Narrow AI is programmed to carry out specific tasks such as facial recognition or online
searches, while general AI has the potential capacity to execute any cognitive task a human
can do. Generative AI specifically refers to a class of artificial intelligence models that
use existing data to create new content that mirrors the underlying patterns of the real-
world data [10]. These models have found significant applications across various domains,
including natural language processing, computer vision, and brain imaging [10,11]. In the
creative arts, generative AI tools have been instrumental in producing high-quality artistic
media, including visual arts, music, literature, video, and animation [12]. We believe that
Sustainability 2023, 15, 13595 3 of 25

the generative capabilities of these AI tools are fundamentally altering creative processes,
leading to a reimagining of creativity across many sectors of society, including education.

2.2. Importance of AI Literacy in STEAM Education

AI literacy is increasingly important in STEAM (Science, Technology, Engineering, Art,
and Mathematics) education. The rapid expansion of artificial intelligence (AI) in various
sectors necessitates the integration of AI literacy in STEAM education. As Lee et al. [13]
highlight, preparing students to become informed citizens and critical consumers of AI
technology is crucial for their future endeavors as AI-empowered workers. This preparation
involves not only the development of technical skills but also the cultivation of critical
thinking. How and Hung [14] demonstrate how STEAM learners can use AI to predictively
simulate various scenarios, thereby enhancing their problem-solving skills.
The importance of AI literacy in STEAM education extends beyond the development
of technical and critical thinking skills. It also involves understanding the ethical and
societal implications of AI. Zhang et al. [15] posit that students must learn about these im-
plications alongside technical concepts and processes. This comprehensive understanding
of AI as a sociotechnical system with socio-political implications is crucial for educating
AI-literate citizens.
Moreover, the inclusion of AI literacy in STEAM education can foster creativity
and innovation. Aguilera and Ortiz-Revilla [16] argue that the inclusion of the arts in
STEM education can enhance student creativity, a key skill in the era of AI. Furthermore,
How and Hung [14] suggest that AI analytics can serve as educational scaffolds in STEAM
education, promoting AI-assisted human-centric reasoning and knowledge development.

2.3. Existing Models and Approaches for Developing AI Literacy in Children

As of July 2023, two articles that propose models of AI literacy have gained at-
tention in the academic community, as evidenced by their citation counts in Google
Scholar. The Long and Magerko [17] article has received 471 citations, while the article by
Touretzky et al. [18] has been cited 348 times.
Long and Magerko [17] define AI literacy as a set of competencies that empower
individuals to critically evaluate AI technologies, communicate and collaborate effectively
with AI, and utilize AI as a tool in various settings like online platforms, home, and
workplaces. Notably, they argue that programming ability is not a prerequisite for AI
literacy. The authors constructed an AI literacy framework around five central questions.
The first question, “What is AI?”, encompasses four competencies: Recognizing AI, Un-
derstanding Intelligence, Interdisciplinarity, and General vs. Narrow AI. The second
question, “What can AI do?”, includes two competencies: AI Strengths & Weaknesses
and Imagining Future AI. The third question, “How does AI work?”, incorporates nine
competencies: Representations, Decision Making, Machine Learning Steps, The Human
Role in AI, Data Literacy, Learning from Data, Critical Interpretation of Data, Action and
Reaction, and Sensors. The fourth question, “How should AI be used?”, is associated with
one competency: Ethics. The final question, “How do people perceive AI?”, is linked to one
competency: Programmability.
On the other hand, Touretzky et al. [18] do not provide a direct definition of AI literacy,
but they provide a conceptual framework that implies a definition. According to this
framework, AI literacy involves understanding the fundamental concepts of AI, the ability
to interact with AI technologies, and an awareness of the societal impacts of AI [18]. They
argue that K-12 students should be equipped with the knowledge to understand how
AI works and how it will shape their future. Furthermore, the authors emphasize the
importance of hands-on experiences with AI technologies and the need for critical thinking
regarding the impacts of AI applications. They suggest that students should be able to
evaluate new AI technologies and articulate the ethical or societal impact questions they
raise. This approach to AI literacy underscores the importance of not only understanding
AI technology but also being able to engage with it in a meaningful and informed way [18].
Sustainability 2023, 15, 13595 4 of 25

2.4. Theoretical Foundations of Constructivist, Constructionist, and Transformative Learning

This study leveraged a design-based research (DBR) approach, the initial phase of
which involves the identification of applicable learning theories that underscore the learn-
ing experience under design, and subsequently, translating these theories’ core concepts
into a set of cogent design principles [19]. We started with constructivist learning theory,
which has two strands. Cognitive constructivist learning theory conceptualizes learning
as a process where individuals construct meaning and knowledge [20]. From cognitive
constructivism we derived the design principles of (1) individual knowledge construction,
(2) schema building or connecting ideas, and (3) knowledge consolidation or reinforce-
ment. In social constructivist learning theory, learning is seen as collective knowledge
construction mediated by artifacts, tools, language, and culture [21]. The design principles
from social constructivism were (1) collaborative knowledge construction, (2) scaffolding,
(3) working in the zone of proximal development, and (4) mediating artifacts, tools, and tech-
nologies. The constructionist theory advances the constructivist perspectives by positing
that knowledge is active—that is, it should be treated as a verb rather than a noun—which
suggests that learning occurs when students construct physical, digital, or representational
artifacts [22]. As students construct artifacts that reflect the knowledge they are construct-
ing, the construction of the artifact further informs the construction of knowledge in a
positive feedback loop. From constructionist theory we derived the design principles of
(1) making or generativity, (2) tinkering, exploration, and productive failure, (3) learner
agency, authority, and autonomy, and (4) authentic audience or purpose with real-world
impact. Transformative learning theory defines learning as a change in assumptions, beliefs,
and frames [23]. The design principles from this theory were (1) questioning beliefs or
assumptions, (2) developing new perspectives, and (3) changing beliefs or assumptions.

3. Methodology
This study employed a design-based research approach, and focused on the fourth
iteration. Design-based research is an iterative, collaborative process conducted in real-
world settings, driven by both theory and empirical evidence, with the ultimate goal of
improving educational practices. It integrates the design of learning environments with
their analysis, acknowledges the importance of context, and aims to produce contextual
outcomes that can positively impact education [19,24–26]. In conducting our design-based
research, we adapted the design framework proposed by Anderson and Shattuck [24]. The
first step in DBR was identifying a practical problem in a real-world context of AI literacy.
In this case, our research commenced with grounding the design in the principles derived
from cognitive constructivism, social constructivism, constructionist, and transformative
learning theories, which framed our understanding of how students learn. This step was
crucial to ensure that our intervention was theoretically sound and informed by relevant
pedagogical perspectives.
The second DBR step was developing a solution informed by existing learning theories
and research about AI literacy. Following the establishment of our theoretical grounding,
we moved to the design phase of our intervention. Guided by the principles derived from
our selected theories, we crafted a learning environment that was both supportive and con-
ducive for students to construct knowledge, explore and experiment, work collaboratively,
and engage in transformative learning.
The third step in the DBR process was to iteratively test and refine the solution in
the context of improving students’ AI literacy. Once the intervention was designed, we
then implemented it in a real-world setting. The intervention was carefully redesigned
based on findings from our analysis of the previous intervention in iteration 3. Throughout
and following the implementation of the intervention, we collected data to evaluate its
effectiveness. We used students’ self-reflections as the primary source of data. In the
reflections, students were required to explain what worked, what didn’t work, and what
new idea(s) they learned from the learning activities. This step was critical in understanding
Sustainability 2023, 15, 13595 5 of 25

how students interacted with the intervention, what they learned, and how they felt about
the learning experience.
The fourth step in DBR was to evaluate the findings from the previous step to produce
changes to the design of the intervention. In this step, we analyzed data to craft design
moves. After data collection, we undertook a thorough analysis of the data to identify pat-
terns of relationships between challenges, successes, and alignment of aspects of learning
experiences with principles from the theories in which the design was initially grounded.
We developed a network map of significantly correlated aspects of learning experiences
described in students’ reflections to identify patterns in students’ struggles, what worked
particularly well for them, and principles from learning theories aligned with their ex-
periences. These patterns allowed us to leverage strengths of the design and principles
from theories to address struggles. The insights derived from this analysis informed our
design moves as part of lesson modifications and refinements to improve the intervention.
This step helped us understand what worked, what didn’t, and what needed to change to
enhance the effectiveness of the intervention.
The final DBR stage was to derive design principles from the design moves that can
guide future work, followed by sharing the results to contribute to theory and knowledge
about AI literacy. This step involved sharing findings and the refined intervention with the
broader research community, contributing to the knowledge about AI literacy.
The analysis in this study focuses primarily on the fourth iteration, with a brief
description of prior iterations to provide the reader with sufficient context. In line with the
DBR approach, this intervention did not happen as a one-off occurrence. Instead, it was a
systematic and iterative process consisting of four iterations. Each iteration was carefully
informed by data analysis from the preceding cycle. The aim of these four iterations was
not merely to create, but to continually refine and improve the intervention, ensuring its
effectiveness in supporting student learning.

3.1. Participants and Context

The study was conducted in Salatiga, Central Java Province, Indonesia, and involved
two public schools centrally located in the city. The first school was the site for iterations
1-3, with a total of 55 fifth-grade primary school students participating. The second school
was involved in iteration 4, where 25 fifth-grade primary school students participated.
Both schools had parental consent for the students involved. According to teacher inter-
views, the students in both schools predominantly come from middle-low-income families.
While almost all students have access to smartphones and the internet, the second school
additionally provided one computer per student with internet access. It should be noted
that the students’ computer literacy is at a beginner level, and although they had no prior
knowledge of AI, they were all familiar with platforms like YouTube and the Google
search engine.

3.2. Intervention Design and Implementation

The goal of the intervention was to refine best practice and strategies to improve
primary school students’ AI literacy by using principles of AI literacy from the literature.
The design principles were derived from learning theories mentioned in the literature
review section. This study focuses on analysis of data from the fourth iteration. However,
to help the reader better understand the fourth iteration, we will first describe each of
the previous iterations and the design moves resulting from analysis of those iterations.
The design moves were constructed to leverage the principles and strengths in order to
systematically resolve the struggles, based on an analysis of the relationships between
struggles, successes, and theories evident in the learning experience network map. This
exemplifies an evidence-based approach to refining the intervention by allowing the data
on student learning experiences to guide incremental improvements grounded in theory.
 ous iterations and the design moves resulting from analysis of those iterations. The design
moves were constructed to leverage the principles and strengths in order to systematically
resolve the struggles, based on an analysis of the relationships between struggles, suc-
cesses, and theories evident in the learning experience network map. This exemplifies an
Sustainability 2023, 15, 13595 evidence-based approach to refining the intervention by allowing the data on student 6 of 25
learning experiences to guide incremental improvements grounded in theory.

3.2.1. Iteration 1: Design and Design Moves

3.2.1. Iteration 1: Design and Design Moves
Iteration one was divided into three sessions. Each session consisted of a 60 min les-
Iteration one was divided into three sessions. Each session consisted of a 60 min lesson.
son. The following paragraphs explain each session in detail.
The following paragraphs explain each session in detail.
Session 1—Interaction
Session 1—Interaction
In this
In this session,
session, students
students used
used AIAI to
to solve
solve problems
problems where
where children
children collaborated
collaborated with
with
others using AI. This session consisted of three main
others using AI. This session consisted of three main activities. activities.
Session 11 Activity
Session Activity 1—Categorizing
1—Categorizing and and classifying
classifying jumbled
jumbled pictures
pictures (unplugged):
(unplugged):
Students in
Students in the
the first
first activity
activity developed
developed aa strategy
strategy to
to categorize
categorize images.
images. Then
Then they
they were
were
expected to explain the process of classifying images. Lastly, they
expected to explain the process of classifying images. Lastly, they needed to recognize needed to recognize
patterns and
patterns and cognitive
cognitive dissonance
dissonance in in the
the categorization
categorization processes.
processes. Students
Students were
were given
given
jumbled pictures. Then they worked together in groups of 4–5 to sort
jumbled pictures. Then they worked together in groups of 4–5 to sort the pictures into at the pictures into at
least 3 categories (see Figure 1). The students were not given guidance
least 3 categories (see Figure 1). The students were not given guidance or hints about whator hints about what
categoriesare
categories arepossible.
possible.OnceOncethey
theyfinished,
finished, another
another unidentified
unidentified andand uncategorized
uncategorized pic-
picture
tureadded,
was was added,
enablingenabling
them tothem
havetoahave
littleadebate
little debate
amongamong the members
the members of theto
of the group group to
decide
decide the appropriate categorization of the new picture. Then, each
the appropriate categorization of the new picture. Then, each group briefly shared their group briefly shared
their chosen
chosen categories
categories with
with the the other
other groups. groups.

Figure1.1. AI
Figure AIanime-style
anime-styleillustration
illustrationofofstudents’
students’ collaboration
collaboration in categorizing
in categorizing andand classifying
classifying jum-
jumbled
bled pictures. Identifying details have been modified for
pictures. Identifying details have been modified for anonymity. anonymity.

This
This unplugged
unplugged activity
activity was
was designed to help the student build a basic understand-
ing
ing about data
data training
trainingininmachine
machinelearning.
learning.In In machine
machine learning,
learning, categorization
categorization is a
is a fun-
fundamental
damental partpart of training
of training a model.
a model. ForFor example,
example, in supervised
in supervised learning,
learning, datadata points
points are
are categorized into different classes or labels, and the model is trained to predict
categorized into different classes or labels, and the model is trained to predict these labels these
labels accurately.
accurately. The task
The task of grouping
of grouping pictures
pictures intointo categoriesteaches
categories teachesstudents
students to
to identify
identify
common characteristics, features, or patterns within a set of data that can be used for
classification. Recognizing cognitive dissonance in the categorization process can serve as
a useful introduction to dealing with noisy or inconsistent data. In the real world, data is
often imperfect and can be challenging to classify consistently. Cognitive dissonance in this
context might come from images that could plausibly fit into multiple categories or from
images that don’t clearly fit into any category. This mirrors the situations that a machine
learning model often needs to deal with, and strategies to manage this uncertainty (such
as probabilistic classification, or creating a separate “other” category) can be explored by
the students.
 useful introduction to dealing with noisy or inconsistent data. In the real world, data is
often imperfect and can be challenging to classify consistently. Cognitive dissonance in
this context might come from images that could plausibly fit into multiple categories or
from images that don’t clearly fit into any category. This mirrors the situations that a ma-
chine learning model often needs to deal with, and strategies to manage this uncertainty
Sustainability 2023, 15, 13595 (such as probabilistic classification, or creating a separate “other” category) can be7 of 25
ex-
plored by the students.
Session 1 Activity 2—Collaborate with others and AI using Teachable Machine: In the
second activity
Session of the first
1 Activity session, students
2—Collaborate withused
othersGoogle’s
and AITeachable MachineMachine:
using Teachable [27] to col-In
laborate with an AI and with friends to solve a problem, which
the second activity of the first session, students used Google’s Teachable Machine in this activity involved
[27] to
finding a way
collaborate withto an
make an AI
AI and recognize
with friendstheir faces,
to solve things, orwhich
a problem, gestures. In groups
in this activityof 3 with
involved
one computer
finding a way toformake
each angroup, students their
AI recognize werefaces,
askedthings,
to buildorclassifiers in Google’s
gestures. In groups ofTeach-
3 with
ablecomputer
one Machine for (see Figure
each group,2). students
They were wereinstructed
asked to buildto choose the following
classifiers in Google’scategories:
Teachable
“boy-girl”;
Machine “happy-sad”;
(see Figure 2). They or make your own to
were instructed category.
choose the Then the students
following built these
categories: clas-
“boy-girl”;
sifiers by taking as many pictures as possible with their webcams. The
“happy-sad”; or make your own category. Then the students built these classifiers by taking pictures had to be
representative of the 2 categories in the classification scheme they
as many pictures as possible with their webcams. The pictures had to be representative had selected. These pic-
tures
of thewere input automatically
2 categories in the Teachable
in the classification scheme they Machinehad (they did not
selected. have
These to savewere
pictures the
photosautomatically
input to their computerin thefirst). After the
Teachable classifiers
Machine (they were
did built, theyto
not have tested
savethe
thealgorithm
photos to
with computer
their at least 3 new pictures
first). After thecorrectly classified
classifiers in a row.
were built, theyIftested
it wasthe
notalgorithm
classifyingwithcorrectly,
at least
3they
newhad to either
pictures re-trainclassified
correctly the classifier or figure
in a row. out why
If it was it was not working
not classifying correctly,and
theyexplain
had to
the problem.
either re-train the classifier or figure out why it was not working and explain the problem.

Figure 2.
Figure 2. AI anime-style
anime-styleillustration
illustrationof
ofstudents’
students’collaboration
collaborationwith
withAI
AITeachable
TeachableMachine.
Machine.Identifying
Identify-
ing details
details havehave
beenbeen modified
modified for anonymity.
for anonymity.

Teachable Machine is a web-based

web-based tooltool that
that allows
allows users
users to
to train
train machine
machine learning
learning
models visually,
models visually,without
withoutcoding.
coding.There
There waswas a whole-class
a whole-class discussion
discussion after
after students
students fin-
finished
ishedwork.
their their work. This activity
This activity was designed
was designed to helpto help students
students learnthe
learn about about the concept
concept of trainingof
training
data data by
by using using
their their webcams
webcams to take to take pictures
pictures that arethat areused
then then to
used to the
train trainAI.theThey
AI.
They experience
experience how these
how these data
data are are to
used used to inform
inform theunderstanding
the AI’s AI’s understanding and categori-
and categorization.
zation.
This This provides
provides a concrete
a concrete example example
of howofAIhow AI systems
systems learn data.
learn from from data. Students
Students also
also learn
learn about
about problem-solving
problem-solving and debugging.
and debugging. If theIfAI
theisAI
notis classifying
not classifying correctly,
correctly, students
students will
need
will to troubleshoot
need and figure
to troubleshoot outfigure
and why. This outprocess
why. canThisimprove
processtheir
canunderstanding
improve their of
how AI works and the types of issues that can arise, such as bias in the training data. This
also enhances critical thinking skills and collaboration, which are crucial for AI literacy.
Session 1 Activity 3—Self-reflection: A reflection activity was used not only to help
students consolidate their learning, but also as part of data collection. It was conducted
with the students writing manually on paper. The questions students addressed in their
reflections were (1) “What worked really well today? What part of it did you enjoy most?”;
(2) “What did you struggle with today? Why did you struggle?”; (3) “What new ideas
did you have today?” They were expected to explain as deeply as possible how they
experienced the learning activities, including the challenges they faced. This data, along
with data from reflections at the end of the next two sessions, was then used for analysis,
resulting in a list of design moves for improving the next iteration.
Sustainability 2023, 15, 13595 8 of 25

Session 2—Interaction, Explaining, and Evaluating

In this session, students interacted with AI products. Then, grounded in the principles
of constructionist learning theory, students utilized AI for art and creative expression, while
also developing an understanding of the underlying logic and processes that the AI uses.
This session consisted of 5 main activities.
Session 2 Activity 1—Interaction demo with AI: The instructor used a smart speaker
and smart plug to introduce this new type of AI. Then we let students interact with the
products by giving commands, asking questions, or otherwise communicating with the
smart speaker in a game called Hey Google! In this game, the students needed to prompt
Google to mention a word the instructor provided. This activity was designed because
smart speakers and smart plugs are ubiquitous, tangible, and interactive examples of AI
technologies. Interacting with these devices provides students with firsthand experience of
AI in action, helping them understand its capabilities and limitations.
Session 2 Activity 2—Pictionary game (unplugged): In this activity, the instructor
started a drawing, and the student guessed. The student who guessed correctly took the
next turn to draw on the whiteboard. In this activity, the instructor needed to start with
an easy example and a difficult example. The purpose of this activity was to demonstrate
the limitations and potential ambiguity in AI. Just as an image drawn by a person may be
ambiguous and lead to incorrect guesses, AI algorithms can also struggle with ambiguity,
leading to misclassification or errors. It can also help students understand that the quality
of the output (the guess) heavily depends on the quality of the input (the drawing).
Session 2 Activity 3—Guessing game using Google Quick Draw: The instructor had
students volunteer to come to the front of the room and draw on the teacher’s computer
(projected on the classroom overhead projector) using Google’s Quick Draw [28] and
the other students guessed what they were drawing. Then students had a whole-class
discussion about how they guessed and how the AI guessed. The goal of this activity was
to deepen the students’ understanding of the crucial role quality data plays in training
effective machine learning models. By using Quick Draw, they experience firsthand how
both they and the AI make guesses based on the data (drawings) provided. The subsequent
discussion aims to highlight the parallels between their human processes of interpreting
drawings and the AI’s process of recognizing patterns. Through these experiences, students
learn that the accuracy of AI predictions heavily relies on the quality and clarity of the
training data.
Session 2 Activity 4—Making art using Google AutoDraw: In groups of 4, each student
opened AutoDraw [29] on their computer. They started drawing for 15 s. When the timer
stopped, they had to stop drawing and move to another student’s computer in their group.
Sustainability 2023, 15, x FOR PEER REVIEW
Then they added to their friend’s drawing for 15 s. Then they moved to another student’s 9 of 26
computer. They repeated this until all students had contributed to all their group-mate’s
drawings, which took approximately 3 min. Then everyone in each group looked at each
of their member’s
emergent idea (seedrawings and
Figure 3). discussed
Then the original
the instructor idea versus
had each the emergent
team clear idea (see
all their drawings
Figure 3). Then the
and do 2 more rounds. instructor had each team clear all their drawings and do 2 more rounds.

Figure 3.
Figure 3. Examples of student work in the
the AutoDraw activity.
activity.

This learning activity was designed to help students recognize that each addition of
data impacts the end result. This also helps them recognize some of the limitations of AI.
Session 2 Activity 5—Self-reflections: The self-reflection activity was similar to the
reflection activity in the first session.
Sustainability 2023, 15, 13595 9 of 25

This learning activity was designed to help students recognize that each addition of
data impacts the end result. This also helps them recognize some of the limitations of AI.
Session 2 Activity 5—Self-reflections: The self-reflection activity was similar to the
reflection activity in the first session.

Session 3—Recognition and Ethics

The objectives of session 3 were to help students be able to recognize the product
of AI, as well as increasing their ethical awareness about AI. Thus, after completing the
session 3 activities, students should be able to understand the good and bad effects of AI
on society, create a poster of how to identify AI products, and create a poster of how to
protect yourself and others from the bad effects of AI. This session consisted of 4 activities.
Session 3 Activity 1—Observing AI products: This first activity began by watching a
clip on YouTube from the American Got Talent show about the use of AI to create “hyper
real content”, in this case a deepfake of Simon Cowell (one of the judges) singing on the
stage [30]. Then students discussed briefly how it works and what the purposes may have
been. By watching the video, students will have an example of how AI is being used
in a real-world context. The purpose of this activity was to demonstrate how AI can be
used in creative and entertaining ways, potentially inspiring students to consider how
they could use AI for good purposes. Then, students watched the Indonesian YouTube
video of news about how an AI was used to create deepfake content of the Indonesian
president [31]. By watching the video, the goal was to open up a discussion about the
ethical considerations of AI use. They are expected to learn that deepfakes can also be
misused, such as for spreading misinformation or fraud. Understanding this helps students
to consider the wider implications and responsibilities that come with developing and
using AI technologies.
Session 3 Activity 2—Instagram AI filters: In this activity, the instructor connected a
smartphone to the projector and had students volunteer to come up and mess up the video
and image Instagram AI filters (see Figure 4). Instagram filters are a tangible demonstration
of how AI technology is embedded in our everyday lives.
This activity was designed to help students realize that AI isn’t just about robots or
complex algorithms, but also about the apps and tools they use daily. Using Instagram
filters can also open a discussion about the ethical and social implications of AI, for instance,
how altering images can create unrealistic beauty standards, or how image recognition
might be used to spread misinformation.
Session 3 Activity 3—Evil genius plan group project: This activity engaged students in
collaborative work in groups of 3 to develop an “evil genius” plan of using AI for bad things.
Then students were asked to discuss in small groups about how we can protect ourselves
(and people around us) from AI-produced harmful effects, deepfakes, or fake news. Then
each group created a short presentation of their strategies for protecting ourselves (see
Figure 5).
This activity was designed to help students become aware of the potential misuse of
AI by developing “evil genius” plans. This includes deepfakes, targeted misinformation,
manipulation of personal data, and more. This helps them understand the ethics and
darker side of technology, promoting a more balanced view of AI’s pros and cons. The
presentations could be in the form of a poster or PowerPoint presentation as a means
through which students reify the knowledge they constructed.
Session 3 Activity 4—Self-reflection: The self-reflection included questions similar
to the previous questions in session 1 and 2 but with the addition of 3 more questions:
(4) “From all 3 days we worked together, what was the most wonderful thing?”; (5) “From
all 3 days we worked together, what was the worst problem?”; and (6) “From all 3 days we
worked together, what was the most important new idea you had?”
Sustainability 2023,15,
Sustainability2023, 15,13595
x FOR PEER REVIEW 10 of
10 of 25
26

Figure 4.4. AI
Figure AIanime-style
anime-styleillustration
illustration
of of students’
students’ Instagram
Instagram AI filters
AI filters activity.
activity. Identifying
Identifying detailsdetails
have
have modified
been been modified for anonymity.
for anonymity.

3.2.2.This
Iteration 1 Design
activity Moves to help students realize that AI isn’t just about robots or
was designed
complex algorithms,
Through analysisbut alsonetwork
of the about the apps
map and tools they
of significantly use daily.
correlated Using Instagram
struggles (the data
filters can
analysis also open
process will abediscussion
describedaboutin thethe ethical
section onand social 4implications
iteration of AI,detail),
below in greater for in-
stance,
what how altering
worked images
particularly can and
well, create unrealisticaligned
experiences beauty with
standards, or how
principles fromimage recog-
theory we
nition mightabe
constructed setused to spread
of design misinformation.
moves. The first design move involved modifying the session
1 activity to use
Session different
3 Activity typesgenius
3—Evil of photographs
plan groupthat moreThis
project: closely resemble
activity engagedthe students
types of
categorizations
in collaborative theyworkwill do in of
in groups activity 2. In this
3 to develop case, genius”
an “evil the instructor
plan ofwill
using need to bad
AI for use
photos
things. of a diversity
Then studentsofwere
people doing
asked a diversity
to discuss of things
in small wearing
groups abouta how
diversity
we canof clothes.
protect
This activity
ourselves willpeople
(and also bearound
modifiedus) to limit
from the numberharmful
AI-produced of categories each
effects, group makes
deepfakes, to
or fake
2news.
categories.
Then eachAnother
groupdesign
createdmove waspresentation
a short that when students conduct data
of their strategies training with
for protecting our-
Teachable
selves (seeMachine,
Figure 5).the instructor will give more time for students to tinker with training
Sustainability 2023, 15, 13595 11 of 25

the AI. This tinkering time will be done collaboratively in groups and will allow for optimal
learner agency. During the tinkering, the facilitator will encourage them to talk a lot about
what they are thinking. The instructor will have to start with a discussion right after the
teacher demonstration about what kinds of data the AI does not handle well. This activity
Sustainability 2023, 15, x FOR PEER REVIEW 11 of to
will also be modified to limit the number of categories each group uses for data training 26
2 categories. This will keep their work within their zone of proximal development.

Figure 5.
Figure 5. Evil
Evil genius
genius plan
plan example.
example.

The
Thisnext design
activity wasmove was that
designed in session
to help 2 the
students teacheraware
become will guide
of thethe students
potential in what
misuse of
kinds of things the AI (Google Home speaker) can understand, and will
AI by developing “evil genius” plans. This includes deepfakes, targeted misinformation, give them hints
for better recognition
manipulation during
of personal the and
data, activity.
more.Moving to session
This helps them2understand
activity 2, intheguessing
ethics and the
picture, it will be modified to be a whole-class activity where the teacher
darker side of technology, promoting a more balanced view of AI’s pros and cons. The translates (elicit
translations
presentations first) eachbe
could prompt
in theinform
Quickdraw before
of a poster or pressing the “Got
PowerPoint it!” button.
presentation as The time
a means
for discussion
through whichwill be changed
students from
reify the 10 min to
knowledge 5 min.
they In using AutoDraw, there will be
constructed.
2 “rounds”
Session of3drawing
Activityin teams, but between
4—Self-reflection: Thethe 1st and 2nd round,
self-reflection included students
questionswillsimilar
be asked to
to discuss as a whole class about what the AI needs from them, such
the previous questions in session 1 and 2 but with the addition of 3 more questions:as what types of data(4)
AI will use.
“From all 3 In activity
days 2, the instructor
we worked together, will
whathave
wastotheconnect a smartphone
most wonderful to the(5)
thing?”; projector
“From
and have students volunteer to come up and play with the video and
all 3 days we worked together, what was the worst problem?”; and (6) “From all 3 days image Instagram
AI
wefilters.
worked together, what was the most important new idea you had?”
Moving to the design moves for session 3, students will have a whole-class discussion
about protecting
3.2.2. Iteration ourselves.
1 Design MovesStudents may have a discussion while doing a new DALL·E
activity. The instructor will use DALL·E for making deepfakes on the instructor’s computer
Through analysis of the network map of significantly correlated struggles (the data
using deepfake ideas from the evil plan activity. The activity will be modified also to be
analysis process will be described in the section on iteration 4 below in greater detail),
more collaborative—each team has to come up with only one evil genius plan and teams
what worked particularly well, and experiences aligned with principles from theory we
will compete for the title of “evil genius”—and the students will collaboratively create and
constructed a set of design moves. The first design move involved modifying the session
present posters for their presentation.
1 activity to use different types of photographs that more closely resemble the types of
categorizations they will do in activity 2. In this case, the instructor will need to use photos
of a diversity of people doing a diversity of things wearing a diversity of clothes. This
activity will also be modified to limit the number of categories each group makes to 2
categories. Another design move was that when students conduct data training with
Sustainability 2023, 15, 13595 12 of 25

3.2.3. Iteration 2: Design and Design Moves

The list of design moves from the analysis of iteration 1 described above were imple-
mented in iteration 2. Analysis of the data resulted in a new list of design moves. First, all
activities in the next iteration should end with knowledge consolidation and reinforcement
discussions, especially focusing on categorizing and classifying data. In the first activity,
the reason for the activity will more clearly be stated in the beginning and at the end of
the activity. The instructor will emphasize that it is not about right and wrong, but instead,
it is important to explain the logical reasons for each group decision in categorizing and
classifying data.
Another design move was that in the second activity, the instructor should tell the
student sitting at the computer to act like a “driver”, and that they cannot make any
decisions. Rather, the teammates will take on the role of “navigators” and tell them what
to do. If the “driver” has an idea, they have to ask their teammates, “can I do something
like this?” The goal of this is to prevent the person sitting at the computer from being
overly dominant. Students will also be asked to move away from the computer during
instructions and discussion time. Students will be told that they can huddle up at the
front of the classroom like a sports team before a game. Then, the instructor will have the
students explain to their classmates what they were planning to do and check to make sure
they understood. In the second activity, students also will be given a chance to present the
result of their work. In their presentation, students will have to provide the reasons for
why the data training failed or was successful.
In design moves for session 2, the Google Home speaker assignment was changed to
emphasize that we are testing the limits of AI, rather than testing students’ ability, because
some students in this iteration felt disappointed when Google Home could not give them
the answer they wanted. During the interaction with Google Home speaker, the student
volunteers will also be told to raise their hand right before they say “Ok Google”. The
other students can give advice, but they have to become completely silent when the student
speaking says “Ok Google”. At the end of each activity in session 2, students will engage
in a knowledge consolidation discussion about what they learned. The design for the
Quickdraw activity will be changed to emphasize that we are not testing the speed of or
the student’s ability in drawing, but rather that we are testing the AI speed and accuracy.
Thus, if they run out of time, that means that their tinkering is good, but the AI-human
collaboration has failed. The AutoDraw assignment originally was planned to be in teams
of 4 students, but during implementation, it was changed to be a whole-class activity in
which students lined up to move like a snake. In the next iteration, the activity may be
changed to be done in teams of 5 students, and the instructions also should emphasize
adding ideas, but never erasing previous students’ ideas.
The major design move for session 3 was that the evil plan activity should be modified
to be divided into two steps. In step 1—before the instructor forms the groups—each
student will silently write two to three evil genius ideas individually on sticky notes with
1 idea on each note. Then, the instructor will divide the students into several groups, after
which each group member will present all of their ideas. Finally, as a team, they should
vote on which idea was the most interesting. In step 2 students will collaboratively create
the evil plan poster. Also in session 3, related to the Instagram assignment, instead of
using 1 phone, there will be 3 phones operated by the instructor and two other facilitators
(university students) who will be recruited by the instructor. Then the instructor will divide
the class into three large groups. Each group will take a phone and play with the video of
the facilitator. At the end, each of the other 2 facilitators should send the screen recordings
to the primary facilitator, who will show them on the overhead projector. During the
knowledge consolidation at the end of the evil genius activity, there should be a greater
focus on what new things they have learned. At the end of each activity in session 3, there
should be a knowledge consolidation discussion. During the DALL·E deepfake activity, the
instructor will continually refocus the discussion on questions such as “AI can trick us, but
can we trick AI? How can the data we give the AI determine the quality of the AI? How
Sustainability 2023, 15, 13595 13 of 25

can we fight back to have control over AIs that might be trying to control us or manipulate
our data?”

3.2.4. Iteration 3: Design and Design Moves

All of the design moves derived from the analysis of patterns of learning experiences in
iteration 2 were implemented in iteration 3. Data were collected and analyzed to construct
a new list of design moves for the subsequent iteration. In terms of design moves for
the first session, the instructor will need to be observing more and asking questions of
students about their decisions to help them develop tinkering skills and encourage them to
be confident in making decisions without being afraid of making mistakes. The purpose of
this design move is to promote productive failure. Also, the instructor will tell the students
again that the main goal of the activity is to find logical reasons for making decisions. This
should be emphasized in the beginning, middle, and end of the activity. Students will also
be reminded again that making mistakes is part of learning. In this case, when students
fail, the instructor needs to ask the group why they failed and lead them to the process of
tinkering and exploration to find the best solution. If there is enough time, the instructor
will lead the students in discussion with the goal of guiding them towards arriving at their
own conclusion that people learn through mistakes and AIs also learn through mistakes.
When categorizing the pictures, the instructor and facilitators will need to ask leading
questions in each group such as “what is the ‘big idea’ of this category?” and “what is the
‘story’ of this category?” They have to make sure the students are talking a lot about the
meaning of the category instead of talking about unrelated things. Also, the instructor
and facilitators will help them just a little bit so they stay on time. In the activity with the
Teachable Machine, the instructor will ask the students to be as “silly” as possible when
capturing data. Also, they will need to push each group to provide more data samples for
AI training.
The design moves for the second session included changing the “Hey Google!” activity
to be shortened to 6 min, as well as shortening the demo to 30 s. It was decided that a new
set of keywords for the instructor to provide the students will need to be tested with various
people outside the class before the session. During the second activity, the instructor should
frequently remind them to slow down and not attempt to compete with the computer.
Before students start doing the AutoDraw activity, the instructor will show students the
functions of each button of the tool.
The major design move for the third session involved changing the evil genius activity
such that the instructor will start by forming groups and have the groups discuss several
evil genius ideas, then individual students will write on sticky notes more ideas individ-
ually. Then the activity will continue as planned. In the activity with Instagram, at the
beginning of when the groups form, each facilitator will explain the navigation and func-
tions of Instagram filters. Then, during the time students are tinkering with the Instagram
filters, the facilitators will provide more guidance when they see students struggling or
doing things that obviously won’t have the desired effect. Also, more facilitator talking
and interaction should be encouraged such as asking students “what are you trying to
do there?”.

3.2.5. Iteration 4 Design

The design moves from iteration 3 were implemented as planned with some additional
changes of the design for iteration 4. In the design of iteration four, in session 1, before the
categorizing and classifying activities (unplugged), the instructor adds an analogy of being
a programmer. As a programmer, students will imagine that they are going to program a
robot to decide which picture belongs to which category (in which they decide the category
together as a team). The purpose of doing that is to build the schema for the students about
how data training works. In session 2, before the class begins, the instructor will need to
make sure that the Google Home speaker and the smart plug are working properly, as some
technical issues may delay the learning process. In session 3, three projectors connected to
Sustainability 2023, 15, 13595 14 of 25

smartphones are needed. Initial setup before the class should have been conducted. The
purpose of doing that is to increase time efficiency and effectiveness for students to explore
the use of Instagram AI filters in 3 big groups. In the last session, the instructor will need to
provide both paper surveys and online surveys. Students in this case may choose to do the
reflection offline or online.

3.3. Iteration 4 Data Collection and Analysis

This paper presents the analysis of iteration 4. The data collection and analysis for
iteration 4 was conducted as follows.

3.3.1. Data Collection

Reflection papers were collected online using Google Forms at the end of each session.
The digital format allowed students to formulate and rethink their responses over time,
without the pressure of crafting a perfect response in one sitting. They could revise and
resubmit their answers as necessary. Also, using the online form allowed automated
collection and organization of responses. In these reflection papers, students were asked to
write about what worked well, struggles they encountered, and new things they learned. In
addition to producing data for this study, this reflective practice promotes a comprehensive
understanding and self-assessment of their learning experiences, which is crucial for the
ongoing success of our study. A total of 77 reflection papers were collected.

3.3.2. Qualitative Coding

After the reflection papers were collected, the next step was to code the data in
MAXQDA Analytics Pro qualitative analysis software. The challenges and difficulties
identified in the reflections were coded as “struggles”. The successes students experienced
are coded as “what worked.” The student experiences were coded for alignment with
principles from cognitive constructivist learning theory, social constructivist learning theory,
constructionist learning theory, and transformative learning theory. Coding resulted in
38 codes covering 1057 coded segments.

3.3.3. Network Analysis

The rationale for conducting learning experience network [32] analysis was to lever-
age the relationships between struggles, what worked, and alignment of student expe-
riences with principles from learning theories in order to leverage what worked, and
alignment of experiences with theory to address struggles. Also, it is a holistic and expan-
sive method that keeps the complexity of learning intact, rather than reducing learning to
quantitative measures.
Once the reflection papers were coded in MAXQDA, correlation analysis was con-
ducted in order to examine significantly correlated relationships between all possible
pairs of codes. We used Pearson’s 1-tailed tests because the type of network analysis we
conducted required that we only analyze positive correlations. The resulting Pearson corre-
lation values were exported to MS Excel as a symmetrical correlation matrix. In preparation
for network analysis, the matrix was edited to remove all values except those that were
significant at the p < 0.05 level. The matrix was then transformed into a network database
in the UCINET network analysis software.
Creating network maps allows us to analyze the complex interdependencies between
multiple codes, representing the complexity of relationships between different aspects of
learner experiences. Learning experience network maps are crucial in analyzing complex
interdependencies among various codes, representing the intricate relationships between
different facets of learner experiences [33].
In UCINET, we utilized the Girvan–Newman algorithm, a method for detecting clus-
ters within a network based on the concept of betweenness centrality [34]. The algorithm
identifies the edges in a network that have the highest betweenness centrality score, re-
moves these edges, and continues this process iteratively. This results in the division of
Sustainability 2023, 15, 13595 15 of 25

the network into distinct clusters or communities. The Girvan–Newman algorithm is

particularly effective in detecting community structure where there is no prior information
about the communities and provides a powerful tool for identifying and understanding
the organization and relationships within complex networks. We used cluster analysis to
identify distinct patterns in relationships between various aspects of learning experiences
characterized as struggles, what worked, and principles from theory, and to do so in a way
that honors the complexity of learning.
We also calculated the betweenness centrality of every node. Betweenness is a measure
that captures the extent to which a node within a network lies on the shortest path between
other nodes, highlighting the influence a specific node exerts over the interactions of other
nodes in the network [35]. This quantifies the number of times a node acts as a bridge along
the shortest path between other nodes, reflecting the amount of control a particular node
(in our data, a node represents a type of learning experience) exerts over the interactions
of other nodes in the network. We adjusted the node sizes such that the nodes that have
the highest betweenness centrality were the biggest nodes to indicate visually the types of
experience that were most critical and could serve as leverage points when making changes
to the learning experience design.
After constructing the learning experience network map, we identified all the struggles,
strengths, and theory principles within each cluster. Utilizing a table format (see Table 1 for
an example) serves to organize the data more effectively, making the connections among
various data points easier to visualize, and thereby facilitating the formulation of solutions.
The design moves table consisted of rows representing each cluster and four columns,
with the first column representing the struggles encountered by the students. The second
column highlighted the strengths of the learning experience as represented by aspects
that the students reported as working particularly well. The third column captured the
principles from learning theories, showcasing which learning theory principles were most
aligned with the students’ struggles and successes within each cluster.

Table 1. Learning experience network analysis design moves table template.

Struggles Strengths Theory Principles Design Move

(Design move to be
These students’ experiences constructed—leverage the
(black triangle cluster) Some For these students, . . . worked
aligned with the . . . theory theory principles and
students struggled with . . . really well.
principles of . . . strengths in this cluster to
address the struggles)
(Design move to be
These students’ experiences constructed—leverage the
(blue square cluster) Some For these students, . . . worked
aligned with the . . . theory theory principles and
students struggled with . . . really well.
principles of . . . strengths in this cluster to
address the struggles)
(Design move to be
These students’ experiences constructed—leverage the
(red circle cluster) Some For these students, . . . worked
aligned with the . . . theory theory principles and
students struggled with . . . really well.
principles of . . . strengths in this cluster to
address the struggles)
(Design move to be
These students’ experiences constructed—leverage the
(grey diamond cluster) Some For these students, . . . worked
aligned with the . . . theory theory principles and
students struggled with . . . really well.
principles of . . . strengths in this cluster to
address the struggles)

The final column of the design moves table was where we proposed potential solutions
that leveraged the principles from theory and strengths to address the related struggles.
Sustainability 2023, 15, 13595 16 of 25

4. Results
In our study, we utilized the Girvan–Newman cluster analysis method to identify
four distinct clusters of aspects of learning experiences correlated at the p < 0.05 level.
A modularity value (Q) of 0.697 was obtained, indicating a robust community structure.
These clusters represented various facets of student learning, which included the learners’
successes, their struggles, as well as the alignment of their experiences with learning theory
principles. The size of the nodes within each cluster was shaped by their betweenness
centrality values; larger nodes signified greater importance within the network. We cat-
egorized the clusters as the black triangle cluster, blue square cluster, red circle cluster,
and grey diamond cluster. Detailed descriptions of the learning network analyses for each
cluster are presented as follows.
Through Girvan–Newman cluster analysis, four clusters were identified with a Q
value of 0.697, where the size of the nodes signifies their betweenness centrality values
(see Figure 6). Based on the network map presented with labels, the clusters allow us to
visualize the connections between the struggles, effective practices, and learning theories.
In order to create design moves for each struggle in the clusters, the analysis was conducted
in four main steps:
1. In-depth Node Analysis: We began by investigating each cluster in detail. We describe
the nodes and the relationships between them within each cluster, including evidence
from the raw data, such as direct quotations from students. This enables us to
understand the struggles students faced, ensure an evidence-based solution, and
identify the successful learning activities implemented.
2. Relational Analysis: We examined the relationships between struggles, successful
practices, and alignment with theoretical principles. This provides valuable insights
into the learning theory principles that contribute to students’ learning success and
helps identify the link between successful activities and supporting learning theories.
3. Leveraging Success and Theoretical Principles: We used what worked well and the
principles from learning theories to address the struggles in each cluster. The nodes
with the highest betweenness values (i.e., struggles, successful practices, and princi-
ples from theories) can be expected to have the most impact when either successfully
addressed or leveraged to handle related struggles.
4. Addressing Struggles: Lastly, we carefully addressed the learning difficulties students
experienced. If there was a cluster without any struggle nodes but connected to
another cluster, the successful practices and principles from theory nodes in the first
cluster could be used to address struggles in the connected cluster. The learning
theories are utilized to provide solutions for the struggles found in each cluster.

4.1. Black Triangle Cluster Analysis

The black triangle cluster (see Figure 6) is connected to the blue square cluster, in-
dicating that there is a relationship between both clusters. This cluster reflects students’
successes in learning AI when they worked together with their classmates, collaborated
with AI and friends, and enjoyed fun activities with enthusiasm specifically in the activities
where they collaborated with AI and friends to express their imagination. Students in this
cluster did not report experiencing any struggles in their learning. Thus, in the connection
with the blue square cluster, the black triangle cluster takes the role of providing leverage
points in terms of principles from theories and aspects of the experience that worked
particularly well. These leverage points were used to address struggles in the blue square
cluster. The first principle from learning theories included in the black triangle cluster
was individual knowledge construction from cognitive constructivist theory. One of the
students said, “I found out that when I sketch, AI can identify what I want to draw”; another
said, “I enjoy face training [training AI for face recognition] because it makes me laugh and
I can learn new things”. This means students were engaging in an individual cognitive
process when they expressed their ideas and were actively involved in the experience. The
second principle was making (or generativity) from constructionist theory. One student
 to understand the struggles students faced, ensure an evidence-based solution, and
identify the successful learning activities implemented.
Sustainability 2023, 15, 13595 17 of 25
2. Relational Analysis: We examined the relationships between struggles, successful
practices, and alignment with theoretical principles. This provides valuable insights
into the learning theory principles that contribute to students’ learning success and
saidhelps like drawing
that “Iidentify and between
the link coloring because AI helps
successful suggestand
activities images”, suggesting
supporting the presence
learning theories.
of AI as a collaboration partner in the creative process enhanced the students’
3. Leveraging Success and Theoretical Principles: We used what worked well and the engagement.
Furthermore,
principlesAI’sfromability to provide
learning theoriessuggestions offered
to address the a forminofeach
struggles co-constructed
cluster. Thegener-
nodes
ativity, inspiring the students’ imagination and enabling them to explore
with the highest betweenness values (i.e., struggles, successful practices, new ideas
and or
princi-
artistic directions. Through this collaborative interaction with AI, the students are actively
ples from theories) can be expected to have the most impact when either successfully
constructing their knowledge and expressing their imaginations in a meaningful way. The
addressed or leveraged to handle related struggles.
last principle from theory in this black triangle cluster was the social constructivist theory
4. Addressing Struggles: Lastly, we carefully addressed the learning difficulties stu-
principle of scaffolding. One student reported, “Nothing is difficult because I can do it with
dents experienced. If there was a cluster without any struggle nodes but connected
friends”, indicating that these students found the learning process was feasible because
to another cluster, the successful practices and principles from theory nodes in the
they could collaborate with their friends through discussion and sharing of knowledge.
first cluster
Through could be used
these interactions, theto addressreceived
students struggles in the connected
scaffolding support,cluster. The learning
which helped them
theories are utilized to provide solutions for
understand and engage with difficult concepts or problems. the struggles found in each cluster.

Girvan–Newman betweenness
Figure6.6.Girvan–Newman
Figure betweenness values
values with
with labels
labels for
for iteration
iterationfour.
four.
4.2. Black Triangle Cluster Design Moves
4.1. Black Triangle Cluster Analysis
As mentioned before, the black triangle cluster had no struggles. Therefore, the
The black triangle cluster (see Figure 6) is connected to the blue square cluster, indi-
strengths and the theory principles in this cluster can be leveraged to help address the
cating thatstudents
struggles there is ahad
relationship
in the bluebetween both clusters.
square cluster. However, This cluster
this reflects
also means students’
that suc-
there were
cesses in learning AI when they worked together
no design moves specific to the black triangle cluster. with their classmates, collaborated with
AI and friends, and enjoyed fun activities with enthusiasm specifically in the activities
where they
4.3. Blue collaborated
Square with AI and friends to express their imagination. Students in this
Cluster Analysis
cluster did not report experiencing
In the blue square cluster (Figure any6),
struggles in their
there were learning.
2 struggles thatThus, in the
students connection
experienced,
with the blue square cluster, the black triangle cluster takes the role of providing
3 aspects of the learning experience that worked particularly well, and 2 principles leverage
from
points in terms
theories alignedof with
principles
studentfrom theories andThe
experiences. aspects
two of the experience
struggles that workedand
were categorizing par-
ticularly well.
classifying These
data and leverage
students’points
failurewere used tothe
in training address
AI. Forstruggles
instance,in thestudent
one blue square
said,
“The challenging part is when training AI to recognize faces because my face is almost similar to
[my classmate’s]”. Other students said, “I cannot provide data to AI” and “I have difficulty to
use webcam”. Yet another student reported that “The difficult part is when grouping images
Sustainability 2023, 15, 13595 18 of 25

because I am confused about which group to place them in”. Based on the data, we can see that
students tried to build their understanding, but problems arose when conducting data
training with Teachable Machine such as having similar data, having difficulty operating
the camera, and experiencing difficulty in providing good data to the AI. Some students
also had difficulty in the unplugged activity of categorizing and classifying images. At this
level, students gained an understanding about the importance of the data for the effective
result of AI data training. On the other hand, in the same cluster, these students achieved
some successes in recognizing AI and the products of AI, categorizing and classifying data,
and data training (input datasets, training, prediction). For instance, a student said that “I
just learned that training AI to recognize faces is done in such a way that during the training, there
should be no person next to or behind the face being recognized. If the face is covered or if there is
someone next to or behind it, the AI cannot recognize it”. The students’ experiences in this case
aligned with the constructionist principle of tinkering, exploration, and productive failure.
The student in this case isn’t just learning from what works but also from what doesn’t
work, which is a central tenet of the constructionist approach. These failures become
valuable learning opportunities, fostering deeper understanding and problem-solving
abilities. Another student said, “I like when categorizing images successfully because I am
discussing (with friends)”. In this context, the students’ learning experience aligns with the
social constructivist principle of collaborative knowledge construction. This suggests that
students co-constructed understandings of how AI works, which they developed through
collaborative discussions or activities. These students did not passively receive information,
but actively applied it in a real situation and created a product (a trained AI model).

4.4. Blue Square Cluster Design Moves

Based on the interdependent relationships between what worked, principles from
theory, and struggles in session 1, there are several solutions and changes needed for the
next iteration. The first design move will be adding a “think-aloud protocol” as part of the
first activity, a practice rooted in social constructivist theory. This theory emphasizes that
learning happens through social interactions and the active co-construction of knowledge.
By instructing students to vocalize their thoughts while classifying and categorizing images,
they will be encouraged to articulate their thought processes and decision-making criteria.
This not only helps individual students clarify their own understanding for their own
benefit and the benefit of peers, but also allows peers and instructors to provide immediate
feedback and guidance, fostering a richer collaborative learning environment.
The second design move is that the instructor needs to remove overly difficult images
from the categorization activity and replace them with simpler ones. This relates to the
principle of scaffolding in social constructivist learning theory. By starting with easier
tasks, students can build foundational skills and understanding before moving on to more
complex problems. This step-by-step approach with gradual release of structure allows
learners to gradually develop deeper comprehension and skills, reducing frustration and
encouraging engagement.
The third design move will be adding a quick “test” round before the main Teachable
Machines activity, aligning with the constructionist principle of tinkering, exploration,
and productive failure. This provides students with an opportunity to experiment and
make mistakes in a low-stakes setting before they attempt the main task. The test round
acts as a space for trial and error, allowing students to explore the AI tool, understand its
functionality, and learn from any initial mistakes.

4.5. Red Circle Cluster Analysis

Students in the red circle cluster (see Figure 6) experienced struggles while participat-
ing in the evil plan activity. This activity requires students to collaborate with friends to
create an evil plan for using AI in bad ways. The purpose of the activity was to increase
their ethical awareness of the potential negative effects of AI. Some of the students’ quotes
are: “The difficult part when planning a crime is because I don’t like crime”; and another student
Sustainability 2023, 15, 13595 19 of 25

reported “The difficult part is when [looking for] ways to use AI for crime”. On the other
hand, some other students in this cluster experienced successes in the evil plan activity,
as well as the activity involving manipulating data with AI. In the data manipulation
activity, students used Instagram to edit images using AI camera filters. The aspects of
the experience that students in this cluster reported as working particularly well were
in line with principles from learning theories in the cluster. First, in social constructivist
theory, the zone of proximal development (ZPD) refers to the distance between what a
learner can do independently and what they can achieve with assistance. This concept was
evident in the students’ use of Instagram filters. A student reported on what they found
easy: “Using Instagram filters because it is not difficult to use them”. Another student said
“Taking photos using IG filters because it’s fun/cool. It’s easy because assisted by AI and mentors”.
They quickly learned to use these filters independently because it was not difficult, and
they reported that it was fun or cool. Their learning was further supported by AI and
mentors, which could be seen as scaffolding in the ZPD. Second, the transformative theory
principle of developing new perspectives was related to other nodes in this cluster. One
of the students realized that AI can be misused for criminal activities. The student said,
“I just found out that AI can be used for criminal activities”. This shows a shift in perspective
about AI technology. This student was previously unaware of the potential misuse of AI,
but now had a broader understanding, demonstrating transformative learning. Also, in
this cluster was the transformative theory principle of changing beliefs and assumptions.
One student said, “[The activity that worked well was] Deepfakes/filters can change voice, face,
etc., can be used for good or evil”. In this case, the statement about deepfakes and filters being
able to change voices, faces, and possibly being used for good or evil represented a change
in understanding. Before developing this knowledge, students might have assumed these
tools are purely for entertainment. After learning about their potential misuse, however,
their beliefs about these tools’ scope and potential risks evolved.

4.6. Red Circle Cluster Design Moves

In order to address struggles that the students in the red cluster encountered, a set of
design moves were constructed regarding session 3 activities. The evil plan activity will
have an introduction in a form of discussion about an analogy: “How do police detectives
find out who did something bad? Do they have to imagine how bad people think? So, good
people can do good work by imagining what evil genius people might be thinking so that
we can stop them or protect ourselves and our loved ones.” Using this analogy aligns with
the social constructivist theory principles of work within the zone of proximal development
(ZPD) as it offers the necessary assistance to bridge the gap between what students can
understand independently and the new concepts they are trying to grasp. The analogy also
provides a context for the exercise, thus helping to shift the focus away from the problematic
framing of “planning a crime” to thinking strategically about how AI could be misused and
how such misuse could be prevented. This can alleviate students’ discomfort with the idea
of “planning a crime,” making the activity more palatable. Also, the analogy encourages
students to step into the shoes of others (in this case, police detectives), prompting them
to think from a different perspective. This is aligned with transformative theory, which
emphasizes how learning often involves a change in perspective. It can help students see
AI from a different angle—as a tool that, like any other, can be used for both good and evil
purposes, deepening their understanding of the ethical dimensions of AI.

4.7. Grey Diamond Cluster Analysis

Students in the grey diamond cluster (see Figure 6) experienced two kinds of struggles,
which were related to handling technical errors in using computers and having difficulties
in working with classmates or collaborating with friends. Students said, “There’s something
difficult, which is drawing using auto draw, because I don’t like to draw”, and another one said,
“The difficult part is when playing IG [Instagram]”. Yet, another student reported that “The
difficult part is when working in a group”. On the other hand, these students reported positive
Sustainability 2023, 15, 13595 20 of 25

experiences of interacting with AI when they use, play, talk, and ask questions. A student
said, “I like it when AI listens to our voice when we want to ask something because AI provides the
answer to what we ask of it”. These students reported experiences that aligned with the social
constructivist theory principle of mediating artifacts, tools, and technologies. In this case,
AI was acting as a mediating artifact. Another student said, “I just found out that AI can turn
on lights or fans using a Google [home speaker], so we can turn on lights or fans just by speaking”.
In social constructivist theory, artifacts are objects that are used to support learning, and
AI was being used to facilitate the students’ collaborative knowledge construction. The
students were learning by interacting with AI, posing questions to it, and then getting
responses that aided their understanding.

4.8. Grey Diamond Cluster Design Moves

The design moves from the grey diamond cluster will address the struggles students
encountered in both session 2 and session 3. In session 2, the drawing activity will be
changed so that students can sit together with their friends that enjoy drawing. Those that
do not want to draw may sit together with their friends and help by contributing ideas for
drawing. This will avoid any technical difficulties for students who do not enjoy or have
confidence in their technical skills of drawing. Allowing students to group together based
on their comfort level with drawing can alleviate some of the stress for those who are not
confident in their drawing skills. This approach aligns with the social constructivist theory
principle that learning is inherently a product of social interaction. For students who enjoy
drawing, they will serve as mediators, helping their peers understand the process. This
supports the social constructivist principle of learning with mediating artifacts, tools, and
technologies: in this case, the more confident drawers become “human tools” assisting
their peers.
In session 3, the struggles with working with classmates and collaboration showed
up primarily in the evil plan activity. This activity will be modified to start with a short
discussion about why collaboration is more powerful than individual work, framing collab-
oration through the concept of the result being greater than the sum of the parts. Initiating
a discussion on the advantages of teamwork prior to the evil plan activity is expected to
reinforce the idea that collective brainstorming can lead to richer, more innovative, and
more diverse ideas. This is a key component of social constructivist theory, which empha-
sizes that knowledge is co-constructed through social interaction and shared experiences.
This can help students understand that everyone’s input is valuable and that a variety
of perspectives can enhance problem-solving. Another design move is that the evil plan
activity will be modified to let students form their own groups so they can have a more
playful atmosphere with their friends. Allowing students to form their own groups for
the evil plan activity aligns with the social constructivist view of learning as a socially
mediated activity. By working with friends, students may feel more comfortable and
engaged, fostering a more positive and productive collaborative experience. This creates
a supportive environment where students can express their ideas freely, fostering more
effective collaborative learning and mutual support.

5. Discussion
This section discusses the design principles for developing AI literacy in elementary
students, grounded in the study findings and learning theories underpinning the inter-
vention. The results are interpreted in relation to the literature, and strengths, limitations,
contributions, and future directions are described.

5.1. Design Principles for Developing AI Literacy in Elementary School Students

Based on the design moves in iteration four, in order to develop AI literacy for elemen-
tary schools’ students, we need to first ground the intervention based on learning theories,
create safe and comfortable learning environments, and solve the students’ struggles
Sustainability 2023, 15, 13595 21 of 25

Sustainability 2023, 15, x FOR PEER REVIEW 22 of 26

through scaffolding. Figure 7 shows the design principles we derived through categorizing
the design moves constructed for iteration four.

Figure 7. AI
Figure 7. AI literacy
literacy learning
learning design
design principles.
principles.

We believe that to enhance students’ students’ AI AI literacy,

literacy, it is crucial to ground
ground allall interven-
interven-
tions across every iteration in effective
effective learning theories. These theories include cognitive
constructivism,
constructivism, socialsocialconstructivism,
constructivism,constructionism,
constructionism,and andtransformative
transformative learning
learning the-
theory,
ory, in which
in which student–student
student–student collaboration
collaboration and and student–AI
student–AI collaboration
collaboration are central
are central to
to the
the learning
learning process.
process. ThisThis
typetype of interaction
of interaction will increase
will increase students’
students’ creativity
creativity and em-
and empower
power
students students
to make to stuff
makeinstead
stuff instead of merely
of merely remembering
remembering things.things. The unique
The unique and and sup-
support-
portive interactions
ive interactions thatthat were
were created
created in in
thethe study
study servedasasthe
served thefoundation
foundationfor foraa safe
safe and
comfortable learning environment, enabling enabling students
students to to select
select their
their own
own groups
groups or or pairs,
pairs,
thus promoting optimal learning, and allowing them to freely express express their
their ideas.
ideas. This
learning environment
environmentmust mustallow
allowtinkering,
tinkering,exploration,
exploration,and and productive
productive failure
failurein in
learning.
learn-
Students
ing. will also
Students willfeel
alsosafe
feeltosafe
voicetotheir
voice ideas,
theirlearn from
ideas, learntheir mistakes,
from and ask questions
their mistakes, and ask
without
questions any hesitation.
without Should students
any hesitation. Shouldencounter any difficulties
students encounter in their learning
any difficulties in theirjourney,
learn-
the instructor needs to employ scaffolding techniques. These could
ing journey, the instructor needs to employ scaffolding techniques. These could involve involve asking ques-
tions that guide towards solutions. For instance, when students struggled
asking questions that guide towards solutions. For instance, when students struggled with with failures in
training the AI model during a Teachable Machine activity, the instructor
failures in training the AI model during a Teachable Machine activity, the instructor pro- provided tailored
scaffolding
vided by asking
tailored guiding
scaffolding byquestions such as, questions
asking guiding “What do you suchthink would do
as, “What happen
you ifthink
you
removed
would any blurry
happen if youpictures
removed andanyreplaced them with
blurry pictures andbetter quality
replaced themphotos?” and “Could
with better quality
you checkand
photos?” if there
“Couldis too
youmuch
checksimilarity
if there isbetween
too much categories?” This enabled
similarity between students
categories?” to
This
troubleshoot issues and incrementally build the skills needed to train
enabled students to troubleshoot issues and incrementally build the skills needed to train an effective model.
an effective model. In order to reduce cognitive load, the instructor will need to employ
framing to help students interact with difficult concepts, for example by constructing
Sustainability 2023, 15, 13595 22 of 25

In order to reduce cognitive load, the instructor will need to employ framing to help stu-
dents interact with difficult concepts, for example by constructing analogies. As a result,
students are more likely to engage deeply with the material and overcome any problems
they encounter.

5.2. Interpretation of the Results in the Context of the Literature Review

The findings of this study are in line with Long and Magerko’s [17] claim that pro-
gramming ability is not a prerequisite for AI literacy. Our results demonstrate that students
could effectively understand AI concepts and interact with AI technologies without nec-
essarily having any programming skills. This highlights the feasibility of integrating AI
literacy into broader STEAM education, aligning with the arguments of Lee et al. [13] and
How and Hung [14], who have emphasized the importance of AI literacy for preparing
students for future AI-empowered roles. Additionally, the data aligns with the principle
that generative AI, as defined by Gong et al. [10], can foster creativity in STEAM education.
The students in our study demonstrated increased creativity when using generative AI
tools, affirming the notions of Epstein et al. [12] who found generative AI’s significant ap-
plication in the creative arts. Our findings also reinforced the importance of understanding
the ethical and societal implications of AI, as suggested by Zhang et al. [15]. Participants
showed a growing awareness of AI’s broader societal impacts, illustrating the success of an
AI literacy approach that incorporates ethical considerations. Moreover, the use of construc-
tivist, constructionist, social constructivism, and transformative learning theories [20–23] in
shaping the learning experience and design moves generated positive outcomes. Consistent
with these theories, students constructed meaning from their individual and collaborative
experiences, were active in the learning process, and demonstrated changes in their as-
sumptions and beliefs about AI. The results are also in line with Touretzky et al.’s [18] claim
that hands-on experiences with AI technologies are vital for fostering AI literacy. Our data
suggest that hands-on engagement enhanced the students’ ability to critically evaluate AI
technologies and articulate the ethical or societal impacts of AI applications.

5.3. Strengths and Limitations of the Study

The study design was grounded in established learning theories, such as cognitive
constructivism, social constructivism, and transformative learning theory, providing a
robust framework for understanding and analyzing students’ experiences and outcomes.
The use of learning experience network analysis to help address students’ problems in
learning activities turned out to be an effective evidence-based approach to crafting practical
solutions to real challenges. However, there are some limitations in this study. First, the
results of this study may not be generalized to all student populations, as the participants
were a specific group of students in Indonesia and the source data were qualitative in
nature. More research will be needed to validate these findings across diverse student
populations. The study was also limited to the immediate context and did not include any
long-term follow-up to assess the sustainability of the improvement in students’ AI literacy.
It will be interesting to track these students over time to see how their AI literacy develops
in the long term.

5.4. Contribution to the Field of AI Literacy and STEAM Education

This research study offers several valuable contributions to the field of AI literacy and
STEAM education. Firstly, it emphasizes the importance of grounding AI education in
multiple established learning theories. It showcases how cognitive constructivism, social
constructivism, constructionism, and transformative learning theory can be used to guide
the design and iteration of educational interventions. These theoretical foundations provide
students with a more holistic and profound understanding of AI, fostering both theoretical
knowledge and practical skills.
Secondly, this research identifies the necessity of a safe and comfortable learning
environment in promoting optimal learning outcomes. The study demonstrates that
Sustainability 2023, 15, 13595 23 of 25

students are more likely to engage deeply with the material when they feel at ease to
voice their ideas, embrace learning from their mistakes, and ask questions without fear of
retribution. This highlights the role of the learning environment as not just a backdrop but
a crucial element in students’ AI literacy development.
Thirdly, our findings emphasize the role of scaffolding in supporting students’ learn-
ing. This is especially important in a complex and rapidly evolving field like AI, where
students may struggle with the technical intricacies and theoretical concepts. By providing
appropriate support at the right moments, educators can help students overcome these
challenges, thereby fostering deeper learning and comprehension.
In addition, the study highlights the potential of integrating AI into more traditional
STEAM subjects, demonstrating how AI can be a tool for enhancing learning in these
areas. This not only boosts students’ AI literacy but also helps them see the real-world
applications of their STEAM knowledge, which can enhance motivation and engagement.
Finally, this research contributes a model for designing AI literacy interventions in
elementary school settings. By articulating specific “design moves”, it offers a framework
that other educators can adapt and apply in their own contexts. This represents a sig-
nificant step towards creating more effective and inclusive AI literacy development for
younger learners.

5.5. Future Directions for Research and Practice

Future research directions can expand the demographic scope of this study to include
diverse student populations from different geographic regions, educational levels, and
socio-economic backgrounds. Implementing long-term follow-up studies could also help
understand the impact of AI literacy improvements over time. Educators and policymakers
should also consider incorporating the learning experience network analysis approach into
their teaching methodologies, especially in areas with a significant emphasis on AI literacy.
Furthermore, considering the rapidly changing nature of AI technologies, continuous
revision and updating of AI literacy instructional design would be crucial to keep pace
with industry trends and future careers.

6. Conclusions
The study found that meaningful collaboration and the use of analogies and scaffold-
ing support grade 5 students in understanding AI concepts without needing programming
skills, thereby supporting the integration of AI literacy into broader STEAM education
and highlighting students’ creativity and awareness of AI’s ethical implications. Ground-
ing AI literacy education in theories like cognitive constructivism, social constructivism,
constructionism, and transformative learning can create an optimal learning environment.
Techniques like scaffolding and analogies enhance engagement, while integrating AI with
traditional STEAM subjects boosts literacy and real-world application [16,20–23]. Foster-
ing AI literacy from an early age prepares children for an AI-empowered future, with
early exposure contributing to responsible and ethical use, potentially mitigating ethical
issues associated with AI [18]. This study offers a tentative set of design principles for
incorporating AI literacy in elementary schools, paving the way for future research and
practice. Expanding the demographic scope and including long-term studies could deepen
understanding of AI literacy development strategies’ impact over time.

Author Contributions: Conceptualization, S.C.R. and J.P.D.; Methodology, S.C.R. and J.P.D.; Formal
analysis, S.C.R.; Investigation, S.C.R.; Data curation, S.C.R.; Writing—original draft, S.C.R. and J.P.D.;
Writing—review & editing, Y.C.L.; Visualization, S.C.R.; Supervision, Y.C.L.; Funding acquisition,
Y.C.L. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: The study was conducted according to the Declaration of
Helsinki and approved by the Human Research Ethics Committee of the Education University of
Hong Kong guidelines (protocol code: 2019-2020-0062; date of approval: 7 November 2019).
Sustainability 2023, 15, 13595 24 of 25

Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data supporting the results reported in this study are not publicly
available. The data contain confidential information that is subject to privacy restrictions as per
institutional and ethical guidelines. To maintain the anonymity and confidentiality of participants
involved in the study, we are unable to share the raw data.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Chitturu, S.; Lin, D.Y.; Sneader, K.; Tonby, O.; Woetzel, J. Artificial Intelligence and Southeast Asia’s Future. Singapore Summit.
2017. Available online: https://www.mckinsey.com/my/~/media/mckinsey/featured%20insights/artificial%20intelligence/
ai%20and%20se%20asia%20future/artificial-intelligence-and-southeast-asias-future.pdf (accessed on 30 July 2023).
2. Gordon, G.; Spaulding, S.; Westlund, J.K.; Lee, J.J.; Plummer, L.; Martinez, M.; Das, M.; Breazeal, C. Affective Personalization of
a Social Robot Tutor for Children’s Second Language Skills. In Proceedings of the AAAI Conference on Artificial Intelligence,
Phoenix, AZ, USA, 12–17 February 2016; Volume 30.
3. Beirl, D.; Rogers, Y.; Yuill, N. Using Voice Assistant Skills in Family Life. In Proceedings of the Computer-Supported Collaborative
Learning Conference, CSCL, Lyon, France, 17–21 June 2019; International Society of the Learning Sciences, Inc.: Bloomington,
Indiana, 2019; Volume 1, pp. 96–103.
4. Oranç, C.; Ruggeri, A. ‘Alexa, Let Me Ask You Something Different’ Children’s Adaptive Information Search with Voice Assistants.
Hum. Behav. Emerg. Technol. 2021, 3, 595–605. [CrossRef]
5. Hsu, T.C.; Abelson, H.; Lao, N.; Chen, S.C. Is It Possible for Young Students to Learn the AI-STEAM Application with Experiential
Learning? Sustainability 2021, 13, 11114. [CrossRef]
6. Kim, N.J.; Kim, M.K. Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. Front.
Educ. 2022, 7, 142. [CrossRef]
7. Narahara, T.; Kobayashi, Y. Personalizing Homemade Bots with Plug & Play AI for STEAM Education. In SIGGRAPH Asia 2018
Technical Briefs; Association for Computing Machinery: New York, NY, USA, 2018; pp. 1–4. Available online: https://dl.acm.org/doi/
abs/10.1145/3283254.3283270 (accessed on 30 July 2023).
8. Leonard, N. Entanglement Art Education: Factoring ARTificial Intelligence and Nonhumans into Future Art Curricula. Art Educ.
2020, 73, 22–28. [CrossRef]
9. Russell, S.J.; Peter, N. Artificial Intelligence: A Modern Approach, 3rd ed.; Pearson Education, Inc.: Hoboken, NJ, USA, 2016.
10. Gong, C.; Jing, C.; Chen, X.; Pun, C.M.; Huang, G.; Saha, A.; Nieuwoudt, M.; Li, H.X.; Hu, Y.; Wang, S. Generative AI for Brain
Image Computing and Brain Network Computing: A Review. Front. Neurosci. 2023, 17, 1203104. [CrossRef] [PubMed]
11. Cui, H.; Wang, C.; Maan, H.; Pang, K.; Luo, F.; Wang, B. ScGPT: Towards Building a Foundation Model for Single-Cell Multi-Omics
Using Generative AI. bioRxiv 2023. [CrossRef]
12. Epstein, Z.; Hertzmann, A.; Investigators of Human Creativity; Akten, M.; Farid, H.; Fjeld, J.; Frank, M.R.; Groh, M.; Herman, L.;
Leach, N.; et al. Art and the Science of Generative AI. Science 2023, 380, 1110–1111. [CrossRef] [PubMed]
13. Lee, I.; Ali, S.; Zhang, H.; DiPaola, D.; Breazeal, C. Developing Middle School Students’ AI Literacy. In Proceedings of the 52nd
ACM Technical Symposium on Computer Science Education, Virtual Event, 13–20 March 2021; pp. 191–197.
14. How, M.L.; Hung, W.L.D. Educing AI-Thinking in Science, Technology, Engineering, Arts, and Mathematics (STEAM) Education.
Educ. Sci. 2019, 9, 184. [CrossRef]
15. Zhang, H.; Lee, I.; Ali, S.; DiPaola, D.; Cheng, Y.; Breazeal, C. Integrating Ethics and Career Futures with Technical Learning to
Promote AI Literacy for Middle School Students: An Exploratory Study. Int. J. Artif. Intell. Educ. 2022, 33, 290–324. [CrossRef]
[PubMed]
16. Aguilera, D.; Ortiz-Revilla, J. STEM vs. STEAM Education and Student Creativity: A Systematic Literature Review. Educ. Sci.
2021, 11, 331. [CrossRef]
17. Long, D.; Magerko, B. What Is AI Literacy? Competencies and Design Considerations. In Proceedings of the 2020 CHI Conference
on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–16.
18. Touretzky, D.; Gardner-McCune, C.; Martin, F.; Seehorn, D. Envisioning AI for K-12: What Should Every Child Know about AI?
In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33,
pp. 9795–9799.
19. Barab, S.; Squire, K. Design-Based Research: Putting a Stake in the Ground. J. Learn. Sci. 2004, 13, 1–14. [CrossRef]
20. Piaget, J. The Development of Thought: Equilibration of Cognitive Structures; Viking Press: Los Angeles, CA, USA, 1977.
21. Vygotsky, L.S. Mind in Society: The Development of Higher Mental Process; Harvard University Press: Cambridge, MA, USA, 1978.
22. Kafai, Y.B. Constructionism. In The Cambridge Handbook of the Learning Sciences; Keith Sawyer, R., Ed.; Cambridge Handbooks in
Psychology; Cambridge University Press: Cambridge, UK, 2006; pp. 35–46. [CrossRef]
23. Mezirow, J. An overview of transformative learning. In Contemporary Theories of Learning: Learning Theorists—In Their Own Words,
2nd ed.; Routledge, Taylor & Francis Group: Abingdon, UK, 2018; pp. 114–128.
24. Anderson, T.; Shattuck, J. Design-Based Research: A Decade of Progress in Education Research? Educ. Res. 2012, 41, 16–25.
[CrossRef]
Sustainability 2023, 15, 13595 25 of 25

25. Hoadley, C.M. Methodological Alignment in Design-Based Research. Educ. Psychol. 2004, 39, 203–212. [CrossRef]
26. Puntambekar, S. Design-Based Research (DBR). In International Handbook of the Learning Sciences; Routledge: Abingdon, UK, 2018;
pp. 383–392.
27. Google. Teachable Machine. Available online: https://teachablemachine.withgoogle.com/ (accessed on 4 July 2023).
28. Google. Quick, Draw! Available online: https://quickdraw.withgoogle.com/ (accessed on 5 July 2023).
29. Google. AutoDraw. Available online: https://www.autodraw.com/ (accessed on 6 July 2023).
30. America’s Got Talent. Simon Cowell Sings on Stage?! Metaphysic Will Leave You Speechless|AGT 2022. 2022. Available online:
https://youtu.be/mPU0WNUzsBo (accessed on 7 July 2023).
31. Metro, T.V. Lebih Jahat Dari Hoax|15 Minutes Metro TV. 2020. Available online: https://www.youtube.com/watch?v=4W0
MekwgbnE (accessed on 5 July 2023).
32. Donaldson, J.P.; Han, A.; Yan, S.; Lee, S.; Kao, S. Learning experience network analysis for design-based research. Inf. Learn. Sci.
2023, in press.
33. Borgatti, S.P.; Everett, M.G.; Freeman, L.C. Ucinet for Windows: Software for Social Network Analysis. Harv. MA Anal. Technol.
2002, 6, 12–15.
34. Girvan, M.; Newman, M.E. Community Structure in Social and Biological Networks. Proc. Natl. Acad. Sci. USA 2002, 99,
7821–7826. [CrossRef] [PubMed]
35. Freeman, L.C. A Set of Measures of Centrality Based on Betweenness. Sociometry 1977, 40, 35–41. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.