Southern Journal of Arts & Humanities Vol.

02, Issue 01, 2024

Impact of ChatGPT on Student Learning Outcomes: A Systematic

Review
Fajar Aziz, Abdul Bari khan
Institute of Southern Punjab, Multan, Pakistan

Abstract: To design a sustainable education system is a challenge. Enhancing the efficacy and
sustainability of blended learning systems is a promising area for artificial intelligence (AI). This study
focuses on the integration of Generative Pre-trained Trans-former (ChatGPT) on student learning outcomes
with a blended learning system. The objective of the study is to investigate academic performance and
enhance the sustainability of the education system. A qualitative research design was used to collect data
from 120 students and teachers of BS level Multan. The initial thematic quantitative work has suggested
satisfactory progress with ChatGPT, however,no qualitative exploration has been carried out to evaluate the
process. For this purpose, in-depth qualitative interviews were conducted with the participants. The data
were analyzed by using a thematic analysis approach which shows a combination of positive and negative
effects of the program. Furthermore, the findings include an improvement in the ChatGPT skills of students
on the one hand while lacking creativity and thinking abilities on the other hand. Despite its drawbacks,
ChatGPT is thought to be useful for increasing learning efficiency and productivity by respondents. As a
result, using ChatGPT might be seen as an attractive learning tool as long as one uses it critically and
makes sure the information it provides.

Keywords: Chatgpt, Higher Education System, Productivity, Student Learning Outcomes

Email: fajarfatimaaziz@gmail.com

1. Introduction:

The field of education has seen significant education sector. They have been utilized to
changes as a result of the advancement of support and help users in a variety of
information and communication technology scenarios. Computer software created to
in recent years. Online learning is now more have interactions with users is called a
flexible and accessible because of chatbot Sandu & Gide, (2019). Generative
technological advancements. Many Pre-trained Trans-former (ChatGpt) is one
educational institutions now primarily use e- of the increasingly common types of
learning to give students a more engaging chatbots. Human language can be
and productive learning environment. understood and processed by Chat GPT, an
Artificial intelligence (AI) is expanding artificial intelligence model based on natural
quickly in tandem with technological language. With this paradigm, the chatbot
advancements. Chatbots are one component can converse with people naturally and
of AI that has drawn interest from the respond in a way that is pertinent and
18
Fajar et al

appropriate for the context. Chat GPT has leverage AI chatbots' ability to enhance
the potential to enhance student engagement student learning outcomes. However, given
and learning outcomes in technology-based the limits of this study, more research in the
learning in an educational setting. Students future might examine the use of AI chatbots
can use Chat GPT to interact with the in a variety of educational settings and topic
chatbot as they would with a classmate, and areas to gain a deeper understanding of their
receive immediate feedback, assistance, and effects.
explanations regarding the course materials
1.1 Background of the Study
Shoufan, (2023). This study is new since it
Artificial Intelligence (AI) has expanded
examines the application of Chat GPT in
rapidly, bringing about new forms and
technological learning, compares its effects
changes in many aspects of modern life. One
through experimentation, and investigates
of its more recent inventions that has an
the potential applications of chatbots in the
impact on teaching and studying English is
classroom. The primary goal of this study is
ChatGPT. It came about as a result of the
to determine whether using Chat GPT
extensive body of AI research that
improves student learning outcomes. This
technology-based language instruction has
research may have useful ramifications for
been using for many years.
the advancement of education in the digital
age. Furthermore, this study demonstrates a The early research on integrating technology

dedication to gaining a deeper into EFL and ESL programs concentrated on

comprehension of chatbots' function in learning methodologies, motivation,

educational settings, while candidly attitudes, and integration. The introduction

recognizing its limitations and suggesting of ChatGPT, dubbed "a giant auto-complete
avenues for future research Lee et al., machine," has opened up new research

(2022). It is anticipated that this study's avenues for the study of technological

findings will contribute to a better integration in teaching second and foreign

understanding of the impact of Chat GPT on languages, including English Ali et al.,

technology-based learning Nasim et al., (2023a). Some voices claim that AI-

(2022). The results have the potential to language bots are "incapable of generating

significantly impact the evolution of insights or deep analysis," despite efforts to

education in the digital age and offer integrate AI into language programs as a

educators useful guidance on how to best solution to current issues Esra & Sevilen,
19
Fajar et al

(2021). This study shows, a group of 44 30, 2022, with the expectation that it will
authors who contended that "the AI model's have a substantial impact on all facets of our
output cannot be taken at face value. They society. The potential educational
recommended that all AI-based outputs be implications of this natural language
thoroughly examined to avoid mistakes, processing method remain unclear though.
omissions of important details, or the An argument from a different study
making of unrelated assertions. One of the suggests that ChatGPT's potential may have
more recent AI innovations, ChatGPT, may a major impact on educational learning
provide false results if the existing data objectives, assessment and evaluation
needs to be updated or completed. Perhaps procedures, learning activities, and
authors would be able to reconsider before evaluation procedures. The rapid rise of
going so far as to name ChatGPT as a co- ChatGPT, which looked at artificial
author if such assertions were made. Before intelligence (AI) in teaching and learning in
ChatGPT, which emerged from a lengthy higher education, demonstrates that how AI
history of AI research, technology was is accelerating the pace at which higher
employed in language learning and teaching education services are made available
settings for a variety of reasons. A outside of the classroom. The study covered
preliminary investigation of the use of the potential integration of AI into higher
technology in English language programs education, as well as the immediate and
examined language school students' access long-term effects on several fields within
to real-world communication channels. The higher education Zhai et al., (2021).
use of computers and other contemporary Similarly, the study looked at whether
technology by students is very beneficial. artificial intelligence could be used to
The study demonstrated how using summarize literature review articles. The
technology could increase students' authors claim that AI has the potential to
motivation, involvement, and level of transform academic publication by relieving
engagement in language learning Ali et al., human laborers of some of their workload
(2023)b. and allowing academics to focus on their
1.2 Emergence of ChatGPT research Lockey et al., (2021). They also
used ChatGPT to create an academic paper
Open AI launched ChatGPT, a general-
and used plagiarism detection software to
purpose conversation chatbot, on November
make sure that ChatGPT's content was
20
Fajar et al

original. Researchers can use ChatGPT to 2. What is the impact of ChatGPT on

help them write articles that are accurate, content improvement of student
methodical, logical, and enlightening, learning outcomes?
according to the findings of another study
3. Research Method and Technique:
that used ChatGPT. The study suggested
rewriting the learning objectives to Investigating the effects of ChatGPT on

emphasize creativity and critical thinking English language learners and then

above generic skill development and to let determining whether there are differences in

students use AI technologies to complete the views of teachers and students is a

subject-domain tasks. Students can logical first step. This study situates

outsource their assessment work with constructivist paradigm was used to explore

ChatGPT. the perception of BS level students and

It's crucial to move from general to specific teachers of Multan. The researcher

and from global to local in these converses with the subjects for this aim and

investigations. gathers information by using logic.

The researcher conducts several contacts
1.3 Objectives
with the participants to carry out these
1. To investigate the impact of
conversations, extracting meaning and
ChatGPT on the academic
integrating it to create information.
performance of students and
Therefore, in-depth qualitative interviews
teachers.
with the important stakeholders were
2. To investigate the impact of conducted to investigate the influence of
ChatGPT on content improvement Chat GPT. Data was continuously tracked
of student learning outcomes and evaluated throughout the process to
(SLOs). create context for the study participants'
responses.
1.4 Research Questions
1. What is the impact of ChatGPT on 2.1 Hypothesis development
the academic performance of
students and teachers? Students who use Chat GPT may benefit in

Figure 1 shows the conceptual framework

for the investigation. The study put out the
following hypothesis, which regression
21
Fajar et al

analysis will be used to test: a few ways, Academic improvement of students. Ho: The use
most notably in terms of their drive to learn, of ChatGPT has an impact on Content

as was discussed in the previous section. improvement for student`s

H0: The use of ChatGPT has an impact on the

Student Learning Outcomes (Content

The use of
Improvement, Academic Improvement)
Chat GPT

Figure 1: Research Framework

2.2 Data Analysis:

The management and interpretation of data The data analysis procedure employed in
comprised the analysis of data. Transcribing this study is depicted in the image below;
the audio recordings and sorting the
handwritten field notes was the first stage
Siregar et al., (2023). After that, every
transcript was read aloud several times to
ensure that the data was well understood
before analysis. Data analysis is done using
a thematic analysis approach Creswell,
(2009).

22
Fajar et al

Figure 2. Source (Creswell, 2009)

Population and Sample:

The population of the study is entire education sectors including, institutions,

students from BS level District Multan. universities, colleges, and affiliated

However we selected 120 students as a campuses. The respondent's demographic

sample, they were from different higher are shown in Table 1, 2.

23
Fajar et al

Table 1: A Demographic Table of Academic Improvement

Category Variable n(Frequency)
male 55(87%)
Academic
female 40(73%)
Improvement
teachers 25(90%)

Table 2: A demographic table of Content Improvement

Category Variable n(Frequency)
male 60(82%)
Content female 50(72%)
Improvement
teachers 10(84s%)

2.3 Findings: 4. Conclusion:

The interview has been analyzed by using The field of artificial intelligence (AI) has
thematic analysis. After reviewing each advanced significantly in recent years and
interview, in cases where many participants is now applied in many different fields. AI
submitted identical information using has the power to completely change the
different words, the best wordings were education industry by enhancing the quality
chosen, and the codes of each participant of teaching and learning, personalizing
who contributed that information were listed instruction, simplifying administrative
in front of the literal text. tasks, and creating new research
The analysis of the interview revealed the opportunities. Artificial intelligence (AI)
following responses from the participants: technologies, such as Chat GPT, and
intelligent tutoring systems, can help
1. Relevance
students with writing assignments, offer
2. Acceptability
comments and revision recommendations,
3. Challenges
and improve their learning experience. This
4. Suggestions
paper discusses the impact of ChatGPT on
5. Improving learning Outcomes
student learning outcomes with a
6. Making a sound education base
moderating effect of about 67%. According
7. Use of technology for learning
to the study, younger students used Chat
GPT more frequently than older students

24
Fajar et al

and male students used it more frequently

than female students.

References:

Ali, J. K. M., Shamsan, M. A. A., Hezam, T. Nasim, S. F., Ali, M. R., & Kulsoom, U.
A., & Mohammed, A. A. (2023). (2022). Artificial intelligence
Impact of ChatGPT on learning incidents & ethics a narrative
motivation: teachers and students' review. International Journal of
voices. Journal of English Studies in Technology, Innovation and
Arabia Felix, 2(1), 41-49 Management(IJTIM), 2(2), 52-64.
Esra, M. E. Ş. E., & Sevilen, Ç. (2021). Sandu, N., & Gide, E. (2019, September).
Adoption of AI-Chatbots to enhance
Factors influencing EFL students’
student learning experience in higher
motivation in online learning: A
education in India. In 2019 18th
qualitative case study. Journal of
International Conference on
Educational Technology and
Information Technology Based
Online Learning, 4(1), 11-22
Higher Education and Training
Lee, Y. F., Hwang, G. J., & Chen, P. Y.
(ITHET) (pp. 1-5). IEEE.
(2022). Impacts of an AI-based chat
Shoufan, A. (2023). Exploring Students
bot on college students’ after-class
Perceptions of CHATGPT:
review, academic performance, self-
Thematic Analysis and Follow-Up
efficacy, learning attitude, and
Survey. IEEE Access.
motivation. Educational technology
Siregar, F. H., Hasmayni, B., & Lubis, A.
research and development, 70(5),
H. (2023). The Analysis of Chat
1843-1865
GPT Usage Impact on Learning
Lockey, S., Gillespie, N., Holm, D., &
Motivation among Scout Students.
Some, I. A. (2021). A review of trust
International Journal of Research
in artificial intelligence: Challenges,
and Review, 10(7), 632-638.
vulnerabilities, and future directions
Zhai, X., Chu, X., Chai, C. S., Jong, M. S.
Y., Istenic, A., Spector, M., ... & Li,
Y. (2021). A Review of Artificial
Intelligence (AI) in Education.

25