Integration of AI-based Teaching

Assistant BOT (TA BOT) in Academic

Courses to Facilitate Learning among
Large Student Groups
S. L Rohanadeera, K. P. Hewagamage
About the team
❖Prof K. P. Hewagamage
• Principal Investigator – TA BOT project
• Senior Professor in Computer Science
• University of Colombo School of Computing

❖Sudam Rohanadeera
• Research Assistant-TA BOT project
• University of Colombo School of Computing
 Introduction
• Teaching Assistant (TA) – Important role in Higher
Education Institutes (HEI)
• TA supports depend on the number students
assigned to a TA
• TA should be subject specialist as well as a good
communicator with the students
• TA builds the lifeline between learner and education
• Finding Good TAs and large numbers when intake
increases, almost impossible task
• We present digital TA backed by AI Technology as
alternative service provider in HEI
Problem Statement/Motivation
• Large academic courses often lack personalized
assistance, impacting student comprehension and
motivation.
• The traditional ratio of one teaching assistant to 15-25
students in such courses poses limitations.
• We present a solution to this challenge through the
integration of Generative AI-based Teaching Assistant
BOT.
Background Study
• Initially reliant on rule-based systems, chatbots have
evolved into sophisticated AI-powered entities capable of
natural language processing and understanding.
• Prominent examples include Jill Watson at Georgia Tech,
an AI-powered virtual teaching assistant, and GSU's
'campus coach', showcasing the efficacy of chatbots in
enhancing student support and engagement.
Background Study

• Recent advancements have seen the integration of large

language models into educational contexts, leveraging
vast amounts of text data to generate human-like
responses and provide interactive learning experiences.
Limitations of Existing
Conversational AI Tools
• While existing tools like Chat GPT and Bard have gained
popularity, they fall short in providing contextually
relevant responses in an educational setting.
• Their context-independence often results in answers
disconnected from the course content, syllabi, and
references.
 Solution Overview
• Introduction of an AI-based Teaching
Assistant Bot (TA BOT)
• TA BOT will be a real-time online
alternative to traditional teaching
assistants, providing immediate and
personalized support tailored to
individual student needs in large
classes.
TA BOT Advantages
• Our Teaching Assistant BOT leverages a Fine-Tuned Large
Language Model (LLM) gpt-3.5-turbo designed for
contextual relevance.
• ‘gpt-3.5-turbo’ is a powerful LLM by OpenAI
• Fine-Tuning ensures that the TA BOT will align with course
content, syllabi, and references,
• The TA BOT ensures accurate and contextually meaningful
responses, addressing the limitations of existing
conversational AI tools.
 Role of TA BOT
• Recognizing the difficulties students
face in comprehending new subjects,
we present the Teaching Assistant BOT
as a real-time online alternative.
• This tool offers immediate,
personalized support tailored to
individual student needs, filling the
gap left by traditional teaching
assistants in large classes.
Demo: ChatGPT
Demo: ChatGPT
Demo: TA BOT
Demo: TA BOT
Development Process – Core
Concepts
❖Large Language Model
• Large Language Models (LLMs) are powerful artificial
intelligence models designed to understand and generate
human-like text at a sophisticated level.
• These models, such as 'gpt-3.5-turbo' from OpenAI, possess
an extensive knowledge base gained from diverse internet
text.
• LLMs play a pivotal role in natural language processing,
enabling applications like our Teaching Assistant BOT to
comprehend and respond to user queries with context-
awareness and linguistic finesse.
Development Process – Core
Concepts
• Fine Tuning
• Fine-tuning is the process of refining a pre-trained language
model to specialize its understanding and generate
contextually relevant responses within a specific domain or
subject.
• The primary purpose of fine-tuning is to adapt a general-
purpose language model, such as 'gpt-3.5-turbo,' to the
specific requirements and nuances of a targeted field, in our
case, Software Engineering.
Development Process – Core
Concepts
• Retrieval Augmented Generation
• Retrieval Augmented Generation (RAG) combines information
retrieval with text generation, utilizing a knowledge base for
relevant data retrieval and a text generation model, such as
'gpt-3.5-turbo,' to create contextually informed responses.
Development Process – Core Concepts
• Finetuning Data Set
• A large set of questions and answers.
• In the context of LLM : ‘prompts’ and ‘reponses’
Development Process – Core
Concepts
• Prompt Engineering
• Prompt Engineering involves crafting effective queries or
prompts to optimize interactions with Large Language Models
(LLMs).
• It's the art of formulating questions or requests that yield
accurate and insightful responses from the LLM, such as 'gpt-
3.5-turbo.'
Development Process –Tools
• Llama Index:
• An open-source python library
• Supports LLM related developments
• Retrieval Augmented Generation
• Training Finetuned Models
• Fine Tuning dataset generation
• https://docs.llamaindex.ai/en/stable/
Methodology-Dataset Generation
• Fine-Tuning a LLM requires a quality and comprehensive
dataset.
• Adopting the Retrieval Augmented Generation (RAG)
methodology, supported by the LlamaIndex framework,
we created a comprehensive dataset using the
reference book of a Software Engineering course.
• This approach ensures the TA BOT's ability to deliver
precise and relevant information.
Dataset Creation
• The dataset, sourced from the reference book of a
Software Engineering course, forms the backbone of
the TA BOT's knowledge.
• Contextually relevant responses are guaranteed by
aligning the AI model with course materials, syllabi, and
references.
Development Process-Fine
Tuning
• Selected ‘gpt-3.5-turbo’ as the base model, as it is
optimized for ‘Chat’ type conversations
• Fine-tuned the base model for using the generated data
set.
• This step was crucial in tailoring the AI model to the
specific needs of our academic context
TA BOT Persona
• In delivering contextual interactions based on subject
matter questions, the TA BOT adopts the persona of a
Teaching Assistant.
• The persona has been embedded into TA BOT with
Prompt Engineering’ techniques.
• This ensures a relatable and engaging experience for
students, fostering a more effective learning
environment.
Prohibiting Academic Offences
• To maintain ethical use, the TA BOT is designed to
prohibit academic offenses and malpractices.
• This measure ensures the tool's integrity and aligns
with the principles of academic integrity
Versatility of TA BOT
• Beyond the dataset, the TA BOT showcases its
versatility by providing responses even when specific
knowledge is not present.
• This highlights its adaptability and potential to handle a
wide range of Software Engineering-related queries.
Plan for Sustainability
• Alignment with UN Sustainable Development
Goals (SDGs):
• Goal 4: Quality Education - Ensuring inclusive and equitable
quality education through personalized support.
• Goal 10: Reduced Inequalities - Providing accessible
assistance regardless of location or time zone.
• Sustainability in Technical Approach:
• Utilization of Fine-Tuning Method - Reducing environmental
impact by optimizing computational resources and minimizing
carbon footprint.
Plan for Sustainability
• Sustainability Measures:
• Continuous Learning - Improving responses through
interaction and feedback.
• Scalability - Adapting to cover more subjects and integrate
with more platforms.
• Accessibility - Ensuring availability to all students, considering
geographic restrictions and language barriers.
Plan for Commercialization
• Market Analysis: Understanding potential customers and their
needs.
• Product Development: Continuous improvement and
expansion.
• Partnerships: Collaborating with educational institutions and
platforms.
• Pricing Strategy: Subscription-based or usage-based models.
• Marketing and Sales: Promoting through various channels.
• Customer Support and Service: Providing training and prompt
assistance.
• Continuous Improvement: Updating based on feedback and
advancements in AI.
Preliminary Tests and Student
Feedback
• Preliminary tests confirm the TA BOT's ability to furnish
precise, high-quality answers to Software Engineering-
related queries.
• Positive student feedback underscores its potential to
enhance the educational experience, marking a
promising solution.
Conclusion and Future
Directions
• AI-based Teaching Assistant BOT offers a promising
solution to sustain educational quality amidst resource
constraints and escalating student demands.
• The tool is publicly accessible for experimentation at
http://chat.ucsc.cmb.ac.lk, paving the way for future
advancements in AI-based education.
• We wish to develop tools to integrate TA BOT with LMS
discussion forums and collaboration apps like Slack,
WhatsApp.
Conclusion and Future
Directions
• Expand the TA BOT for the other subject domains
• Law
• Indigenous Medicine /Ayurvedic Medicine
Thank you !