Development of an Intelligent Travel Chatbot for

Vietnamese Tourists Using Deep Learning

Techniques
Nguyen Le Anh Quan, Nguyen Duy Son, Lai Hoang Minh Phuc
School of Computer Science
FPT University
Ho Chi Minh City, Vietnam
nguyenleanhquan2005@gmail.com

Abstract—In the context of increasing travel demand, tourists and efficiently. This chatbot is built on the DeepSeek R1
frequently face challenges in finding reliable and suitable infor- model, an advanced artificial intelligence platform, combined
mation when planning trips to new destinations. The absence of a with the K-Nearest Neighbors (KNN) algorithm to analyze
flexible, support tool creates a significant barrier, affecting their
overall experience. The primary objective is to facilitate easier and filter relevant information based on two main factors: cost
and more effective trip planning. This proposal puts forward and the user’s personal preferences. This combination enables
an intelligent chatbot system for tourism based on state-of- the chatbot to provide personalized travel recommendations,
the-art Retrieval-Augmented Generation (RAG) architecture, a helping users save time and have experiences that better align
hybrid framework designed to mitigate the factual unreliability with their desires. The project not only meets practical needs
of standalone language models. This architecture strategically
synergizes the generative capabilities of the DeepSeek R1 large but also contributes to the application of modern technologies
language model with a robust K-Nearest Neighbors (k-NN) in the tourism industry.
retrieval algorithm. To specialize the model for the domain,
the DeepSeek R1 generator was fine-tuned on a meticulously II. R ELATED W ORK
curated dataset of 1,000 question-answer pairs, covering key A. Evolution of Intelligent Agents
aspects of Vietnamese tourism such as destinations, activities,
and local costs, thereby enhancing its ability to produce fluent The origin of intelligent conversational agents, or chatbots,
and contextually aware dialogue. In parallel, the k-NN retriever developed in the 1960s by Joseph Weizenbaum’s ELIZA.
component actively searches a dedicated knowledge base, filtering This pioneering system was designed to simulate mental
and retrieving the most relevant informational chunks based on health–focused dialogue, using a constrained knowledge base
multi-faceted criteria including user-defined cost constraints and and predefined conversational pathways to navigate or prevent
travel preferences. This knowledge-grounding process ensures
that the final output is not only conversational but also factually conversational impasses. While foundational, early iterations
accurate. The resulting system is a sophisticated chatbot capable like ELIZA were characterized by their reliance on scripted re-
of delivering highly personalized recommendations and precise sponses, offering no real understanding or adaptability beyond
answers, providing a practical 24/7 support tool that significantly predefined responses. Another early example was PARRY,
enhances user satisfaction and showcases a scalable application developed in 1972 by psychiatrist Kenneth Colby, which
of modern AI in the tourism sector
Index Terms—Chatbot, Travel, Tourism, Vietnamese NLP, simulated a person with paranoid schizophrenia and was more
Python, Dialogue System advanced in its conversational strategy. These early systems
were fundamentally rule-based, their responses dictated by a
I. I NTRODUCTION predefined script.
In the context of the growing demand for travel, most people The following decades were marked by steady, step-by-step
aspire to take a trip to relax after a long period of hard work. improvements. The 1990s brought the rise of the internet and
Many seek out new destinations to gain exciting experiences. with it, chatbots like A.L.I.C.E. (Artificial Linguistic Internet
However, exploring unfamiliar places is not always easy, as Computer Entity), which utilized the Artificial Intelligence
travelers often lack sufficient information about the locations Markup Language (AIML) to create more complex rule-
they plan to visit. Therefore, through this course project, we based conversational flows. These bots were capable of more
aim to develop a chatbot that acts as a virtual tour guide, intricate interactions but were still fundamentally limited by
helping people access travel destination information more their pre-programmed knowledge and inability to learn from
flexibly. This chatbot will support tourists in searching for new interactions.
information anytime, anywhere, ensuring they have the best Throughout the late 20th century and early 2000s, chatbot
possible experience on their journey development was mainly limited to experimental academic
As part of a course project, we has developed a travel settings or basic customer service applications. It wasn’t
chatbot designed to help users plan their trips more easily until the late 2010s that the technology gained significant
commercial traction, particularly within the tourism industry. data is then passed to a Dialogue Manager, which performs
Initial applications focused on providing instant responses to state tracking, manages conversational context, and executes a
common inquiries—such as attraction operating hours, book- policy to determine the optimal subsequent action. This action
ing confirmations, and local suggestions—through predefined is then rendered into a coherent, human-readable response by
answer sets. Despite being limited in personalization, these a Natural Language Generation (NLG) model. The incorpo-
systems represented a major milestone in bringing conversa- ration of advanced deep learning architectures, most notably
tional agents into practical, real-world use. the Transformer (Vaswani et al., 2017), has been particu-
larly transformative. The self-attention mechanisms inherent
B. Traditional Approaches to intelligent tourism chatbot to Transformers allow the model to dynamically weigh the
The initial wave of chatbot technology was dominated significance of all words in an input relative to each other,
by systems that operated on explicit, pre-programmed logic, enabling a superior comprehension of nuance, ambiguity, and
offering control and predictability at the expense of natural long-range dependencies within the conversational history.
interaction. The foundational architectures for early conversa-
tional agents within the tourism sector were predominantly ar- III. M ETHODOLOGY
chitected around deterministic, rule-based frameworks. These Our chatbot is retrieval-based. User input is matched against
systems were, in essence, practical implementations of Finite- a list of pre-defined questions. The response will be filtered
State Machines (FSMs), which model dialogue as a directed through the K-Nearest Neighbors (KNN) algorithm before
graph of discrete, predefined states. Within this model, each delivering the most accurate and optimal reply.
state corresponds to a specific juncture in the conversation,
and transitions between states are triggered by user inputs that A. Data Collection
match explicit, hard-coded rules. The logic governing these The dataset used for fine-tuning the DeepSeek R1 model
transitions typically relied on rudimentary techniques such as has been manually compiled. It consists of 3,000 questions
keyword matching or regular expressions (regex), wherein user and answers that travelers frequently encounter when visiting
utterances were scanned for recognized lexical patterns (e.g., famous tourist destinations such as Da Nang, Da Lat, Nha
IF input CONTAINS ”book flight” THEN TRANSITIONTO Trang, and more. This dataset includes key features, entertain-
flightbookingstate). ment activities at tourist sites, entrance ticket prices, estimated
While this paradigm afforded a high degree of predictability dining expenses, and other essential travel information. With
and control for linear, task-oriented dialogues, it was con- this rich dataset, the chatbot will assist users in easily access-
strained by significant and inherent limitations. The principal ing information, enhancing their travel experience in a more
constraint was its lack of robustness; any input deviating from convenient and efficient manner.
the predefined patterns would precipitate an interaction failure
or a generic fallback response (e.g., ”I’m sorry, I don’t un- B. Method
derstand”). Moreover, these systems were inherently stateless The model fine-tuning process was conducted based on the
with respect to conversational history, lacking the capacity to DeepSeek R1 language model, an advanced deep learning
recall information from previous user turns. This resulted in model with robust natural language processing capabilities.
disjointed interactions. Scalability also presented a formidable This model was retrained on the tourism dataset to optimize its
challenge, as the introduction of new functionalities or conver- ability to generate accurate, contextually relevant, and highly
sational paths necessitated a manual, labor-intensive expansion localized responses. The fine-tuning process involved adjusting
and reconfiguration of the entire state diagram, a process the model’s parameters to enhance its accuracy and semantic
susceptible to logical errors and inconsistencies [1], [2]. understanding of questions related to tourism in Vietnam.
To improve the quality of responses, the K-Nearest Neigh-
C. Modern Advances in intelligent tourism chatbot bors (KNN) algorithm was employed to select optimal answers
A paradigm shift in the development of conversational based on two primary criteria: cost and user preferences.
agents has been catalyzed by the integration of data-driven Specifically, each response in the dataset was labeled with
methodologies, principally from the fields of Natural Language features such as estimated cost levels (low, medium, high)
Processing (NLP) and machine learning (ML). Modern agents and preference types (adventure, relaxation, culture, cuisine).
operate on a sophisticated processing pipeline that substitutes The KNN algorithm utilized Euclidean distance to identify the
probabilistic models for rigid rules. User input is first pro- responses closest to the user’s requirements, ensuring that the
cessed by a Natural Language Understanding (NLU) module, answers were not only accurate but also aligned with the user’s
which executes two critical sub-tasks: budget and personal preferences. The K parameter was tested
Intent Classification: The user’s primary objective (e.g., and optimized through cross-validation techniques to achieve
findhotel, planitinerary) is inferred by classifying the utterance the highest performance.
into a pretrained category. Entity Extraction: Key parameters
and informational units—such as dates (next April), locations IV. I MPLEMENTATION
(Japan), and user preferences (history, good food)—are iden- The implementation of the tourism question-answering sys-
tified and extracted from the unstructured text. This structured tem was carried out in two main phases: fine-tuning the
DeepSeek R1 model and integrating the K-Nearest Neighbors
(KNN) algorithm to select optimal responses. The dataset,
comprising 3,000 question-answer pairs related to tourism at
famous destinations in Vietnam, was preprocessed to ensure
consistency and quality. Preprocessing steps included text nor-
malization (removing unnecessary punctuation, converting to
lowercase), deduplication, and context labeling (e.g., location,
type of tourism). The dataset was then split into three subsets:
80
The DeepSeek R1 model was fine-tuned using transfer
learning techniques. Specifically, the pre-trained model’s pa- Fig. 2. Train/learning rate
rameters were retained as a foundation, while the final layers
were adjusted to align with the tourism dataset. Training was
conducted using the cross-entropy loss function and optimized
with the Adam algorithm, with an initial learning rate of 2e-5.
After the DeepSeek R1 model generated potential responses,
the KNN algorithm was implemented to filter and select the
most suitable answers based on cost and user preferences.
Each response in the dataset was represented as a feature
vector, incorporating numerical values for cost and one-hot
encoded preferences (adventure, relaxation, culture, cuisine).

V. R ESULT Fig. 3. Train/global step

After training, the graph shows the loss dropping sharply

from 1.5 to below 0.5 in just 50 steps, reflecting an efficient sharply in the first 50 steps, reflecting the rapid adjustments
learning process in the early stage. Afterwards, the loss of the model as it learned the data. However, from steps 50
stabilized around 0.2 until step 200, indicating that the model to 200, the values fluctuated around 0.6 to 0.8 with some
has converged well without any clear signs of overfitting. variations, particularly displaying small peaks near steps 100
and 150, suggesting certain irregularities in the optimization
process, possibly due to inconsistent batch sizes or complex
data. Finally, the gradient norm stabilized around 0.2, a
positive indication that the model has reached a converged
state with only minor gradient updates.

Fig. 1. Train/loss

The train/learning rate chart shows the learning rate decreas-

ing steadily from about 0.00015 to nearly 0 over approximately
200 training steps (train/global step). This gradual decline
reflects the use of a learning rate scheduler, possible linear or Fig. 4. Train/grad norm
cosine, that helps the model progressively adjust the learning
rate to avoid abrupt changes as it nears convergence. The
drop from 0.00015 to nearly 0 indicates that the model starts VI. C ONCLUSION
with a relatively high learning rate to quickly adapt to the Based on the training data collected, it can be concluded
data and then gradually lowers it to fine-tune the parameters that the training strategy was effectively implemented. Initially,
more precisely. This is typically a sign of an effective training the loss rapidly dropped from 1.5 to below 0.5 within 50
strategy, especially when considered alongside the earlier loss steps, demonstrating that the model quickly adapted to the
graph (which decreased from 1.5 to 0.2). data. Concurrently, the learning rate steadily decreased from
The train / grad norm chart shows that the gradient norm 0.00015 to nearly 0 over 200 steps, allowing the model to
decreased from about 1.4 to roughly 0.4 over 200 training transition from a high initial learning rate to a phase of careful
steps (train/global step). Initially, the gradient norm decreased parameter fine-tuning as convergence approached. Although
the gradient norm exhibited some slight fluctuations between [16] K. Boes, D. Buhalis, and A. Inversini, “Smart tourism destinations:
steps 50 and 200, with minor peaks near steps 100 and Ecosystems for tourism destination competitiveness,” International
Journal of Tourism Cities, vol. 2, no. 2, pp. 108–124, 2016. [Online].
150—possibly due to varying batch sizes or the complexity of Available: https://doi.org/10.1108/IJTC-12-2015-0032
the data—it ultimately stabilized at around 0.4. Together, these [17] D. Garcı́a-Sanz, E. Onaindia, X. Delorme, G. Gonçalves, and V. Martin,
indicators confirm that the model learned effectively, showed “A big data and time series analysis technology-based multi-agent
system for smart tourism,” Applied Sciences, vol. 8, no. 6, p. 875,
no clear signs of overfitting, and reached a stable state of 2018. [Online]. Available: https://www.mdpi.com/2076-3417/8/6/875
convergence. [18] K. Wüst and K. Bremser, “Artificial intelligence in tourism through
chatbot support in the booking process—an experimental investigation,”
R EFERENCES Tourism and Hospitality, vol. 6, no. 1, p. 36, 2025. [Online]. Available:
[1] D. Jurafsky and J. H. Martin, Speech and Language Processing: An In- https://www.mdpi.com/2673-5768/6/1/36
troduction to Natural Language Processing, Computational Linguistics,
and Speech Recognition, 3rd ed., ser. Speech and Language Processing.
Pearson, 2025.
[2] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances
in neural information processing systems, vol. 30, 2017.
[3] M. Orden-Mejı́a and A. Huertas, “Analysis of the attributes
of smart tourism technologies in destination chatbots that
influence tourist satisfaction,” Current Issues in Tourism,
vol. 25, no. 17, pp. 2854–2869, 2022. [Online]. Available:
https://www.tandfonline.com/doi/full/10.1080/13683500.2021.1997942
[4] W. Wörndl, C. Koo, and J. L. Stienmetz, Eds., Information
and Communication Technologies in Tourism 2021: Proceedings
of the ENTER 2021 eTourism Conference, January 19–22, 2021.
Cham: Springer International Publishing, 2021. [Online]. Available:
https://library.oapen.org/handle/20.500.12657/46110
[5] V. Kasinathan, M. H. A. Wahab, S. Z. S. Idrus, A. Mustapha, and
K. Z. Yuen, “Aira chatbot for travel: Case study of airasia,” Journal of
Physics: Conference Series, vol. 1529, no. 2, p. 022101, 2020.
[Online]. Available: https://iopscience.iop.org/article/10.1088/1742-
6596/1529/2/022101
[6] G. M. Majid, I. Tussyadiah, and Y. R. Kim, “Exploring the
potential of chatbots in extending tourists’ sustainable travel
practices,” Journal of Travel Research, 2024. [Online]. Available:
https://journals.sagepub.com/doi/full/10.1177/00472875241247316
[7] F. J. B. Ballina, “Smart tourism destination, physical experience and
rural tourism,” Smart Tourism, vol. 1, no. 2, 2020. [Online]. Available:
https://aber.apacsci.com/index.php/st/article/view/1697
[8] M. Ari, “Impact of smart tourism technologies on tourist
experience satisfaction and sustainable destination image:
Evidence from istanbul,” 2025, unpublished master’s thesis,
Tomas Bata University, Czech Republic. [Online]. Available:
https://digilib.k.utb.cz/handle/10563/50519
[9] Y. Zhang, M. Sotiriadis, and S. Shen, “Investigating the impact of smart
tourism technologies on tourists’ experiences,” Sustainability, vol. 14,
no. 5, p. 3048, 2022. [Online]. Available: https://www.mdpi.com/2071-
1050/14/5/3048
[10] S. Barandoni, F. Chiarello, L. Cascone, E. Marrale, and
S. Puccio, “Automating customer needs analysis: A comparative
study of large language models in the travel industry,”
arXiv preprint arXiv:2404.17975, 2024. [Online]. Available:
https://arxiv.org/abs/2404.17975
[11] S. Mercer, S. Spillard, and D. P. Martin, “Brief analysis of deepseek r1
and its implications for generative ai,” arXiv preprint arXiv:2502.02523,
2025. [Online]. Available: https://arxiv.org/abs/2502.02523
[12] L. Benaddi, C. Ouaddi, A. Souha, A. Jakimi, M. Rahouti,
M. Aledhari, D. Oliveira, and B. Ouchao, “Seq2seq model-
based chatbot with lstm and attention mechanism for enhanced
user interaction,” arXiv preprint arXiv:2501.00049, 2024. [Online].
Available: https://arxiv.org/abs/2501.00049
[13] DeepSeek-AI, “Deepseek-r1: Incentivizing reasoning capability in llms
via reinforcement learning,” arXiv preprint arXiv:2501.12948, 2025.
[Online]. Available: https://arxiv.org/abs/2501.12948
[14] A. Galvão, F. B. e Abreu, and J. J. de Melo, “Towards a consensual
definition for smart tourism and smart tourism tools,” in Smart Life and
Smart Life Engineering. Springer Nature, 2025, pp. 1–17. [Online].
Available: https://arxiv.org/abs/2402.05557
[15] F. Clarizia, F. Colace, M. D. Santo, M. Lombardi, L. Pascale, and
D. Santaniello, “A context-aware chatbot for tourist destinations,”
in 2019 15th International Conference on Signal-Image Technology
Internet-Based Systems (SITIS). IEEE, 2019, pp. 348–354. [Online].
Available: https://ieeexplore.ieee.org/document/9069580