InfoMate - Virtual Guide for Museums & Historical Places

Contextual Report

B.Sc. (Hons) in Computer Science and Software Engineering

University of Bedfordshire

Student Name: M. K. R. Perera

Student ID: 2023006
Unit Name: Research Methodologies and Emerging Technologies (SLIIT)

Date of submission: 24th of May 2021

 Abstract

With the advancement of technology, many things have converted to computer-based systems
to make our lives much easier and more interesting. But the way people visit a museum, or a
historically valuable place has never been advanced with emerging technologies. Even though
there are few projects that is there to improve the experience, those are not financially feasible
nor practical from the user’s standpoint. The proposed project is an android based application
that will act as a virtual agent to make visiting museums and historical places much more
interesting and informative for people. It can also act as a learning tool for the students. Most
of the people when visiting a museum or a historically important place, visitors do not have
any idea about the place, how it ended up here, origin story or the historical value of the thing
they are visiting. Even though there are written information on the place most of the people do
not even bother reading them. “InfoMate” is the proposed project that will be solving this little
problem in an interesting way. Its main goal is to provide an interesting way to be informative
and entertaining at the same time. When a person with the proposed app installed come near a
museum item or historical place and tap on the relevant place which has an NFC chip or scan
the QR code. The virtual character automatically starts to read interesting and informative
information about the place or the thing. There is also a simple chatbot designed to give users
more interesting and useful information on a pleasing manner.

Keywords: Virtual Agent, Chatbot, NFC, QR Code

1
 Table of Contents

Abstract ...................................................................................................................................... 1
Table of Contents ....................................................................................................................... 2
List of Abbreviations ................................................................................................................. 4
1. Introduction ........................................................................................................................ 5
1.1. Problem Statement ...................................................................................................... 5
1.2. Project Realisation....................................................................................................... 5
1.3. Scope of the Project..................................................................................................... 6
1.4. Aims and Objectives ................................................................................................... 6
1.5. Feasibility of the Project ............................................................................................. 6
1.6. Structure of Contextual Report ................................................................................... 7
1.6.1. Introduction .......................................................................................................... 7
1.6.2. Literature Review................................................................................................. 7
1.6.3. Project Plan .......................................................................................................... 7
1.6.4. Planning Artefact ................................................................................................. 7
2. Literature Review............................................................................................................... 8
2.1. Introduction ................................................................................................................. 8
2.2. Virtual Character as a Conversational Agent .............................................................. 8
2.3. Chatbot based on Artificial Intelligence...................................................................... 9
2.4. Near Field Communication Technologies................................................................. 11
2.5. Optical Label based Scanning (QR Codes) ............................................................... 12
2.6. Similar Applications.................................................................................................. 13
2.7. Research Gap............................................................................................................. 14
3. Project Plan ...................................................................................................................... 15
3.1. Work-Breakdown Structure ...................................................................................... 15
3.2. Project Timeline (Gantt Chart) .................................................................................. 16
4. Planning Artefact ............................................................................................................. 17
4.1. Requirements for "InfoMate” .................................................................................... 17
4.2. Design and Implementation ...................................................................................... 18
4.2.1. Use Case Diagram.............................................................................................. 18
4.2.2. Development Methodology ............................................................................... 19
4.3. Testing ....................................................................................................................... 20
4.4. Evaluation.................................................................................................................. 20
5. References ........................................................................................................................ 21

2
 Table of Figures

Figure 1:Indoo.rs Museum (Indoo.rs, 2018) ............................................................................ 13

Figure 2:Royal BC Museum Wayfinding App (Jennifer Kite-Powell, 2012) ......................... 13
Figure 3:Work-Breakdown Structure ...................................................................................... 15
Figure 4:Gantt Chart ................................................................................................................ 16
Figure 5:Use-Case Diagram ..................................................................................................... 18
Figure 6: Agile Methodology (360logica, 2020) ..................................................................... 19

3
 List of Abbreviations

• NFC - Near Field Communication

• RFID - Radio-Frequency Identification
• QR - Quick Response
• SDK - Software Development Kit
• API - Application Programming Interface
• AI - Artificial Intelligence
• NASA - National Aeronautics and Space Administration
• ARPANET - Advanced Research Projects Agency Network
• NLP - Natural Language Processing
• P2P - Peer-to-Peer

4
 1. Introduction

1.1. Problem Statement

When a person visits a museum or a historically important place, most of the time visitors don’t
have any idea about the place, how it ended up here, the origin story or the historical value of
the thing they are visiting. Even though there is written information on the place most of the
people don’t even bother reading them. Still to date, the way people visit a museum, or a
historically valuable place has never been advanced with emerging technologies.

Even though there are few projects that is there to improve the experience, those are not
financially feasible nor practical from the user’s standpoint. Also, most of the current
applications aren’t available to use by tourists as most of them are still in the testing phases and
some of the projects has been suspended too.

1.2. Project Realisation

The idea of creating a mobile application to act as a simple tour guide to help improve the
experience of tourists was done by conducting thorough research on this subject matter and the
lack of existing applications on this matter. It was found that none of the existing applications
doesn’t have all the functionalities & features in one place as of the proposed application.

The proposed application will be developed using Flutter SDK with Android Studio to build
user interfaces and almost all the application front-end along with Python to create an
integrated chatbot. By using mentioned languages and techniques, the proposed application
will be developed.

5
 1.3. Scope of the Project

“InfoMate” is an artificial intelligence based assistant to guide tourists in a way that creates the
illusion as if they are with a very helpful companion to help them guide a museum or a
historically valuable place and help them learn more about the place or the thing more
interesting and informative way.

1.4. Aims and Objectives

The main aim of this virtual guide is to develop an informative and helpful artificial
intelligence-based assistant to help guide through museums and historically important places.

Objectives are as follows:

• To identify existing applications of chatbots systems and virtual assistants to help you
guide and evaluate.
• To find and analyse different approaches to develop the proposed system and evaluate
them individually to select the most suitable approach.
• To design and develop a more user friendly and easily understandable virtual guide.
• To design a virtual guide application to find never seen places and keep track of them.
• To compare the proposed application with other existing ones’ and identify their
strengths and update upon them.

1.5. Feasibility of the Project

Creating a virtual guide system for a country like Sri Lanka is technically and operationally
feasible in most circumstances, because of attraction of tourists and the current pandemic
situation.

Also, the cost of the hardware (NFC Tags) to implement the system is low. Averaging about
0.25 USD per tag. And the data can be stored offline in the app and there will not be extra
server storage costs.

6
 1.6. Structure of Contextual Report

This report consists of four main chapters. A brief description of those four chapters will be
below consisting of Introduction, Literature Review, Project Plan and Planning Artefact.

1.6.1. Introduction

This chapter consists of a brief introduction about the proposed project which will give the
reader an overall brief idea of the proposed project which includes the scope of the project,
aims and objectives and the feasibility of the project

1.6.2. Literature Review

The literature review, which is done in order gather information as a secondary method by
referring various journals, articles and research papers. This will give the reader an idea about
the existing work related to the proposed system.

1.6.3. Project Plan

The project plan gives the reader an idea of what tasks that need to be done and how much time
will it take to complete the project. By using a work breakdown structure which will consists
of all the major tasks and a Gantt chart, the reader will get a better understanding of the
timeframe of the project.

1.6.4. Planning Artefact

This chapter consists of a brief description of the development methodology and the reasons
behind it and the design implementation. It also consists of how the testing and evaluation is
done to ensure the optimal state of the project.

7
 2. Literature Review

2.1. Introduction

An extensively detailed review of a few existing work and research that other people have done
that is related to the proposed project. The purpose of this document is to justify the financial
and technical feasibility of the proposed project and give a comprehensive review of the
existing research and products. This covers the areas that related to virtual character
development, AI chat bot, near field communication technologies, scanning of optical label-
based scanning, use of new APIs and challenges that occurs when developing this system.

2.2. Virtual Character as a Conversational Agent

A virtual character or an avatar is an artificially made human like virtual person to interact with
people but can only act within its virtual boundaries. Virtual character is also a type of software
agent combined with an added visual interaction.

Virtual character (avatar) history goes way back into 1970s. With the introduction to use
ARPANET between universities, Steve Colley and Howard Palmer who works at NASA,
invented a multiplayer game called MazeWar (Kellie, 2007).

Museums could use digital characters to guide visitors to tours based on their interests.
Different degrees of interaction and forms of communication have been explored in online
museum web sites with digital characters. This study's focus is on a tour guide character who
provides tour detail. The tour guide character speaks through audio production and gestures to
provide tour details and answers to users' textual inputs. Furthermore, the tour guide character
tries to provide adaptive directions, tell entertaining stories, and encourage users to participate
in the virtual tour. (Almeida et al., 2003)

8
 2.3. Chatbot based on Artificial Intelligence

In today’s world computers and mobile phones play a very important role in day-to-day life of
our society. Chatbots, also known as virtual assistants, are modern technologies that aim to
make human-computer interaction easier. A chatbot is a program designed to counterfeit a
smart communication on a text or spoken ground (M. Dahiya, 2017). Chat bots are also known
as “online human-computer dialog system[s] with natural language” (Jack Cahn, 2017).

A chatbot based on an Artificial Intelligence can start conversations with a person by emulating
natural language using web applications, mobile applications, and messaging application or
even through a phone call. The main reason for a chatbot to be important is that most of the
time chatbots are described as one of the most advanced ways to interact between humans and
machines. A chatbot is simply the evolution of a natural language processing (NLP) based
Question Answering machine. One of the most common examples of Natural Language
Processing in use is different enterprises' end-use applications formulating answers to the
questions in a natural language. (Expert.ai Team, 2020)

From a functionality perspective, modern chatbot frameworks can be categorised into these
three categories: Dialogic Agents, Rational Agents and Embodied Agents (Sansonnet, 2006)

A. Dialogic Agents: The main requirement of a dialogic agent is to understand what the
user is saying or asking. Dialogic agents are provided with a textual or oral input from
the user, which is then analysed with tools such as natural-language-processing and
using such tool it generates appropriate responses accordingly (Sansonnet, 2006).

B. Rational Agents: This type of chatbot agents were built on the foundation of accessing
an external base of knowledge and common sense so that it can provide the function of
competence and answering users’ questions. Rational agents store context-specific
information such as names and such (Sansonnet, 2006).

9
 C. Embodied Agents: Built on providing the function of presence and liveliness. In early
days this is regarded as optional. But in today’s world this function has proven to be
very crucial to a chatbot. In the earliest days’ bots were given names to satisfy that. But
today by using language tricks, ordinary users are encouraged to build trust and give
impression of an embodied agent (Sansonnet, 2006).

Alan Turing attributed the first conceptualisation of the chat bot by asking, “Can machines
think?” in 1950 (Turing, 1950). Turing envisioned the issue as an "imitation game" (now
known as the Turing Test), in which an "interrogator" asked human and computer subjects
questions with the intention of identifying the human. We say the computer will think if the
human and machine are indistinguishable (Turing, 1950).

The use of the chat bot will play a very important role in the proposed system as the chatbot is
one of the most interactive part in the application.

10
 2.4. Near Field Communication Technologies

Near Field Communication (NFC) is a wireless technology which works using a magnetic
induction field to transfer a small amount of data between two devices within a short distance.
NFC is based on a standardised technology which is used for Radio Frequency Identification
(RFID). NFC technology has been developed by Philips Semiconductors and Sony
Corporation. The two companies’ joint venture focused to create a standardise technology
based on “Smart Cards Mifare” which was developed by Philips and “FliCa” which was
developed by Sony (Juho & Eric, 2012).

NFC is an open standard. NFC uses radio waves in range of 13.56 MHz with a maximum range
of 20 centimetres and speeds up to 424 kbit/s (Romeo et al., 2007). Wireless readers are used
to pickup signals from other radios which are nowadays embedded into almost all smartphones.
Active and Passive are the two modes of NFC technology that exist.

A. Active Communication Method: As the name suggests, in active communication

method the initiating device and the target device generates its own electromagnetic
field which consumes electricity. In this method success rate of a successful read is very
high and the two devices do not need to be as close as possible (Romeo et al., 2007).

B. Passive Communication Method: In the passive communication method, only the

initiating device generates an electromagnetic field, and the target device responds by
modulating on the existing field (Romeo et al., 2007).

Also, there are three main unique ways that NFC devices can operate in. Reader/Writer mode
& Peer-to-Peer mode & Card Emulation mode.

The most viable way to use the proposed system is in passive communication method with
reader/writer mode.

11
 2.5. Optical Label based Scanning (QR Codes)

Innovative technologies have been significantly changed people’s lives including the
capabilities of storing and transfer data using a virtual environment. This is where inventors
come up with a robust method of storing data in the two-dimension barcode called Quick
Response (QR) code. Considering the QR code uses a two-dimensional barcode is the
important connecting link between the virtual world and life (Wang, 2018). It is a type of code
that stores information in a set of pixels using one square-shaped grid. The best tool for decode
or decrypt automatically QR code with aid of using a smartphone even without entering
previous information or data during the decrypting phase (Emmanuel, 2020).

Nowadays QR code has been used in a variety of fields to store information as to track or
identify. A publishing company introduced the method of the tour guide system that uses QR
code to present as information and service for tourist might want to know more details about
object or place. Due to the fast method of decoding QR codes are used as a payment technology
for tourist to enhance their satisfaction (Lou, 2017). Using this technique of storing information
in barcode might help to develop advanced systems that use the smartphone camera to provide
fast and readily information.

12
 2.6. Similar Applications

Royal BC Museum Wayfinding App (Wifarer, 2012), Art Camera Museum (Navigine, 2018),
Moscow Kremlin Museum – Armoury (Navigine, 2019) & Indoors (Indoo.rs, 2018) are the
existing similar applications that are available and were related to the proposed project. These
applications consist of features such as indoor location tracking, interactive content and
assistant for people with special needs. Most of the applications are very outdated and not user
friendly. Most systems only use beacons placed in indoors to track location. These applications
are used to understand the features that current systems are offering and the ones they are
missing.

User profile & content sharing, getting notifications, recommendations, history of places
visited, chatbot functionality, virtual character and not being limited to only one place are the
missing features from current applications that the proposed system will be built on.

Figure 1:Indoo.rs Museum (Indoo.rs, 2018) Figure 2:Royal BC Museum Wayfinding App (Jennifer Kite-Powell, 2012)

13
 2.7. Research Gap

According to a research conducted by Xueni Pan, Marco Gillies and Mel Slater of University
College of London people mostly like to hear someone read them out to loud as a virtual
character. Even the technology has evolved very far from when the previously mentioned
applications were developed, the applications aren’t caught up to today’s technological
advancements. Almost all the applications had similar functionalities.

The major differences between the proposed application and the current applications are the
use of a chatbot, individual user curated recommendations and the virtual character. Moreover,
content sharing, push notifications, recommendations and a history of places visited are
missing from the current applications.

14
 3. Project Plan
3.1. Work-Breakdown Structure

The work-breakdown structure shows the way the project is organized with all the tasks
mentioned. This will give the reader a better understanding of how the project is being
managed.

"InfoMate"
Mobile Application

2. Requirement
3. System 4. Complete
1. Planning Gathering & 5. Documentation
Development Evaluation
Analysis

1.1 Choosing a 2.1 Review 3.1 Front-end 3.2 Back-end 4.1 Testing the 5.1 Project
2.2 Analysis
topic researches development development AI chatbot proposal

2.1.1 Find 3.1.1 Sketching 3.3

1.2 Feasibility 4.2 Testing the 5.2 Contextual
exisiting and designing Implementing
check final application Report
applications the interface the code

2.1.2 Writing 3.1.2 User 3.3.1 Developing

1.3 Identify 4.3 Get 5.3 Reflective
the literature interface and training the
techniques feedback Report
review development AI chat bot

3.1.3 Testing & 3.3.2 Developing

5.4 Theisis
debugging of the Virtual
Report
interfaces Avatar

3.1.4 Design, 3.3.3 Testing &

5.5 Design
develop and test debugging the
Project Poster
virtual avatar code

Figure 3:Work-Breakdown Structure

15
 3.2. Project Timeline (Gantt Chart)

The Gantt chart shows the overall project timeline. This contains all the dates and time periods
scheduled for each specific task.

Figure 4:Gantt Chart

16
 4. Planning Artefact

4.1. Requirements for "InfoMate”

“InfoMate” will be developed to be run on Android smartphones with source code based on
Flutter and Python. All the libraries and dependencies used to develop will be included in a
separate package. Android Studio and Visual Studio Code will be used to develop the project.

User Requirements:
• Android Smartphone (Android 7.0 or higher. Recommended Android 11)
• Internet Connection

Developer Requirements:
• Windows / Linux / Mac Operating System for the development environment (All the
dependencies and libraries are cross platform compatible)
• Android Studio (latest version recommended)
• Visual Studio Code (latest version recommended)
• Java Development Kit (latest version recommended)
• Node.js (latest version recommended)
• Python (latest version recommended)

17
4.2. Design and Implementation

4.2.1. Use Case Diagram

Figure 5:Use-Case Diagram

18
 4.2.2. Development Methodology

Agile development methodology will be used to develop this project. Agile development
methodology is well known for its flexibility as in any phase of the development changes can
be made or new features can be added while in the development. At the end of the development
this will ensure a more sophisticated application.

The agile technique is shown in the diagram below. All the is done in a circular manner until
the application is finished, including planning, production, testing, and everything else. This
approach allows for the development of a bug-free application.

Figure 6: Agile Methodology (360logica, 2020)

19
 4.3. Testing

For the testing of the project, following methods will be used other than the continuous testing
throughout the development phase.

• Acceptance Test Driven Development - specifies the functionality of the system

during each testing scenario.
• Exploratory Testing – focuses testing on a working software rather than doing
individual tests.

Goal of all this testing is to create a bug free application. Special testing will be done with the
chat bot as it is the most interactive part of the application. Also, a quality assurance test will
also be conducted to further polish the application experience.

4.4. Evaluation

Evaluation is a very important step when releasing anything to public. “InfoMate” application
will be thoroughly evaluated before releasing as it will make sure that the application will be
working optimally as it intended to be. A good evaluation will make out all the small problems
that wasn’t identified during the testing period and it will make sure that the all functions are
functional as intended.

20
 5. References

1. Cahn, J., 2017. CHATBOT: Architecture, design, & development. University of

Pennsylvania School of Engineering and Applied Science Department of Computer
and Information Science.

2. Dahiya, M., 2017. A tool of conversation: Chatbot. International Journal of Computer

Sciences and Engineering, 5(5), pp.158-161.

3. A., S. and John, D., 2015. Survey on Chatbot Design Techniques in Speech
Conversation Systems. International Journal of Advanced Computer Science and
Applications, 6(7).

4. Pilato, G., Augello, A., Vassallo, G. and Gaglio, S., 2008. EHeBby: An Evocative
Humorist Chat-Bot. Mobile Information Systems, 4(3), pp.165-181.

5. Coskun, V., Ozdenizci, B. and Ok, K., 2013. A survey on near field communication
(NFC) technology. Wireless personal communications, 71(3), pp.2259-2294.

6. Pesonen, J. and Horster, E., 2012. Near field communication technology in tourism.
Tourism Management Perspectives, 4, pp.11-18.

7. Curran, K., Millar, A. and Mc Garvey, C., 2012. Near field communication.
International Journal of Electrical and Computer Engineering, 2(3), p.371.

8. Pesonen, J. and Horster, E., 2012. Near field communication technology in tourism.
Tourism Management Perspectives, 4, pp.11-18.

9. Rudametkin, W., Touseau, L., Perisanidi, M., Gómez, A., Donsez, D. and Bull,
S.A.S., 2008, July. NFCMuseum: An open-source middleware for augmenting
museum exhibits. In International Conference on Pervasive Services (ICPS'08),
Demonstration track, July (pp. 6-10).

21
10. Amato, F., Chianese, A., Mazzeo, A., Moscato, V., Picariello, A. and Piccialli, F.,
2013. The talking museum project. Procedia Computer Science, 21, pp.114-121.

11. Martínez-Graña, A.M., Goy, J.L. and Cimarra, C.A., 2013. A virtual tour of
geological heritage: Valourising geodiversity using Google Earth and QR code.
Computers & Geosciences, 61, pp.83-93.

12. Yim, J. and Jo, G., 2016. Design of a QR Code–based Tour Guide System.
International Journal of Software Engineering and Its Applications, 10(10), pp.139-
148.

13. Fino, E.R., Martín-Gutiérrez, J., Fernández, M.D.M. and Davara, E.A., 2013.
Interactive tourist guide: connecting web 2.0, augmented reality and qr codes.
Procedia Computer Science, 25, pp.338-344.

14. Liu, C.H. and Lee, C.F., 2009. The Design of a Mobile Navigation System Based on
QR Codes for Historic Buildings.

15. Whitchurch, M.J., 2011. QR codes and the library: the library audio tour.

16. Cassell, J., 2000. Embodied conversational interface agents. Communications of the
ACM, 43(4), pp.70-78.

17. De Almeida, P. and Yokoi, S., 2003. Interactive Character as a Virtual Tour Guide to
an Online Museum Exhibition.

18. Boiano, S., Gaia, G. and Caldarini, M., 2003. Make Your Museum Talk: Natural
Language Interfaces for Cultural Institutions.

19. Borovikov, E. and Yershov, S., 2016. On Virtual Characters that Can See.
PercepReal, USA: Procedia Computer Science.

20. Badin, P., Borel, P., Bailly, G., Revéret, L., Baciu, M. and Segebarth, C., 2000, May.
Towards an audiovisual virtual talking head: 3D articulatory modeling of tongue, lips

22
 and face based on MRI and video images. In 5th Speech Production Seminar (pp.
261-264).

21. Lou, L., Tian, Z. and Koh, J., 2017. Tourist satisfaction enhancement using mobile
QR code payment: An empirical investigation. Sustainability, 9(7), p.1186.

22. Emmanuel, A.A., Adedoyin, A.E., Mukaila, O. and Roseline, O.O., 2020. Application
of smartphone qrcode scanner as a means of authenticating student identity card.
International Journal of Engineering Research and Technology, 13(1), pp.48-53.

23. Wang, Y.M., Sun, C.T., Kuan, P.C., Lu, C.S. and Wang, H.C., 2018, April. Secured
graphic QR code with infrared watermark. In 2018 IEEE International Conference on
Applied System Invention (ICASI) (pp. 690-693). IEEE.

23