LEVEL 5

Artificial Intelligence

Lecturer Guide

AI Student Guide final 28.10

 Modification History

Version Date Revision Description

V1.0 Jan 2021 For release

© NCC Education Limited, 2021

All Rights Reserved

The copyright in this document is vested in NCC Education Limited. The document must not be
reproduced by any means, in whole or in part, or used for manufacturing purposes, except with the
prior written permission of NCC Education Limited and then only on condition that this notice is
included in any such reproduction.

Published by: NCC Education Limited, The Towers, Towers Business Park, Wilmslow Road,
Didsbury, Manchester M20 2EZ, UK.

Tel: +44 (0) 161 438 6200 Fax: +44 (0) 161 438 6240 Email: info@nccedu.com
http://www.nccedu.com

Page 2 of 80
AI Student Guide final 28.10
 Title Here
CONTENTS
1. Module Overview and Objectives .............................................................................. 6
2. Learning Outcomes and Assessment Criteria .......................................................... 6
3. Syllabus....................................................................................................................... 7
4. Related National Occupational Standards ................................................................ 9
5. Resources ................................................................................................................. 10
5.1 Additional Hardware and Software Requirements ....................................................... 10
6. Pedagogic Approach ................................................................................................ 10
6.1 Lectures...................................................................................................................... 10
6.2 Tutorials ...................................................................................................................... 10
6.3 Laboratory Sessions ................................................................................................... 11
6.4 Private Study .............................................................................................................. 11
7. Assessment .............................................................................................................. 11
8. Further Reading List ................................................................................................. 11
Topic 1: Introduction to Artificial Intelligence ....................................................................... 12
1.1 Learning Objectives .................................................................................................... 12
1.2 Pedagogic Approach .................................................................................................. 12
1.3 Timings ....................................................................................................................... 12
1.4 Lecture Notes ............................................................................................................. 12
1.5 Tutorial Sessions ........................................................................................................ 13
1.6 Private Study .............................................................................................................. 13
Topic 2: Problem Solving Using Search ................................................................................ 15
2.1 Learning Objectives .................................................................................................... 15
2.2 Pedagogic Approach .................................................................................................. 15
2.3 Timings ....................................................................................................................... 15
2.4 Lecture Notes ............................................................................................................. 15
2.5 Tutorial Sessions ........................................................................................................ 16
2.6 Private Study .............................................................................................................. 17
Topic 3: Knowledge Representation ...................................................................................... 19
3.1 Learning Objectives .................................................................................................... 19
3.2 Pedagogic Approach .................................................................................................. 19
3.3 Timings ....................................................................................................................... 19
3.4 Lecture Notes ............................................................................................................. 19
3.5 Tutorial Sessions ........................................................................................................ 20
3.6 Private Study .............................................................................................................. 21
Topic 4: Uncertain Knowledge ............................................................................................... 22
4.1 Learning Objectives .................................................................................................... 22

Page 3 of 80
AI Student Guide final 28.10
 Title Here
4.2 Pedagogic Approach .................................................................................................. 22
4.3 Timings ....................................................................................................................... 22
4.4 Lecture Notes ............................................................................................................. 22
4.5 Tutorial Sessions ........................................................................................................ 23
4.6 Private Study .............................................................................................................. 24
Topic 5: Fuzzy Logic ............................................................................................................... 26
5.1 Learning Objectives .................................................................................................... 26
5.2 Pedagogic Approach .................................................................................................. 26
5.3 Timings ....................................................................................................................... 26
5.4 Lecture Notes ............................................................................................................. 26
5.5 Tutorial Sessions ........................................................................................................ 27
5.6 Private Study .............................................................................................................. 28
Topic 6: Machine Learning ..................................................................................................... 30
6.1 Learning Objectives .................................................................................................... 30
6.2 Pedagogic Approach .................................................................................................. 30
6.3 Timings ....................................................................................................................... 30
6.4 Lecture Notes ............................................................................................................. 30
6.5 Tutorial Notes ............................................................................................................. 31
6.6 Private Study .............................................................................................................. 32
Topic 7: Neural Networks ....................................................................................................... 33
7.1 Learning Objectives .................................................................................................... 33
7.2 Pedagogic Approach .................................................................................................. 33
7.3 Timings ....................................................................................................................... 33
7.4 Lecture Notes ............................................................................................................. 33
7.5 Laboratory Sessions ................................................................................................... 34
7.6 Tutorial Notes ............................................................................................................. 35
7.7 Private Study .............................................................................................................. 36
Topic 8: Decision Trees .......................................................................................................... 37
8.1 Learning Objectives .................................................................................................... 37
8.2 Pedagogic Approach .................................................................................................. 37
8.3 Timings ....................................................................................................................... 37
8.4 Lecture Notes ............................................................................................................. 37
8.5 Laboratory Sessions ................................................................................................... 38
8.6 Tutorial Notes ............................................................................................................. 42
8.7 Private Study .............................................................................................................. 43
Topic 9: Genetic Algorithms................................................................................................... 45
9.1 Learning Objectives .................................................................................................... 45

Page 4 of 80
AI Student Guide final 28.10
 Title Here
9.2 Pedagogic Approach .................................................................................................. 45
9.3 Timings ....................................................................................................................... 45
9.4 Lecture Notes ............................................................................................................. 45
9.5 Tutorial Sessions ........................................................................................................ 46
9.6 Private Study .............................................................................................................. 47
Topic 10: Expert Systems ......................................................................................................... 48
10.1 Learning Objectives .................................................................................................... 48
10.2 Pedagogic Approach .................................................................................................. 48
10.3 Timings ....................................................................................................................... 48
10.4 Lecture Notes ............................................................................................................. 49
10.5 Laboratory Sessions ................................................................................................... 49
10.6 Tutorial Notes ............................................................................................................. 61
10.7 Private Study .............................................................................................................. 62
Topic 11: Natural Language Processing ................................................................................. 63
11.1 Learning Objectives .................................................................................................... 63
11.2 Pedagogic Approach .................................................................................................. 63
11.3 Timings ....................................................................................................................... 63
11.4 Lecture Notes ............................................................................................................. 63
11.5 Laboratory Sessions ................................................................................................... 64
11.6 Tutorial Notes ............................................................................................................. 69
11.7 Private Study .............................................................................................................. 71
Topic 12: Intelligent Agents...................................................................................................... 76
12.1 Learning Objectives .................................................................................................... 76
12.2 Pedagogic Approach .................................................................................................. 76
12.3 Timings ....................................................................................................................... 76
12.4 Lecture Notes ............................................................................................................. 77
12.5 Tutorial Notes ............................................................................................................. 77
12.6 Private Study .............................................................................................................. 79

Page 5 of 80
AI Student Guide final 28.10
 Title Here
1. Module Overview and Objectives
This unit will provide students with the foundation and branches of artificial intelligence, such as
search strategies, knowledge representation and reasoning techniques. This unit will also introduce
a range of well-known techniques and applications in artificial intelligence, such as fuzzy logic,
machine learning, expert systems, natural language processing, and intelligent agents.

At the end of the unit, students should be able to:

1. Understand the importance of AI and its applications.
2. Apply a range of well-established AI search strategies and knowledge representation
techniques in problem solving.
3. Assess a range of well-established techniques for reasoning with uncertain knowledge.
4. Understand a range of machine learning techniques.
5. Understand the range of AI techniques that are being applied in industry or research.
6. Implement and evaluate a range of AI models and techniques for solving real-world
problems.

2. Learning Outcomes and Assessment Criteria

Learning Outcomes; Assessment Criteria;
The Learner will: The Learner can:
1. Understand the importance of 1.1 Explain the meaning of AI and its origin.
AI and its applications. 1.2 Identify the characteristics of AI.
1.3 Discuss the limitations and ethics of AI.
1.4 Discuss current and future developments
in the field of AI and its applications.
2. Apply a range of well- 2.1 Construct simple state spaces.
established AI search strategies 2.2 Select and apply appropriate search
and knowledge representation techniques in problem solving.
techniques in problem solving. 2.3 Explain and identify different types of
knowledge.
2.4 Apply knowledge representation using the
logical, semantic network, frame, and
production rules techniques.
3. Assess a range of well- 3.1 Explain the concept of uncertainty.
established techniques for 3.2 Explain the source of uncertain knowledge.
reasoning with uncertain 3.3 Discuss and apply probabilistic reasoning using
knowledge. the Bayes’ rule and the certainty factor theory.
4. Understand a range of 4.1 Explain machine learning.
machine learning 4.2 Compare and contrast supervised
techniques. learning, unsupervised learning, and
reinforcement learning.
4.3 Identify and apply suitable machine learning
techniques in problem solving.

Page 6 of 80
AI Student Guide final 28.10
 Title Here
5. Understand the range of AI 5.1 Describe the key players, components,
techniques that are being characteristics, and limitations of expert systems.
applied in industry or 5.2 Apply reasoning techniques in rule-based expert
research. systems.
5.3 Explain natural language processing and its
components.
5.4 Discuss a range of applications using natural
language processing.
5.5 Explain intelligent agents and PEAS.
5.6 Discuss a range of applications using intelligent
agents.
6. Implement and evaluate a range 6.1 Select suitable tools and techniques for use in
of AI models and techniques for designing AI models.
solving real-world problems. 6.2 Construct an AI model.
6.3 Evaluate the performance of AI models

3. Syllabus
Syllabus
Topic No Title Proportion Content
1 Introduction to AI 1/12 • Definitions
• History of AI
• Characteristics of AI
2 hours of
• Limitations of AI
lectures
• Ethics in AI
2 hours of • Current and future development of AI and its
tutorials applications
Learning Outcome: 1
2 Problem Solving 1/12 • Problem representation in state space
Using Search • Strategies for state space search
• Uninformed search (blind search)
2 hours of
• Informed search (heuristic Search)
lectures
Learning Outcome: 2
2 hours of
tutorials
3 Knowledge 1/12 • Types of knowledge
Representation • Logical representation
• Semantic network representation
2 hours of
• Frame representation
lectures
• Production Rules
2 hours of Learning Outcome: 2
tutorials
4 Uncertain 1/12 • Uncertainty and its sources
Knowledge • Basic probability
• Bayes’ rule
2 hours of
• Bayesian reasoning
lectures
• Certainty factors
2 hours of Learning Outcome: 3
tutorials

Page 7 of 80
AI Student Guide final 28.10
 Title Here
5 Fuzzy Logic 1/12 • Fuzzy logic
• Linguistic variables
• Fuzzy sets and operations
2 hours of
• Fuzzy rules
lectures
• Fuzzy system
2 hours of Learning Outcome: 3
tutorials
6 Machine Learning 1/12 • Introduction
• Supervised learning
• Unsupervised learning
2 hours of
• Reinforcement learning
lectures
• Applications of machine learning
2 hours of Learning Outcome: 4
tutorials
7 Neural Networks 1/12 • Basic structure
• Perceptron
• Multilayer neural networks
2 hours of
• Backpropagation learning
lectures
• Accelerated learning
2 hours of • Recurrent neural networks
tutorials • Learning Outcome: 4, 6

3 hours of
laboratory
sessions
8 Decision Trees 1/12 • Structure and terminologies
• Attribute selection measures
• Learning Outcome: 4, 6
2 hours of
lectures
2 hours of
tutorials

3 hours of
laboratory
sessions
9 Genetic Algorithms 1/12 • Simulation of Natural Evolution
• Basic genetic algorithms
• Learning Outcome: 4
2 hours of
lectures
2 hours of
tutorials

Page 8 of 80
AI Student Guide final 28.10
 Title Here
10 Expert Systems 1/12 • The development team of an expert system
• Components of an expert system
• Characteristics of an expert system
2 hours of
• Rule-based expert system
lectures
• Learning Outcome: 5, 6
2 hours of
tutorials

3 hours of
laboratory
sessions
11 Natural Language 1/12 • Terminologies
Processing • Components of natural language
processing
2 hours of • Phases in natural language processing
lectures
• Natural language processing pipeline
2 hours of • Applications of natural language processing
tutorials • Learning Outcome: 5, 6

3 hours of
laboratory
sessions
12 Intelligent Agents 1/12 • Agents and environments
• Rationality
• PEAS
2 hours of
• Types of intelligent agents
lectures
• Game Playing
2 hours of • Algorithms in games
tutorials • Learning Outcome: 5

4. Related National Occupational Standards

The UK National Occupational Standards describe the skills that professionals are expected to
demonstrate in their jobs in order to carry them out effectively. They are developed by employers
and this information can be helpful in explaining the practical skills that students have covered in this
module.

Related National Occupational Standards (NOS)

Sector Subject Area: IT and Telecoms
Related NOS:
TECIS806401 Develop and communicate data visualisations
TECIS806301 Assist in developing data visualisations
TECIS804301 Assist in the deployment of artificial intelligence solutions
TECIS804401 Manage the deployment of artificial intelligence solutions
TECIS805401 Develop and implement machine learning algorithms
TECIS805301 Assist in developing and validating machine learning solutions
ESKITP4022/3/4/5/6 Data Analysis Levels 2/3/4/5/6 Role

Page 9 of 80
AI Student Guide final 28.10
 Title Here
5. Resources
Lecturer Guide: This guide contains notes for lecturers on the organisation of each topic, and
suggested use of the resources. It also contains all of the suggested
exercises and model answers.

PowerPoint Slides: These are presented for each topic for use in the lectures. They contain many
examples which can be used to explain the key concepts. Handout versions
of the slides are also available; it is recommended that these are distributed to
students for revision purposes as it is important that students learn to take
their own notes during lectures.

Student Guide: This contains the topic overviews and all of the suggested exercises. Each
student will need access to this and should bring it to all of the taught hours
for the module.

5.1 Additional Hardware and Software Requirements

Hardware: A computer with 64-bit Windows/Mac OS/ Linux (Desktop/Laptop)

Software: Chapter 7: Waikato Environment for Knowledge Analysis (WEKA)

https://waikato.github.io/weka-wiki/downloading_weka/

Chapter 8: Anaconda Python (included Jupyter Notebook, Scikit-learn)

https://www.anaconda.com/products/individual

Chapter 10: SWI-Prolog https://www.swi-prolog.org/download/stable

Chapter 11: Anaconda Python (included Jupyter Notebook) pip install NLTK
https://www.anaconda.com/products/individual

6. Pedagogic Approach
Suggested Learning Hours
Guided Learning Hours Assessment Private Study Total
Lecture Tutorial Seminar Laboratory
24 24 - 12 40 100 200

The teacher-led time for this module is comprised of lectures, laboratory sessions and tutorials. The
breakdown of the hours is also given at the start of each topic, with 5 hours of contact time per topic.

6.1 Lectures
Lectures are designed to introduce students to each topic; PowerPoint slides are presented for use
during these sessions. Students should also be encouraged to be active during this time and to
discuss and/or practice the concepts covered. Lecturers should encourage active participation and
field questions wherever possible.

6.2 Tutorials
Tutorials provide tasks to involve group work, investigation and independent learning for certain
topics. The details of these tasks are provided in this guide and also in the Student Guide. They are

Page 10 of 80
AI Student Guide final 28.10
 Title Here
also designed to deal with the questions arising from the lectures, laboratory sessions and private
study sessions.

6.3 Laboratory Sessions

During these sessions, students are required to work through practical tutorials and various
exercises. The details of these are provided in this guide and also in the Student Guide. Some
sessions will require more support than others as well as IT resources. More detail is given in this
guide.

6.4 Private Study

In addition to the taught portion of the module, students will also be expected to undertake private
study. Exercises are provided in the Student Guide for students to complete during this time.
Teachers will need to set deadlines for the completion of this work. These should ideally be before
the tutorial session for each topic, when Private Study Exercises are usually reviewed.

7. Assessment
This unit will be assessed by means of an assignment worth 100% of the total mark. These
assessments will cover the learning outcomes and assessment criteria given above. Samples
assessments are available through the NCC Education Virtual Learning Environment
(http://vle.nccedu.com/login/index.php) for your reference.

8. Further Reading List

A selection of sources of further reading around the content of this module must be available in your
Accredited Partner Centre’s library. The following list provides suggestions of some suitable
sources:

• Russell, S., & Norvig, P. (2016). Artificial Intelligence: A Modern Approach: Pearson
• Negnevitsky, M. (2011). Artificial Intelligence: A Guide to Intelligent Systems: Pearson
Education Limited.
• Luger, G. F. (2011). Artificial Intelligence: Structures and Strategies for Complex Problem
Solving: Pearson Education.
• Poole, D.L. and Mackworth, A.K. (2017). Artificial Intelligence: foundations of computational
agents: Cambridge University Press.
• Giarratano, J. C., & Riley, G. (2005). Expert Systems: Principles and Programming:
Thomson Course Technology
• Sharda, R., Delen, D., & Turban, E. (2019). Analytics, Data Science, & Artificial Intelligence:
Systems for Decision Support: Pearson.

Page 11 of 80
AI Student Guide final 28.10
 Topic 1

Topic 1: Introduction to Artificial Intelligence

1.1 Learning Objectives

This topic provides an overview of Artificial Intelligence (AI). It helps students to understand the
meaning of intelligence in general, way to measure intelligence, the meaning of AI, and the
comparison between human intelligence and machine intelligence. A brief history of AI highlighting
several key ideas, events, and research will also be covered to develop appreciation and
understanding in the motivation and background of AI.

To further discover what AI can do, the characteristics, limitations, and ethics of AI are discussed.
This topic also presents an overview of some of the important applications and emerging
technologies in AI.

On completion of the topic, students will be able to:

• Explain the meaning of AI and its origin.

• Identify the characteristics of AI.
• Discuss the limitations and ethics of AI.
• Discuss current and future developments in the field of AI and its applications.

1.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

1.3 Timings

Lectures: 2 hours

Private Study: 8 hours

Tutorials: 2 hours

1.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• What is Intelligence?
• Measuring Intelligence
• Artificial Intelligence - Definition
• Human vs. Artificial Intelligence
• The History of Artificial Intelligence
• Characteristics of AI

Page 12 of 80
AI Student Guide final 28.10
 Title Here
• Human, AI System, and Conventional System
• Limitations of AI
• Ethics of AI
• AI and its Applications
• Emerging Technologies in AI

1.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

Define in your own words:

(a) intelligence,
(b) artificial intelligence,
(c) reasoning,
(d) knowledge,
(e) ethics.

Exercise 2:

In your opinion, discuss what are the main characteristics that differentiate an AI application and a
conventional application?

Exercise 3:

To what extent are the following computer systems instances of artificial intelligence:
• Supermarket bar code scanners.
• Web search engines.
• Voice-activated telephone menus.
• Internet routing algorithms that respond dynamically to the state of the network.

1.6 Private Study

The time allocation for private study in this topic is expected to be 8 hours.

Exercise 1:

Read Turing’s original paper on AI (Turing, 1950). In the paper, Alan Turing predicts that, by the
year 2000, a computer will have a 30% chance of passing a five-minute Turing Test with an
unskilled interrogator. What chance do you think a computer would have today? In another 50 years,
what do you see as the future of AI? Justify your answer.

Page 13 of 80
AI Student Guide final 28.10
 Title Here
Exercise 2:

List and discuss potentially two positive effects and two negative effects on society of the
development of artificial intelligence technologies.

Page 14 of 80
AI Student Guide final 28.10
 Topic 2

Topic 2: Problem Solving Using Search

2.1 Learning Objectives

This topic introduces search as a problem-solving technique that systematically explores a space of
problem states. The discussion of problem solving begins with definitions of problems and their
formulation using the theory of state space search. This topic also provides several general-purpose
search algorithms that can be used to find solutions for any solvable problems.

On completion of the topic, students will be able to:

• Construct simple state spaces.

• Select and apply appropriate search techniques in problem solving.

2.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

2.3 Timings

Lectures: 2 hours

Private Study: 8.5 hours

Tutorials: 2 hours

2.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• Problem solving
• Search
• State space search
• Search algorithms
• Uninformed vs. informed search
• Breadth-first search
• Depth-first search
• Uniform-cost search
• Best first search
• Greedy search
• A* search

Page 15 of 80
AI Student Guide final 28.10
 Title Here
• Measuring search algorithms performance

2.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

Define in your own words the following terms:

(a) state
(b) state space
(c) search tree
(d) search node
(e) goal
(f) action
(g) transition model
(h) branching factor

Exercise 2:

Consider a state space where the start state is number 1 and each state k has two successors:
numbers 2k and 2k + 1.

(i) Draw the portion of the state space from 1 to 15.

(ii) Suppose the goal state is 11. List the order in which nodes will be visited for BFS and DFS with
limit 3.
(iii) How well would bidirectional search work on this problem? What is the branching factor in each
direction of the bidirectional search?

Exercise 3:

Based on the tree on below, show the tracing for depth-first search. The tracing should include the
open and closed lists at each iteration. Assume the start node is 8, while the goal is 13.

Page 16 of 80
AI Student Guide final 28.10
 Title Here

Exercise 4:

Trace the operation of A* search applied to the problem of getting to Bucharest from Lugoj using the
straight-line distance heuristic. That is, show the sequence of nodes that the algorithm will consider
and the f, g, and h score for each node.

2.6 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1:

Explain why problem formulation must follow goal formulation.

Exercise 2:

Give the graph representation for the farmer, wolf, goat, and cabbage problem:

A farmer with his wolf, goat, and cabbage come to the edge of a river they wish to
cross. There is a boat at the river’s edge, but, of course, only the farmer can row. The
boat also can carry only two things (including the rower) at a time. If the wolf is ever
left alone with the goat, the wolf will eat the goat; similarly, if the goat is left alone with
the cabbage, the goat will eat the cabbage. Devise a sequence of crossings of the
river so that all four characters arrive safely on the other side of the river.

Let nodes represent states of the world; e.g., the farmer and the goat are on the west bank and the
wolf and cabbage on the east. Discuss the advantages of breadth-first and depth-first search for this
problem.

Page 17 of 80
AI Student Guide final 28.10
 Title Here
Exercise 3:

Determine whether goal-driven or data-driven search would be preferable for solving each of the
following problems. Justify your answer.

(a) Diagnosing mechanical problems in an automobile.

(b) You have met a person who claims to be your distant cousin, with a common ancestor named
John Doe. You would like to verify her claim.

(c) Another person claims to be your distant cousin. He does not know the common ancestor’s
name but knows that it was no more than eight generations back. You would like to either find
this ancestor or determine that she did not exist.

(d) A theorem prover for plane geometry.

(e) A program for examining sonar readings and interpreting them, such as telling a large
submarine from a small submarine from a whale from a school of fish.

(f) An expert system that will help a human classify plants by species, genus, etc.

Page 18 of 80
AI Student Guide final 28.10
 Topic 3

Topic 3: Knowledge Representation

3.1 Learning Objectives

This topic presents one of the main themes of AI, which is the representation of information or
knowledge for use in intelligent problem solving on a computer. Several knowledge representation
techniques are discussed including semantic networks, frames, conceptual graphs, and rules.

On completion of the topic, students will be able to:

• Explain and identify different types of knowledge.

• Apply knowledge representation using the logical, semantic network, frame, and production
rules techniques.

3.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

3.3 Timings

Lectures: 2 hours

Private Study: 8 hours

Tutorials: 2 hours

3.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• What is knowledge?
• Types of knowledge
• Knowledge representation
• Knowledge representation schemes
• Knowledge representation techniques
• Logic
• Semantic network
• Frame
• Conceptual graph
• Production rule

Page 19 of 80
AI Student Guide final 28.10
 Title Here
3.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

Translate each of the following sentences into predicate calculus and conceptual graphs:

(a) Jane gave Tom an ice cream cone.

(b) Basketball players are tall.
(c) Paul cut down the tree with an axe.
(d) John is going to Boston by bus.

Exercise 2:

Translate the following two conceptual graphs into english sentences and predicate calculus.

Page 20 of 80
AI Student Guide final 28.10
 Title Here
Exercise 3:

Illustrate the frame representation for the following information:

Animals eat, breath and have skin. Birds are special kind of animal which have wings, feathers and
normally can fly. Crows are black and can fly. Penguins are black and white and ostriches are
brown. Penguins and ostriches are birds but they cannot fly. Tom, a human being, owns a pet crow
called ‘black-head’.

3.6 Private Study

The time allocation for private study in this topic is expected to be 8 hours.

Exercise 1:

Consider the following facts on Italian history:

Marcus was a man.

Marcus was a Pompeian.
All Pompeians were Roman.
Caesar was a ruler.
All Romans were either loyal to Caesar or hated him.
Everyone is loyal to someone.
People only try to assassinate rulers that they are not loyal to.
Marcus tried to assassinate Caesar.
All men are people.

Represent these facts as the well-formed formulas (wffs) of predicate logic.

Exercise 2:

Draw a semantic net to represent the following information:

A fish has fins, scales, is cold blooded and has gills. There are lots of kinds of fish, including cod,
haddock, shark, goldfish, and trout. A fish is a vertebrate - that means it has a backbone. I have a
goldfish called Sammy. Cod, haddock and sharks are salt-water fish (they live in salt water).
Goldfish and trout are freshwater fish (they live in fresh water).

Exercise 3:

A production system is used to represent knowledge in the form of condition-action pairs called
production rules. MYCIN expert system is an example of a production system. Describe this expert
system, and explain the main components of a production system?

Page 21 of 80
AI Student Guide final 28.10
 Topic 4

Topic 4: Uncertain Knowledge

4.1 Learning Objectives
Classical logic permits only exact reasoning. It assumes that perfect knowledge always exists and
the law of the excluded middle can always be applied. Unfortunately, most real-world problems do
not provide us with such clear-cut knowledge. The available information or uncertain knowledge
often contains inexact, incomplete or even unmeasurable data. This topic presents two uncertainty
management techniques used in AI systems, namely the Bayesian method and certainty factors.
This topic also introduces the main sources of uncertain knowledge and briefly review the basic
elements of probability theory.

On completion of the topic, students will be able to:

• Explain the concept of uncertainty.

• Explain the source of uncertain knowledge.
• Discuss and apply probabilistic reasoning using the Bayes’ rule and the certainty factor theory.

4.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

4.3 Timings

Lectures: 2 hours

Private Study: 8.5 hours

Tutorials: 2 hours

4.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• What is uncertainty?
• What are the sources of uncertain knowledge?
• Basic probability theory
• Conditional probability
• Joint probability
• Proof of Bayes’ rule
• Bayes’ rule
• Bayesian reasoning

Page 22 of 80
AI Student Guide final 28.10
• Bias of the Bayesian method
• Certainty factors theory
• Certainty factors: evidential reasoning

4.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

A housekeeper robot called Blinky is supposed to wash the dog and put the bell on the cat. The dog
and cat are never inside at the same time. Blinky remembers that the dog has been inside 30 of the
past 100 days and the cat the other 70. Blinky’s pet-detector goes off 90% of the time if it's aimed at
the dog and 20% if it's aimed at the cat. So, when Blinky's pet-detector goes off, what is the
probability that he aimed at the dog? Draw a probability tree diagram and use Bayes’ Rule to
calculate the probability.

Exercise 2:

Suppose the AIDS positive is one in 100. Suppose the test has a false positive rate of 0.2 (that is,
20% of people without HIV will test positive for HIV) and that it has a false negative rate of 0.1 (that
is, 10% of people with HIV will test negative). Now suppose a guy is declared that his test was
positive. What is the probability that he has HIV?

Exercise 3:

Consider the following rule and certainty factor (CF) information:

R1: IF A AND B AND C CF(A) = 0.85

THEN D CF=0.75 CF(C) = 0.6
CF(F) = 0.84
R2: IF E OR C OR G CF(B) = – 0.39
THEN D CF=0.95
CF(E) = – 0.3
R3: IF (C AND E) OR (F AND G) CF(G) = 0.45
THEN D CF=0.85

(i) Calculate the CF value for each conclusion in R1, R2, and R3.

(ii) Calculate the combined CF value of R1, R2, and R3. Then interpret the combined CF value by
using the CF scale below:

Page 23 of 80
AI Student Guide final 28.10
 Term Certainty Factor
Definitely not _ 1.0
Almost certainly not _ 0.8
_
Probably not _ 0.6
Maybe not _ 0.4
Unknown 0.2 to +0.2
Maybe +0.4
Probably +0.6
Almost certainly +0.8
Definitely +1.0

4.6 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1:

What is uncertainty in expert systems? Explain THREE (3) sources of uncertainty in your own
words.

Exercise 2:

Using the following rules and information, determine the investment recommendations.

CF (interest rate low) = 0.7

CF (interest rate high) = 0.2
CF (inflation low) = 0.6
CF (inflation high) = 0.5
CF (employment good) = 0.6
CF (stock prices low) = 0.4
CF (bond prices low) = 0.6

Page 24 of 80
AI Student Guide final 28.10
Exercise 3:

Given two conditionally independent evidences E1 and E2, and two mutually exclusive and
exhaustive hypotheses H1 and H2. Table below shows the conditional probabilities of observing each
evidence for all possible hypotheses.

Probability Hypothesis
i=1 i=2
p(Hi) 0.6 0.8
p(E1| Hi) 0.7 0.8
p(E2| Hi) 0.3 0.3

(i) Calculate the posterior probabilities for all hypotheses by observing evidence E1. Based on the
calculated results, state which hypothesis is most likely to be reached?

(ii) Calculate the posterior probabilities for all hypotheses by observing evidence E1 and E2. Based
on the calculated results, state which hypothesis is most likely to be reached?

Page 25 of 80
AI Student Guide final 28.10
 Topic 5

Topic 5: Fuzzy Logic

5.1 Learning Objectives

Fuzzy logic is concerned with the use of fuzzy values that capture the meaning of words, human
reasoning and decision making. As a method to encode and apply human knowledge in a form that
accurately reflects their understanding of difficult, complex problems, fuzzy logic provides the way to
break through the computational bottlenecks of traditional systems.

This topic introduces fuzzy logic, including the differences to Boolean logic, the concept of fuzzy
sets, how to represent a fuzzy set, and examine operations of fuzzy sets. Then, the topic presents
fuzzy rules and explain the main differences between classical and fuzzy rules. At the heart of fuzzy
logic lies the concept of a linguistic variable and hedges. The values of the linguistic variable are
words rather than numbers. Finally, this topic introduced the main steps in the fuzzy inference
process and illustrated through an example of a fuzzy system.

On completion of the topic, students will be able to:

• Discuss and apply fuzzy logic for reasoning with vague and ambiguous knowledge.

5.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

5.3 Timings

Lectures: 2 hours

Private Study: 8.5 hours

Tutorials: 2 hours

5.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• What is Fuzzy Logic?

• Boolean Logic vs Fuzzy Logic
• Fuzzy Sets
• Membership Function
• Crisp and fuzzy sets
• Logical Operations of Fuzzy Sets

Page 26 of 80
AI Student Guide final 28.10
• Linguistic Variable
• Hedges
• Fuzzy Rules
• Classical vs Fuzzy Rules
• Firing Fuzzy Rules
• Fuzzy Inference Process

5.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

You are assigned the task of identifying images in an overhead reconnaissance photograph. The
two fuzzy sets representing a car image and a truck image are defined as:

Car = (0.5 / truck, 0.4 / motor, 0.3 / boat, 0.9 / car, 0.1 / house)
Truck = (1 / truck, 0.1 / motor, 0.4 / boat, 0.4 / car, 0.2 / house)

Find the following:

(a) Car  Truck
(b) Car  Truck
(c) not(Car)
(d) Car  not(Truck)
(e) Car  not(Car)
(f) Car  not(Car)

Exercise 2:

Given below the function based on person height for calculating membership values for the fuzzy set
tall:

(a) Draw the membership function for the tall fuzzy set, showing the universe of discourse (height)
and degree of membership.

(b) What is the degree of membership to the fuzzy set tall of Billy, who has a height of 5.75 feet?
And to the fuzzy set not tall?

Page 27 of 80
AI Student Guide final 28.10
Exercise 3:
Given below the membership function based on person height for calculating membership values for
the fuzzy set medium.

Draw the membership function for the medium fuzzy set, showing the universe of discourse (height)
and degree of membership. Set the universe of discourse values in the range of 4 feet to 7 feet.

5.6 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1:

What are a fuzzy set and a membership function? What is the difference between a crisp set and a
fuzzy set?

Exercise 2:

What is a fuzzy rule? What is the difference between classical and fuzzy rules? Give some
examples.

Exercise 3:

What is defuzzification? What is the most popular defuzzification method? How do we determine the
final output of a fuzzy system mathematically and graphically?

Page 28 of 80
AI Student Guide final 28.10
Reflective tasks:

Exercise 4:

Describe two areas where fuzzy systems or applications might be appropriate.

Exercise 5:

Describe the advantages and disadvantages of fuzzy logic.

Page 29 of 80
AI Student Guide final 28.10
 Topic 6

Topic 6: Machine Learning

6.1 Learning Objectives

This topic provides an overview of machine learning (ML), which is a growing technology of artificial
intelligence that focuses on intelligent agents that can learn. Learning is an essential aspect of
intelligence that enables computers to learn automatically from past data and information to improve
reasoning or decision making. This topic also laid out the machine learning process that consists of
five typical phases to guide the development of working ML models. Machine learning types could
be categorised into supervised learning, unsupervised learning, and reinforcement learning. For
each type, suitable algorithms or techniques are introduced for various tasks such as classification,
regression, and clustering. The topic also introduced several common model evaluation methods
and metrics. Finally, we can see there is a wide range of applications of ML for various sectors.

On completion of the topic, students will be able to:

• Explain machine learning.

• Compare and contrast supervised learning, unsupervised learning, and reinforcement learning.

6.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

6.3 Timings

Lectures: 2 hours

Private Study: 8 hours

Tutorials: 2 hours

6.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• What is Machine Learning?

• Why Machine Learning?
• Machine Learning Process
• Types of Machine Learning
• Supervised Learning
• Supervised Learning Techniques
• Unsupervised Learning

Page 30 of 80
AI Student Guide final 28.10
• Unsupervised Learning Techniques
• Reinforcement Learning
• Reinforcement Learning Techniques
• Model Evaluation
• Model Evaluation Metrics
• Generalisation
• Overfitting
• Applications of Machine Learning

6.5 Tutorial Notes

The time allowance for tutorials in this topic is 2 hours.

Exercise 1:

(a) How would you define Machine Learning?

(b) Why is machine learning so important?
(c) What are the two most common supervised tasks?
(d) What is a labelled training set?
(e) What type of algorithm would you use to segment your customers into multiple groups?
(f) What type of Machine Learning algorithm would you use to allow a robot to walk in various
unknown terrains?
(g) Is Spam detection a supervised learning problem or an unsupervised learning problem?
(h) What is the difference between a model parameter and a learning algorithm’s hyperparameter?
(i) If a model performs excellently on the training data but generalises poorly to new instances, what
is happening? What are the possible solutions?

Exercise 2:

Discuss the differences between machine learning and deep learning.

Exercise 3:

What is the main difference between classification and clustering? Explain using concrete examples.

Exercise 4:

Compare the differences of the three types in Machine Learning:

• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning

Give of short descriptions of the categories and explain the differences

Page 31 of 80
AI Student Guide final 28.10
6.6 Private Study

The time allocation for private study in this topic is expected to be 8 hours.

Exercise 1:

What characteristics would you expect to have in a robot that provides emotional support to
patients? How could machine learning help?

Exercise 2:

Why is the recommender system useful? (Relate it to one-to-one targeted marketing.). Explain how
recommendations are generated using ML and state an example of a recommender system.

Exercise 3:

Identify three of Gartner’s top 10 strategic technology that is most related to machine learning. Why?

Exercise 4:

Discuss ethics and fairness in machine learning. Start by reading Pakzad (2018):
https://medium.com/humane-ai/ethics-in-machine-learning-54a71a75875c

Exercise 5:

Look for the three most used ML technologies or products from the internet. Briefly discuss the
products and their features. Which is your favourite product? Why?

Page 32 of 80
AI Student Guide final 28.10
 Topic 7

Topic 7: Neural Networks

7.1 Learning Objectives

This topic introduces connectionist approaches and neurally inspired learning models for intelligent
systems known as neural networks. The concepts of a perceptron as a simple computing element
and the perceptron’s learning or training algorithm are presented for classification tasks. Also, this
topic explores the multilayer perceptron or multilayer neural network using a backpropagation
learning algorithm and how to improve computational efficiency.

On completion of the topic, students will be able to:

• Identify and apply suitable neural network in problem solving.

• Select suitable tools and techniques for use in designing a neural network model.
• Construct a neural network model.
• Evaluate the performance of a neural network model.

7.2 Pedagogic Approach

Information will be transmitted to the students during the lectures. They will then practise the skills
during the laboratory sessions and extend their understanding during private study time. The tutorial
will then provide an opportunity to review the key ideas and obtain further guidance and support.

7.3 Timings

Lectures: 2 hours

Laboratory Sessions: 3 hours

Private Study: 8.5 hours

Tutorials: 2 hours

7.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• Introduction
• Biological Neural Network
• Artificial Neural Network
• Biological vs Artificial Neural Networks
• Basic Structure of Neural Networks
• Perceptron
• Perceptron’s Training

Page 33 of 80
AI Student Guide final 28.10
• Multilayer Perceptron
• Backpropagation
• Backpropagation’s Training Example
• Accelerated Learning
• Learning with Adaptive Learning Rate
• Learning with Momentum and Adaptive Learning Rate
• Other Types of Neural Networks

7.5 Laboratory Sessions

The laboratory time allocation for this topic is 3 hours.

Exercise 1:

• Download and install WEKA: https://waikato.github.io/weka-wiki/downloading_weka/

• Download the manual for the installed WEKA version and learn up the basic features and
WEKA's experimenter application (required for Exercise 2).
https://waikato.github.io/weka-wiki/documentation/#general-documentation

Exercise 2:

For this exercise, you will use WEKA to learn about the neural network classifier (called
MultilayerPerceptron) and perform a statistical comparison of this classifier to a decision tree
classifier (called J48) by using several datasets. Answer all the questions below:

1. Run the MultilayerPerceptron classifier on the weather.arff dataset. Use the default parameter
settings and use the training set as the test option. Include in your submission the printed results
(weights and statistics) from WEKA. Show the graphic of the neural network topology (input
nodes, hidden nodes, output nodes, connections) used by the classifier.

2. WEKA's default parameter settings (among others) for MultilayerPerceptron are -L 0.3 -M 0.2 -N
500 -H a. Explain in your own words what these means.

3. Run the J48 and NeuralNetwork classifiers on three other default datasets supplied with WEKA.
For each run, use the Percentage Split test option with 66% training. Include a table giving the
error rate on the test set for each classifier for each database.

4. Using WEKA's experimenter application, perform the following experiment, then show the results
(e.g. screenshot) in your report.
a. Choose a "New" experiment.
b. For the Results Destination section, select ARFF file and provide a file name in which to
store the experimental results.
c. For Experiment Type, choose the default settings: cross-validation with 10 folds and
classification.
d. For Iteration Control, choose the default settings: 10 iterations and data sets first.
e. Select five datasets that come with WEKA: weather and 4 other default datasets.
f. Select two classifiers with default parameter settings: J48 and MultilayerPerceptron.
g. Run the experiment.

Page 34 of 80
AI Student Guide final 28.10
 h. Analyse the results by loading the ARFF results file, selecting the following configuration,
and perform the test.
• Testing with: Paired T-Tester (corrected)
• Comparison field: Percent_incorrect (NOTE: "incorrect", not "correct")
• Significance: 0.05
• Test base: rules.ConjunctiveRule
• Show std. deviations: (checked)

5. Construct a table of classifiers vs. datasets, and in each entry, enter the error and standard
deviation of that classifier on that dataset from the above experiment. Also, add an asterisk to
the end of the entry for each dataset/classifier pair for which the classifier outperforms J48 at the
0.05 level.

6. Compare the performance of three classifiers (J48, MultilayerPerceptron and another classifier).
Specify which classifiers perform better on which databases and why?

7.6 Tutorial Notes

The time allowance for tutorials on this topic is 2 hours.

Exercise 1:

The following figure shows a perceptron for the logical operation AND with two inputs (x1, x2) and a
threshold value  = -1.5. What will be the output when the input is (0, 0)? What about inputs (0, 1),
(1, 1) and (1, 0)? What if we change the threshold weight to -0.5?

[Notes: Refer to Rosenblatt’s Perceptron model. You can show the results in a tabular form.]

Explain why threshold and bias values are used in neural networks?

Page 35 of 80
AI Student Guide final 28.10
Exercise 2:

The following figure shows a back-propagation neural network with initial inputs (x0 = 1, x1 = 1, x2 =
1), weights (w03 = 0.01, w04 = –0.01, w13 = 0.11, w14 = –0.11, w23 = 0.21, w24= –0.21, w35= –0.2, w45= –
0.15) and threshold levels (Ө3= 1, Ө4 = 1, Ө5 = 1). If all the neurons are sigmoid activation function,
calculate:

(i) The actual output for neuron 5 at output layer, y5.

(ii) The error e5, if the desired output for neuron 5 is yd, 5 = 1.
(iii) The updated weights for w45, and w24, if the learning rate, α = 0.5.

7.7 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise:

What is a recurrent neural network?

How does it learn?

Explain an example of a recurrent neural network with its learning algorithm.

Page 36 of 80
AI Student Guide final 28.10
 Topic 8

Topic 8: Decision Trees

8.1 Learning Objectives

This topic provides an overview of another simplest and yet most successful forms of machine
learning, known as decision trees. The topic first describes the graphical representation of a data set
as tree-like structures. A decision tree consists of nodes, branches and leaves. The tree always
starts from the root node and grows down by splitting the data at each level into new nodes. Then
this topic shows how decision trees are particularly good at solving classification problems with
inductive reasoning and learning algorithm that test the most important attribute first.

On completion of the topic, students will be able to:

• Identify and apply suitable decision tree in problem solving.

• Select suitable tools and techniques for use in designing a decision tree model.
• Construct a decision tree model.
• Evaluate the performance of a decision tree model.

8.2 Pedagogic Approach

Information will be transmitted to the students during the lecture. They will then practise the skills
during the laboratory sessions and extend their understanding during private study time. The tutorial
will then provide an opportunity to review the key ideas and obtain further guidance and support.

8.3 Timings

Lecture: 2 hours

Laboratory Sessions: 3 hours

Private Study: 8.5 hours

Tutorials: 2 hours

8.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• Introduction
• Structure and Terminologies
• Building a Decision Tree
• Attribute Selection Measures
• Gini Coefficient
• Information Gain

Page 37 of 80
AI Student Guide final 28.10
• Decision Rules
• Advantages of Decision Trees
• Drawbacks of Decision Trees

8.5 Laboratory Sessions

The laboratory time allocation for this topic is 3 hours.

Exercise 1:

The scikit-learn library is not installed on your local machine by default. You need to install it on your
own machine. We recommend that you use the Anaconda Python distribution at
https://www.anaconda.com/, which come with python interpreter and useful libraries such as scikit-
learn. The installation includes the Jupyter Notebook, which is an open-source web application that
allows you to create and share documents that contain live code, equations, visualizations, and
narrative text.

If you do not use Anaconda before, please refer to the official installation instructions. After
installing, verify your installation by launching the Jupyter Notebook, create a new notebook and
typing import sklearn in a cell. To run the code, click on the cell to select it, then
press SHIFT+ENTER or press the run button in the toolbar above. You should not see an error
message at this point.

Exercise 2: Building a decision tree in Python using a standard dataset.

scikit-learn comes with a few standard datasets that can be used to investigate different
machine learning algorithms.

In this exercise, we will use a data set of iris. The data set consists of 50 samples from each of
three species of Iris (Iris setosa, Iris versicolor and Iris virginica). Four features were measured from
each sample: the length and the width of the sepals and petals, in centimetres. This is a
classification task, where our task is to predict the species of an Iris flower using decision tree.

https://scikit-learn.org/stable/tutorial/basic/tutorial.html#loading-example-dataset

Follow the steps below to complete this exercise. Show the outputs of corresponding code.

1. First, import the libraries required to build a decision tree in Python.

Page 38 of 80
AI Student Guide final 28.10
2. Load the data set.

3. Display the features and target.

4. Separate the independent and dependent variables.

5. Create and train a classifier that takes the input of two arrays.

Study the parameters of DecisionTreeClassifier here:

https://scikit-
learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.Decisi
onTreeClassifier, find out what are the default values for the parameters. For instance, the gini
criterion is used to measure the quality of a split by default.

6. Once trained, plot the tree.

7. The code below makes the decision tree more interpretable by adding in feature and class
names as well as to allow you to save your image. The parameters for function plot_tree can
be found at: https://scikit-learn.org/stable/modules/generated/sklearn.tree.plot_tree.html

Page 39 of 80
AI Student Guide final 28.10
8. Alternatively, the tree can also be exported in textual format (rules) with the
function export_text. The parameters for this function can be found at: https://scikit-
learn.org/stable/modules/generated/sklearn.tree.export_text.html#sklearn.tree.export_text

9. Use predict() method to predict new values:

10. If all the data are used to build the decision tree, it will result in overfitting problem. It is
important to reserve some data to evaluate how good is the prediction capability of the model.
The function train_test_split could be used to split the dataset into training and testing
sets with four parameters with more details here:
https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html

The output of function includes:

• X_train: The input data (features) used for training
• X_test: The input data (features) used for testing
• Y_train: The output data (target) used for training
• Y_test: The output data (target) used for testing

11. Repeat the code to build, train and test (predict) the data:

Page 40 of 80
AI Student Guide final 28.10
Exercise 3: Building a decision tree with the information gain.

Create, train and test a new decision tree using the information gain criterion. The code as follows:

What is the classification accuracy?

How can you improve the accuracy?

Exercise 4: Building a decision tree using the weather dataset

Build a decision tree for the weather dataset as follows:

Assume the predictors are Outlook, Temperature, and Windy, while the target class is Play. Using
the information gain method as the splitting measurement and the 70/30 training-testing rule of
splitting the dataset, what would be the highest prediction results (in accuracy) that you can
obtained? What are the generated decision rules?

Page 41 of 80
AI Student Guide final 28.10
8.6 Tutorial Notes

The time allowance for tutorials in this topic is 2 hours.

Exercise 1:

Machine learning methods are often categorised as supervised, unsupervised and reinforcement
learning methods. Explain in which category does decision tree learning fall and why?

Exercise 2:

Table below shows a training dataset with different animal groups:

can_fly give_birth live_in_water

Group
(Gini = 0.40) (Gini = 0.47) (Gini = 0.34)
no yes no Mammal
no yes no Mammal
yes yes no Mammal
no yes yes Mammal
no yes yes Mammal
no no no non-mammal
no no no non-mammal
no no yes non-mammal
yes no no non-mammal
no no yes non-mammal

(i) Draw a decision tree, where the dependent variable is group and the predictors are can_fly,
give_birth and live_in_water. The best splitting should be determined based on the given Gini
coefficient of each predictor.

(ii) State ONE (1) example of a strong rule that can be extracted from the decision tree to predict
the mammal group.

(iii) State ONE (1) example of a strong rule that can be extracted from the decision tree to predict
non-mammals group.

Exercise 3:

Table below show a sample of dataset that describes prisoners released on parole, and whether
they reoffended within two years of release.

Page 42 of 80
AI Student Guide final 28.10
 GOOD
ID AGE < 30 DRUG DEPENDENCE REOFFENDED
BEHAVIOUR
1 No Yes No Yes
2 No No No No
3 No Yes No Yes
4 Yes No No No
5 Yes No Yes Yes
6 Yes No No No

The GOOD BEHAVIOR feature has a value of yes if the prisoner had not committed any violations
during imprisonment, the AGE < 30 has a value of yes if the prisoner was under 30 years of age
when granted parole, and the DRUG DEPENDENT feature is yes if the prisoner had a drug
addiction at the time of parole. The target feature, REOFFENDED, has a yes value if the prisoner
was arrested within two years of being released; otherwise it has a value of no.

(i) Calculate the entropy for this dataset.

(ii) Calculate information gain (based on entropy) for the GOOD BEHAVIOUR, AGE < 30, and
DRUG DEPENDENCE features.

(iii) Based on the answers in (ii), which feature will be selected for the first splitting of the dataset
and why?

[Note: Refer to the slides on the equation of information gain.]

8.7 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1:

What are advantages and disadvantages of the decision-tree approach to machine learning?
Explain how decision trees could be useful to business professionals or leaders?

Exercise 2:

Refer back to the dataset in the Tutorial’s Exercise 3:

GOOD
ID AGE < 30 DRUG DEPENDENCE REOFFENDED
BEHAVIOUR
1 No Yes No Yes
2 No No No No
3 No Yes No Yes
4 Yes No No No
5 Yes No Yes Yes
6 Yes No No No

Page 43 of 80
AI Student Guide final 28.10
After the first splitting with the feature, you build the following decision tree:

REOFFENDED
Yes: 3
No: 3
Total: 6

AGE < 30

Yes No
Yes: 2 Yes: 1
No: 0 No: 3
Total: 2 Total: 4

Continue to expand the decision tree with the GOOD BEHAVIOUR and DRUG DEPENDENCE
features using the Information Gain as the feature selection method. Draw the final completed
decision tree. What prediction will be returned for the following query based on the generated
decision tree?

GOOD BEHAVIOR = no, AGE < 30 = no, DRUG DEPENDENT = yes

[Note: Show the working steps or calculations.]

Page 44 of 80
AI Student Guide final 28.10
 Topic 9

Topic 9: Genetic Algorithms

9.1 Learning Objectives

This topic provides an overview of evolutionary computation. It introduces the main steps in
developing a genetic algorithm. The topic also illustrates the actual applications of genetic
algorithms through examples.

On completion of the topic, students will be able to:

• Explain genetic algorithms.

• Identify and apply suitable genetic algorithms in problem solving.

9.2 Pedagogic Approach

Information will be transmitted to the students during the lectures. The tutorial will then provide an
opportunity to review the key ideas and obtain further guidance and support.

9.3 Timings

Lectures: 2 hours

Private Study: 8.5 hours

Tutorials: 2 hours

9.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• Evolutionary Computation
• Simulation of Biological Evolution
• Basic Genetic Algorithms
• Genetic Algorithms: Example 1
• Genetic Algorithms: Example 2
• Where Do Genetic Algorithms Work?

Page 45 of 80
AI Student Guide final 28.10
9.5 Tutorial Sessions

The time allocation for this topic is 2 hours.

Exercise 1:

In a genetic algorithm, consider a fitness function of f(x) = 8 - | n1 – n0| where n1 = number of 1s

and n0 = the number of 0s in a chromosome.

(i) A random population of Generation 0 with four chromosomes is created. Complete Table 1 by
filling in the values in column n1, n0, Chromosome fitness, and Fitness ratio.

Table 1: Generation 0

Chromosome Chromosome n1 n0 Chromosome Fitness

label string fitness ratio (%)
f(x) = 8 - | n1 – n0|
X1 01111110
X2 00100000
X3 11001101
X4 10101010

(ii) Compute the chromosomes of Generation 1 after carrying out the following operations:

Task 1: Implement crossover at mid-point for chromosomes X1 and X4.

Task 2: Implement cloning for chromosome X2.
Task 3: Implement mutation on the 3rd gene from the right of chromosome X3

Exercise 2:

The following crossover operation is proposed based on the initial population in Exercise 1:

(a) Show the resulted Generation 1 chromosomes after applying the above crossover
operations.

(b) Choose a Child from Generation 1 and mutate the gene that lies 5th from the right. What
is its fitness value?

Page 46 of 80
AI Student Guide final 28.10
9.6 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Reflective tasks:

(a) Why are genetic algorithms called genetic? Who was the ‘father’ of genetic algorithms?

(b) What are the main steps of a genetic algorithm? Draw a flowchart that implements these
steps. What are termination criteria used in genetic algorithms?

(c) What is the roulette wheel selection technique? How does it work?

(d) How does the crossover operator work? Give an example using fixed-length bit strings.

(e) What is mutation? Why is it needed? How does the mutation operator work? Give an example
using fixed-length bit strings.

(f) Genetic algorithms have been used to solve a well-known problem: the Traveling Salesman
Problem (TSP). In your own words, describe how GAs could be used to solve an example of a
TSP problem.

Page 47 of 80
AI Student Guide final 28.10
 Topic 10

Topic 10: Expert Systems

10.1 Learning Objectives

This topic provides an overview of expert systems, which are used to solve a wide range of
problems in areas such as medicine, engineering, chemistry, geology, business, law, education,
defence, and computer science. An expert system uses knowledge specific to a problem domain to
provide “expert quality” solutions in that application area. This topic briefly discusses the meaning of
expert systems, as well as the main components and the key participants in the development of
expert systems. Also, some examples of well-known and early expert systems are presented such
as the MYCIN system used in the area of medical diagnosis. Then, the reasoning of expert systems
using forward and backward chaining techniques are presented and possible conflict resolution
strategies. Finally, the advantages and disadvantages of expert systems are reviewed.

In the laboratory session, the development of a simple expert system is used to illustrate the
features and problem solving of a problem by using Prolog programming language.

On completion of the topic, students will be able to:

• Describe the key players, components, characteristics, and limitations of expert systems.
• Apply reasoning techniques in rule-based expert systems
• Describe suitable tools and methods for use in designing an expert system.
• Gain awareness on how to construct an expert system.

10.2 Pedagogic Approach

Information will be transmitted to the students during the lectures. They will then practise the skills
during the laboratory sessions and extend their understanding during private study time. The tutorial
will then provide an opportunity to review the key ideas and obtain further guidance and support.

10.3 Timings

Lectures: 2 hours

Laboratory Sessions: 3 hours

Private Study: 8.5 hours

Tutorials: 2 hours

Page 48 of 80
AI Student Guide final 28.10
10.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• What is an Expert System?

• Components of Expert Systems
• Participants in the Development Expert Systems
• Examples of Expert Systems
• Reasoning in Expert Systems
• Forward Chaining
• Backward Chaining
• Conflict Resolution
• Advantages of Expert Systems
• Limitations of Expert Systems

10.5 Laboratory Sessions

The laboratory time allocation for this topic is 3 hours.

Exercise 1: Downloading and Installing SWI Prolog

Go to https://www.swi-prolog.org/download/stable and download the installer which is suitable for

your operating system. SWI Prolog is an IDE with interpreter/compiler to write and run Prolog code.
After a successful installation click the icon and you will get this:

Page 49 of 80
AI Student Guide final 28.10
Exercise 2: Basic Prolog Syntax

Before we start writing Prolog code you need to understand some of its basic syntax. Like other
programming languages it can get complex but we are not going to go in that depth, but this exercise
will give you the right confidence to create simple expert systems. Read the following explanations on
Prolog.

(a) A Prolog program

• A PROLOG program is a set of facts and rules.

• A fact is basically a thing that is known or proved to be true. For example, Alice is the
parent of Jim, can be written in prolog as:

parent(alice, jim).

Notes: Each statement in Prolog should end with dot (.). Anything that starts with a lower-
case letter in the parentheses is a literal constant (a specific object or value). Relation name
(e.g. parent) should start with a lower case too.

• A simple program may contain just facts as follows:

parent(alice, jim).
parent(jim, tim).
parent(jim, dave).
parent(jim, sharon).
parent(tim, james).
parent(tim, thomas).
female(alice).

• A rule is a thing which are true provided that something else is true. Rules are made from
facts. For example, X is the mother of Y, if X is the parent of Y and X is a female:

mother(X,Y) :- parent(X,Y) , female(X).

Head
or
Condition if Body AND
or
Conclusio
n

Notes: Any word written with a capital letter is a variable. So variables are used to define
generic rules. Semicolon ; represents OR.

(b) Running a Prolog program

• Type out the following code in a text editor (e.g. Notepad) and save the file as family.pl.
The files which contain Prolog code must have an extension .pl.

Page 50 of 80
AI Student Guide final 28.10
 • Open SWI-Prolog, and from menu select File > Consult and select the family.pl file in the
file browser. It will show a green message which indicate that it was imported correctly.

(c) A Prolog query

• Now, with a Prolog program loaded we can ask Prolog questions or queries at SWI-Prolog
console. For examples, Is Alice the parent of Jim? Is Jim the parent of Herbert?

Page 51 of 80
AI Student Guide final 28.10
 • Sometimes there is more than one correct answer to a query. Prolog gives the answers one
at a time. To get the next answer type ;. For example, we can ask the question, Who is the
child of Tim?

Notes: Who is called a variable. Prolog will set a variable to any value which makes the
query succeed.

Exercise 3: A Simple Prolog Program

(a) Create a Prolog program as below and save as family.pl. Test the program with various queries.

%facts male/1 and female/1

male(dave).
male(tim).
male(jim).
male(james).
male(thomas).
female(alice).
female(sharon).

%facts parent/2
parent(alice,jim).
parent(jim,tim).
parent(jim,dave).
parent(jim,sharon).
parent(tim,james).
parent(tim,thomas).

%rule father/2 and mother/2

father(X,Y) :- parent(X,Y),male(X).
mother(X,Y) :- parent(X,Y),female(X).

%rule brother/2 and sister/2

brother(X,Y) :-

Page 52 of 80
AI Student Guide final 28.10
 parent(Z,X),parent(Z,Y),
male(X), X\=Y.

%sibling/2
sibling(X,Y) :- brother(X,Y);sister(X,Y).

%grandparent/2
grandparent(X,Y) :- parent(X,Z), parent(Z,Y).

(b) Add the relations below into your program as new rules. Test the rules with queries.

son/2
grandchild/2
aunt/2

Exercise 4: Built a Simple Expert System

In this example, we show you how to construct a small expert system that identifies birds. The
expertise in identification of certain kinds of birds. The rules are rough excerpts from 'Birds of North
America' by Robbins, Bruum, Zim, and Singer. The rules of the system were designed to illustrate
how to represent various types of knowledge, rather than to provide accurate identification. The
following code can be found from Amzi.com.

Figure 1 Relationships between some of the rules in the Bird identification system

Page 53 of 80
AI Student Guide final 28.10
The following rules distinguish between two types of albatross and swan. They are clauses of the
predicate bird/1:

In order for these rules to succeed in distinguishing the two birds, we would have to store facts
about a particular bird that needed identification in the program. For example if we added the
following facts to the program:

then the following query could be used to identify the bird:

The next step in building the system would be to include rules for identifying the family and the order
of a bird. Continuing with the albatross and swans, the predicates for order and family are:

In the first version, the predicate for family was implemented as a simple fact. Now family is
implemented as a rule. The facts in the system can now reflect more primitive data:

Page 54 of 80
AI Student Guide final 28.10
With the given facts, the same query still identifies the bird:

The Canada goose can be used to add some complexity to the system. Since it spends its summers
in Canada, and its winters in the United States, its identification includes where it was seen and in
what season. Two different rules would be needed to cover these two situations:

A simple user interface can be included, which prompts for information when it is needed, rather
than forcing the user to enter it beforehand. The ask predicate will have to determine from the user
whether or not a given attribute value pair is true. This is easily done by making rules for those
attributes that call ask. For instance, if the system has the goal of finding color(white) it will call
ask, rather than look in the program. If ask (color, white) succeeds, color(white) succeeds.

The simplest version of ask prompts the user with the requested attribute and value and seeks
confirmation or denial of the proposed information. The read will succeed if the user answers "yes",
and fail if the user types anything else. The code is:

Page 55 of 80
AI Student Guide final 28.10
Now the program can be run without having the data built into the program. The same query to bird
starts the program, but now the user is responsible for determining whether some of the attribute-
values are true. The following dialog shows how the system runs:

A new predicate, known/3 is used to remember the user's answers to questions. It is not specified
directly in the program, but rather is dynamically asserted whenever ask gets new information from
the user. Every time ask is called it first checks to see if the answer is already known to be yes or
no. If it is not already known, then ask will assert it after it gets a response from the user. The three
arguments to known are: yes/no, attribute, and value. The new version of ask looks like:

(a) Type out the following complete code and try to run it.

% mode: Prolog %
% Copyright (c) 1990-1995 Amzi! inc.
% All rights reserved

% This is a sample of a classification expert system for identification

% of certain kinds of birds. The rules are rough excerpts from 'Birds of
% North America' by Robbins, Bruum, Zim, and Singer.

% This type of expert system can easily use Prolog's built in inferencing
% system. While trying to satisfy the goal 'bird' it tries to satisfy
% various subgoals, some of which will ask for information from the
% user.

% The information is all stored as attribute-value pairs. The attribute

% is represented as a predicate, and the value as the argument to the
% predicate. For example, the attribute-value pair 'color-brown' is
% stored 'color(brown)'.

Page 56 of 80
AI Student Guide final 28.10
% 'identify' is the high level goal that starts the program. The
% predicate 'known/3' is used to remember answers to questions, so it
% is cleared at the beginning of the run.

% The rules of identification are the bulk of the code. They break up
% the problem into identifying orders and families before identifying
% the actual birds.

% The end of the code lists those attribute-value pairs which need
% to be asked for, and defines the predicate 'ask' and 'menuask'
% which are used to get information from the user, and remember it.

main :- identify.

identify:-
retractall(known(_,_,_)), % clear stored information
bird(X),
write('The bird is a '),write(X),nl.
identify:-
write('I can''t identify that bird'),nl.

order(tubenose):-
nostrils(external_tubular),
live(at_sea),
bill(hooked).
order(waterfowl):-
feet(webbed),
bill(flat).
order(falconiforms):-
eats(meat),
feet(curved_talons),
bill(sharp_hooked).
order(passerformes):-
feet(one_long_backward_toe).

family(albatross):-
order(tubenose),
size(large),
wings(long_narrow).
family(swan):-
order(waterfowl),
neck(long),
color(white),
flight(ponderous).
family(goose):-
order(waterfowl),
size(plump),
flight(powerful).
family(duck):-
order(waterfowl),
feed(on_water_surface),
flight(agile).
family(vulture):-
order(falconiforms),
feed(scavange),
wings(broad).
family(falcon):-
order(falconiforms),
wings(long_pointed),
head(large),
tail(narrow_at_tip).

Page 57 of 80
AI Student Guide final 28.10
family(flycatcher):-
order(passerformes),
bill(flat),
eats(flying_insects).
family(swallow):-
order(passerformes),
wings(long_pointed),
tail(forked),
bill(short).

bird(laysan_albatross):-
family(albatross),
color(white).
bird(black_footed_albatross):-
family(albatross),
color(dark).
bird(fulmar):-
order(tubenose),
size(medium),
flight(flap_glide).
bird(whistling_swan):-
family(swan),
voice(muffled_musical_whistle).
bird(trumpeter_swan):-
family(swan),
voice(loud_trumpeting).
bird(canada_goose):-
family(goose),
season(winter), % rules can be further broken down
country(united_states), % to include regions and migration
head(black), % patterns
cheek(white).
bird(canada_goose):-
family(goose),
season(summer),
country(canada),
head(black),
cheek(white).
bird(snow_goose):-
family(goose),
color(white).
bird(mallard):-
family(duck), % different rules for male
voice(quack),
head(green).
bird(mallard):-
family(duck), % and female
voice(quack),
color(mottled_brown).
bird(pintail):-
family(duck),
voice(short_whistle).
bird(turkey_vulture):-
family(vulture),
flight_profile(v_shaped).
bird(california_condor):-
family(vulture),
flight_profile(flat).
bird(sparrow_hawk):-
family(falcon),
eats(insects).

Page 58 of 80
AI Student Guide final 28.10
bird(peregrine_falcon):-
family(falcon),
eats(birds).
bird(great_crested_flycatcher):-
family(flycatcher),
tail(long_rusty).
bird(ash_throated_flycatcher):-
family(flycatcher),
throat(white).
bird(barn_swallow):-
family(swallow),
tail(forked).
bird(cliff_swallow):-
family(swallow),
tail(square).
bird(purple_martin):-
family(swallow),
color(dark).

country(united_states):- region(new_england).
country(united_states):- region(south_east).
country(united_states):- region(mid_west).
country(united_states):- region(south_west).
country(united_states):- region(north_west).
country(united_states):- region(mid_atlantic).

country(canada):- province(ontario).
country(canada):- province(quebec).
country(canada):- province(etc).

region(new_england):-
state(X),
member(X, [massachusetts, vermont, etc]).
region(south_east):-
state(X),
member(X, [florida, mississippi, etc]).

region(canada):-
province(X),
member(X, [ontario,quebec,etc]).

nostrils(X):- ask(nostrils,X).
live(X):- ask(live,X).
bill(X):- ask(bill,X).
size(X):- menuask(size,X,[large,plump,medium,small]).
eats(X):- ask(eats,X).
feet(X):- ask(feet,X).
wings(X):- ask(wings,X).
neck(X):- ask(neck,X).
color(X):- ask(color,X).
flight(X):- menuask(flight,X,[ponderous,powerful,agile,flap_glide,other]).
feed(X):- ask(feed,X).
head(X):- ask(head,X).
tail(X):- menuask(tail,X,[narrow_at_tip,forked,long_rusty,square,other]).
voice(X):- ask(voice,X).
season(X):- menuask(season,X,[winter,summer]).
cheek(X):- ask(cheek,X).
flight_profile(X):- menuask(flight_profile,X,[flat,v_shaped,other]).
throat(X):- ask(throat,X).
state(X):- menuask(state,X,[massachusetts,vermont,florida,mississippi,etc]).
province(X):- menuask(province,X,[ontario,quebec,etc]).

Page 59 of 80
AI Student Guide final 28.10
% 'ask' is responsible for getting information from the user, and remembering
% the users response. If it doesn't already know the answer to a question
% it will ask the user. It then asserts the answer. It recognizes two
% cases of knowledge: 1) the attribute-value is known to be true,
% 2) the attribute-value is known to be false.

% This means an attribute might have multiple values. A third test to

% see if the attribute has another value could be used to enforce
% single valued attributes. (This test is commented out below)

% For this system the menuask is used for attributes which are single
% valued

% 'ask' only deals with simple yes or no answers. a 'yes' is the only
% yes value. any other response is considered a 'no'.

ask(Attribute,Value):-
known(yes,Attribute,Value), % succeed if we know its true
!. % and dont look any further
ask(Attribute,Value):-
known(_,Attribute,Value), % fail if we know its false
!, fail.

ask(Attribute,_):-
known(yes,Attribute,_), % fail if we know its some other value.
!, fail. % the cut in clause #1 ensures that if
% we get here the value is wrong.
ask(A,V):-
write(A:V), % if we get here, we need to ask.
write('? (yes or no): '),
read(Y), % get the answer
asserta(known(Y,A,V)), % remember it so we dont ask again.
Y = yes. % succeed or fail based on answer.

% 'menuask' is like ask, only it gives the user a menu to to choose

% from rather than a yes on no answer. In this case there is no
% need to check for a negative since 'menuask' ensures there will
% be some positive answer.

menuask(Attribute,Value,_):-
known(yes,Attribute,Value), % succeed if we know
!.
menuask(Attribute,_,_):-
known(yes,Attribute,_), % fail if its some other value
!, fail.

menuask(Attribute,AskValue,Menu):-
nl,write('What is the value for '),write(Attribute),write('?'),nl,
display_menu(Menu),
write('Enter the number of choice> '),
read(Num),nl,
pick_menu(Num,AnswerValue,Menu),
asserta(known(yes,Attribute,AnswerValue)),
AskValue = AnswerValue. % succeed or fail based on answer

display_menu(Menu):-
disp_menu(1,Menu), !. % make sure we fail on backtracking

disp_menu(_,[]).
disp_menu(N,[Item | Rest]):- % recursively write the head of

Page 60 of 80
AI Student Guide final 28.10
 write(N),write(' : '),write(Item),nl, % the list and disp_menu the tail
NN is N + 1,
disp_menu(NN,Rest).

pick_menu(N,Val,Menu):-
integer(N), % make sure they gave a number
pic_menu(1,N,Val,Menu), !. % start at one
pick_menu(Val,Val,_). % if they didn't enter a number, use
% what they entered as the value

pic_menu(_,_,none_of_the_above,[]). % if we've exhausted the list

pic_menu(N,N, Item, [Item|_]). % the counter matches the number
pic_menu(Ctr,N, Val, [_|Rest]):-
NextCtr is Ctr + 1, % try the next one
pic_menu(NextCtr, N, Val, Rest).

(b) Show example of queries that can identify some of the birds. You may screen shot your
answers.

(c) Suggest a new type of bird to be identified with suitable family and attributes. Modify the
program to include it and show example of queries that will result in the new type of bird.

10.6 Tutorial Notes

The time allowance for tutorials in this topic is 2 hours.

Exercise 1:

In your opinion, what are characteristics that differentiate an expert systems and a conventional
system?

Exercise 2:

Based on the figure below , state and explain the FIVE (5) basic components of an expert system.

Page 61 of 80
AI Student Guide final 28.10
Exercise 3:

Database
eats_meat
wild

Knowledge Base
R1: barks → dog
R2: has_feathers → bird
R3: eat_meats → carnivore
R4: carnivore & lives_in_jungle → dangerous
R5: has_fur → mammal
R6: wild → lives_in_jungle
R7: pet → lives_in_house

(i) Based on the above figure, shows the inference process of forward chaining.
(ii) Based on the above figure, shows the inference process of backward chaining if the goal is
dangerous. If there is a conflict, always choose the rule at the top.

10.7 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1:

What are the criteria useful to decide when a rule-based expert system is adequate to solve a
problem?

Exercise 2:

Over the past decade a broad range of expert systems have been built to solve problems in different
areas. List some examples of application of expert systems. Also, state and briefly describe an
example of current expert system that available commercially or in the market.

Exercise 3:

What is an expert system shell? Give an example of expert system shell.

Page 62 of 80
AI Student Guide final 28.10
 Topic 11

Topic 11: Natural Language Processing

11.1 Learning Objectives

This topic begins with the introduction of natural language processing (NLP) and its importance.
Next, this topic discusses two main components of NLP: natural language understanding (NLU) and
natural language generation (NLG). The six linguistics levels in human language is presented,
followed by some common terminologies in NLP. Then, the five steps in NLP are provided along
with some examples of NLP tasks in each step. This topic also introduces major applications of NLP
such as speech recognition, speech synthesis, sentiment analysis, chatbots, machine translation,
question answering, and many more. Lastly, the challenges in NLP are discussed.

On completion of the topic, students will be able to:

• Explain natural language processing and its components.

• Discuss a range of applications using natural language processing.

11.2 Pedagogic Approach

Information will be transmitted to the students during the lectures. They will then practise the skills
during the laboratory sessions and extend their understanding during private study time. The tutorial
will then provide an opportunity to review the key ideas and obtain further guidance and support.

11.3 Timings

Lectures: 2 hours

Laboratory Sessions: 3 hours

Private Study: 8.5 hours

Tutorials: 2 hours

11.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time. Please also refer to
the slides.

The structure of this topic is as follows:

• What is NLP?
• Why NLP?
• Components of NLP
• Structure of Natural Language
• NLP Terminology
• Steps of NLP
• Applications of NLP
• Challenges in NLP

Page 63 of 80
AI Student Guide final 28.10
11.5 Laboratory Sessions

The laboratory time allocation for this topic is 3 hours.

Exercise 1: Install & Import NLTK & Resources

For the first time, you need to install the NLTK extension package. If you are using
Windows/Linux/Mac, you can install NLTK using Python’s Pip package installer: pip install
nltk. After successfully installing NLTK, you can import: import nltk and download additional
resources. For instance, the punkt module is a pre-trained model that helps you tokenize
sentences and words.

Exercise 2: Sentence & Word Tokenization

Sentence tokenizer breaks text paragraph into sentences, while word tokenizer breaks text
paragraph into words.

Note: To create strings that span multiple lines, triple single quotes ''' or triple double
quotes """ are used to enclose the string. By using the escape character \" and \’, we are able
to use double quotes to enclose a string that includes text quoted between double-quotes.

Page 64 of 80
AI Student Guide final 28.10
Exercise 3: Punctuation Removal

Punctuations are of little use in NLP, so they are removed. The isalpha( ) method is used to
separate the punctuation marks from the actual text. Also, we will make a new list called
words_nopunc, which will store the words in lower case but exclude the punctuation marks.

Exercise 4: Stopwords Removal

Notice that many words occur frequently but not very useful in the text analysis, such as “and”, “to”,
“of”, and others. These words are called stopwords and are usually removed as part of text
normalization. Create a list of stopwords and filter out your list of tokens from these words.

Exercise 5: Frequency Distribution

Let’s find the frequency of words in the sample text by using the FreqDist() method. Then, plot a
graph to visualize the word distribution.

Page 65 of 80
AI Student Guide final 28.10
Exercise 6: Word Cloud

Word Cloud is another text visualization technique, which more frequent or essential words are
displayed in a larger and bolder font, while less frequent or essential words are displayed in smaller
or thinner fonts. It is a beneficial technique in NLP that gives us a glance at what text should be
analysed.

Exercise 7: Stemming

Stemming is a process of text normalization, which reduces words to their based word or stem word
by truncating the derivational affixes. NLTK has different stemmers that implement different
methodologies like Porter Stemmer, Lancaster Stemmer, Snowball Stemmer, Lovin Stemmer,
Dawson Stemmer and etc. Notice that stemming may not give us a dictionary, grammatical word for
a particular set of words.

Exercise 8: Lemmatization

Lemmatization is another process of reducing inflexion from words. It reduces words to their base
word, which is recognisable and linguistically correct lemmas. Lemmatization is usually more
sophisticated than stemming. Stemmer works on an individual word with knowledge of the context.
For example, the word "better" has "good" as its lemma.

Lemmatization takes into account Part of Speech (PoS) values: verb (v), noun (n), adjective (a), and
adverb (r). It may generate different outputs for different values of POS. If the PoS is not indicated,
then the default value in lemmatization is a noun. Download the WordNet resource.

Page 66 of 80
AI Student Guide final 28.10
Exercise 9: Part of Speech Tagging

Part-of-Speech(POS) tagging is to identify the grammatical group of a given word: NOUN,

PRONOUN, ADJECTIVE, VERB, ADVERBS, etc. based on the context. POS Tagging looks for
relationships within the sentence and assigns a corresponding tag to the word. It is crucial for
syntactic and semantic analysis. There are different methods to tag, but we will be using the
universal style of tagging. Download the Averaged Perceptron Tagger resource.

Exercise 10: Chunking

Chunking (a.k.a shallow parsing) means to extract meaningful phrases from unstructured text. It
takes PoS tags as input and provides chunks as output, which breaks the text into phrases (groups
of words) that are more meaningful than individual words.

Exercise 11: Bag of Words

Bag of Words (BoW) is a method of extracting essential features from row text to use it for machine
learning models. A bag of words model converts the raw text into words, and it also counts the
frequency for the words in the text. It ignores the documents' grammar and context and is a mapping
of words to their counts in the text. Scikit-learn's CountVectorizer is used.

Page 67 of 80
AI Student Guide final 28.10
Exercise 12: Named Entity Recognition

Named entity recognition (NER) aims to find named entities in text and classify them into pre-
defined categories (names of persons, locations, organizations, times, etc.). Download the MaxEnt
NE Chunker and Words resources. NLTK provides a classifier that has already been trained to
recognize named entities, accessed with the function nltk.ne_chunk_sents().

Page 68 of 80
AI Student Guide final 28.10
11.6 Tutorial Notes

The time allowance for tutorials in this topic is 2 hours.

Exercise 1:
Give ONE example of a sentence for the following ambiguity. Justify your answers.

(a) Phonetic
(b) Syntactic
(c) Semantic
(d) Pragmatic

Page 69 of 80
AI Student Guide final 28.10
Exercise 2:
Consider using a parser with the following (partial) CFG grammar:

and (also partial) lexicon:

Show the top-down parsing by drawing the parse tree (or syntax tree) for the sentence:

The exports from the UK to Germany are increasing in 2020.

How many parse trees can be derived? If more than one, what does this indicate?

Page 70 of 80
AI Student Guide final 28.10
Exercise 3:

For each of the following NLP’s application, explore and name ONE specific
product/software/system currently available. Give a brief introduction on the product stating how
NLP is used.

(a) Speech recognition

(b) Speech synthesis
(c) Sentiment analysis
(d) Chatbots
(e) Machine translation
(f) Question answering
(g) Spam detection
(h) Spelling correction
(i) Text summarisation
(j) Text classification
(k) Information retrieval
(l) Information extraction

11.7 Private Study

The time allocation for private study in this topic is expected to be 8.5 hours.

Exercise 1: Semantic Analysis of Tweets

This analysis will use sample tweets from the NLTK package for NLP with different data cleaning
methods. Once the dataset is ready for processing, you will train a model on pre-classified tweets
and use it to classify the sample tweets into negative and positives sentiments. Type out the Python
code below by following the steps and find out all the outputs.
# In[1]:
import nltk
import re, string
import random
from nltk.corpus import twitter_samples
from nltk.tag import pos_tag
from nltk.stem.wordnet import WordNetLemmatizer
from nltk.corpus import stopwords
from nltk import FreqDist
from nltk import classify
from nltk import NaiveBayesClassifier
nltk.download('twitter_samples')
nltk.download('punkt')
nltk.download('wordnet')
nltk.download('averaged_perceptron_tagger')
nltk.download('stopwords')

# In[2]:
#Import three datasets from NLTK that contain various tweets
#to train and test the model
positive_tweets = twitter_samples.strings('positive_tweets.json')
negative_tweets = twitter_samples.strings('negative_tweets.json')
text = twitter_samples.strings('tweets.20150430-223406.json')

#Tokenize tweets
tweet_tokens = twitter_samples.tokenized('positive_tweets.json')
print(tweet_tokens[0]) #What is the output?

Page 71 of 80
AI Student Guide final 28.10
 # In[3]:
#PoS tagging
tweet_pos = pos_tag(tweet_tokens[0])
print(tweet_pos) #What is the output?

# In[4]:
#Lemmatization
def lemmatize_sentence(tokens):
lemmatizer = WordNetLemmatizer()
lemmatized_sentence = []
for word, tag in pos_tag(tokens):
if tag.startswith('NN'):
pos = 'n'
elif tag.startswith('VB'):
pos = 'v'
else:
pos = 'a'
lemmatized_sentence.append(lemmatizer.lemmatize(word, pos))
return lemmatized_sentence

print(lemmatize_sentence(tweet_tokens[0])) #What is the output?

# In[5]:
#Removing noise & stopwords
def remove_noise(tweet_tokens, stop_words = ()):
cleaned_tokens = []

for token, tag in pos_tag(tweet_tokens):

token = re.sub('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+#]|[!*\(\),]|'
'(?:%[0-9a-fA-F][0-9a-fA-F]))+','', token)
token = re.sub("(@[A-Za-z0-9_]+)","", token)

if tag.startswith("NN"):
pos = 'n'
elif tag.startswith('VB'):
pos = 'v'
else:
pos = 'a'

lemmatizer = WordNetLemmatizer()
token = lemmatizer.lemmatize(token, pos)

if len(token) > 0 and token not in string.punctuation and token.lower() not in

stop_words:
cleaned_tokens.append(token.lower())
return cleaned_tokens

stop_words = stopwords.words('english')
print(remove_noise(tweet_tokens[0], stop_words)) #What is the output?

# In[6]:
#Cleaning positive and negative tweets
positive_tweet_tokens = twitter_samples.tokenized('positive_tweets.json')
negative_tweet_tokens = twitter_samples.tokenized('negative_tweets.json')

positive_cleaned_tokens_list = []
negative_cleaned_tokens_list = []

for tokens in positive_tweet_tokens:

positive_cleaned_tokens_list.append(remove_noise(tokens, stop_words))

for tokens in negative_tweet_tokens:

negative_cleaned_tokens_list.append(remove_noise(tokens, stop_words))

print(positive_tweet_tokens[500]) #What is the output?

print(positive_cleaned_tokens_list[500]) #What is the output?

Page 72 of 80
AI Student Guide final 28.10
 # In[7]:
#Word Distributions
def get_all_words(cleaned_tokens_list):
for tokens in cleaned_tokens_list:
for token in tokens:
yield token

all_pos_words = get_all_words(positive_cleaned_tokens_list)

freq_dist_pos = FreqDist(all_pos_words)
print(freq_dist_pos.most_common(10)) #What is the output?

# In[8]:
#Converting tokens to a dictionary
def get_tweets_for_model(cleaned_tokens_list):
for tweet_tokens in cleaned_tokens_list:
yield dict([token, True] for token in tweet_tokens)

positive_tokens_for_model = get_tweets_for_model(positive_cleaned_tokens_list)
negative_tokens_for_model = get_tweets_for_model(negative_cleaned_tokens_list)

# In[9]:
#Splitting the dataset for training and testing the model
positive_dataset = [(tweet_dict, "Positive")
for tweet_dict in positive_tokens_for_model]

negative_dataset = [(tweet_dict, "Negative")

for tweet_dict in negative_tokens_for_model]

dataset = positive_dataset + negative_dataset

random.shuffle(dataset)

train_data = dataset[:7000]
test_data = dataset[7000:]

# In[10]:
#Building and testing the model with Naive Bayes Classifier
classifier = NaiveBayesClassifier.train(train_data)

#Accuracy is the % of tweets in the testing dataset for

#which the model was correctly able to predict the sentiment.
#What is the output?
print("Accuracy is:", classify.accuracy(classifier, test_data))

#Most informative features

#every row in the output shows the ratio of occurrence of
#a token in +ve:-ve tagged tweets in the training dataset
#What is the output?
print(classifier.show_most_informative_features(10))

References: https://www.digitalocean.com/community/tutorials/how-to-perform-sentiment-analysis-
in-python-3-using-the-natural-language-toolkit-nltk

Page 73 of 80
AI Student Guide final 28.10
Exercise 2: Semantic Analysis on Restaurant Reviews

The purpose of this analysis is to build a prediction model to predict whether a review on the
restaurant is good (1) or bad (0). To do so, we will work on Restaurant Review dataset and load it
into predictive algorithms.

Note: The dataset file (.tsv - tab-separated values) contains thousands of reviews for a restaurant.
Download from (https://codespeedy.com/wp-content/uploads/2019/07/Restaurant_Reviews.tsv) or at
Kaggle, and save the file in the same folder as the Python file.

Type out the Python code below and find out all the outputs.

# In[1]:
# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Importing the dataset

dataset = pd.read_csv('Restaurant_Reviews.tsv',
delimiter = '\t', quoting = 3)
dataset #Output?

Continued on next page

Page 74 of 80
AI Student Guide final 28.10
# In[2]:
# Cleaning the texts
import re
import nltk
nltk.download('stopwords')
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
corpus = []
for i in range(0, 1000):
review = re.sub('[^a-zA-Z]', ' ', dataset['Review'][i])
review = review.lower()
review = review.split()
ps = PorterStemmer()
review = [ps.stem(word) for word in review if not
word in set(stopwords.words('english'))]
review = ' '.join(review)
corpus.append(review)

# In[3]:
# Creating the Bag of Words model
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(max_features = 1500)
X = cv.fit_transform(corpus).toarray()
y = dataset.iloc[:, 1].values

# In[4]:
# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20,
random_state = 0)

# In[5]:
# Fitting Naive Bayes to the Training set
from sklearn.naive_bayes import GaussianNB
classifier = GaussianNB()
classifier.fit(X_train, y_train)

# In[6]:
# Predicting the Test set results
y_pred = classifier.predict(X_test)

# In[7]:
# Making the Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
cm #Output?

# In[8]:
from sklearn.metrics import accuracy_score
accuracy_score(y_test, y_pred) #Output?

Reference: https://www.codespeedy.com/natural-language-processing-and-its-implementation-in-
python/

Page 75 of 80
AI Student Guide final 28.10
 Topic 12

Topic 12: Intelligent Agents

12.1 Learning Objectives

This topic provides an overview of developing a small set of design principles for building successful
agents - systems that can reasonably be called intelligent. It begins by examining agents, properties
of intelligent behaviours, their structure and problem specification using the PEAS representation.
How well an agent can behave depends on the nature of the environment; some environment
characteristics are more difficult than others. The topic also described a number of basic “skeleton”
for intelligent agent designs.

Next, this topic introduces competitive multi-agent environments, in which the agents’ goals are in
conflict, giving rise to adversarial search problems—often known as games. The unpredictability of
these game playing agents can introduce contingencies into the agent’s problem-solving process
using heuristic search such as minimax and alpha-beta pruning algorithms.

On completion of the topic, students will be able to:

• Explain intelligent agents and PEAS.

• Discuss a range of applications using intelligent agents.

12.2 Pedagogic Approach

Information and theory of the topic will be presented to the students during lectures. They will then
practise the skills during the tutorial sessions. Students are expected to undertake their own private
study to understand the theory fully and put the lectures in context.

12.3 Timings

Lectures: 2 hours

Tutorials: 2 hours

Private Study: 8 hours

Page 76 of 80
AI Student Guide final 28.10
12.4 Lecture Notes

The following is an outline of the material to be covered during the lecture time and should be read
in conjunction with the slides provided.

The structure of this topic is as follows:

• What is an Agent?
• Properties of Intelligent Agents
• Structure of Intelligent Agents
• PEAS Representation
• Environment Characteristics
• Types of Intelligent Agents
• Game Playing Agent
• Types of Games
• Game Playing and Adversarial Search
• Minimax Search
• Alpha-Beta Pruning

12.5 Tutorial Notes

The time allowance for tutorials in this topic is 2 hours.

Exercise 1:

Define in your own words the following terms:

(a) agent

(b) agent function

(c) agent program

(d) rationality

(e) autonomous

(f) reflex agent

(g) model-based agent

(h) goal-based agent

(i) utility-based agent

(j) learning agent

Page 77 of 80
AI Student Guide final 28.10
Exercise 2:

For each of the following statement, determine whether it is True or False. Support your answer
with examples where appropriate.

(a) An agent that senses only partial information about the state cannot be perfectly rational.

(b) There exist task environments in which no simple reflex agent can behave rationally.

(c) There exists a task environment in which every agent is rational.

(d) The input to an agent program is the same as the input to the agent function.

(e) Every agent function is implementable by some program/machine combination.

(f) Suppose an agent selects its action uniformly at random from the set of possible actions. There
exists a deterministic task environment in which this agent is rational.

(g) It is possible for a given agent to be perfectly rational in two distinct task environments.

(h) Every agent is rational in an unobservable environment.

(i) A perfectly rational poker-playing agent never loses.

Exercise 3:

For each of the following activities, give a PEAS description of the task environment and
characterise it in terms of the properties: observable? agents? deterministic? episodic? static?
discrete?

(i) Playing soccer.

(ii) Exploring the subsurface oceans of Titan.
(iii) Shopping for used AI books on the Internet.
(iv) Practicing tennis against a wall.
(v) Bidding on an item at an auction.

Page 78 of 80
AI Student Guide final 28.10
Exercise 4:

Given the following game tree, perform:

(a) Minimax

(b) Alpha-beta pruning

12.6 Private Study

The time allocation for private study in this topic is expected to be 8 hours.

Exercise 1:

The vacuum environments have all been deterministic. Discuss possible agent programs for each of
the following stochastic versions:

a. Murphy’s law: twenty-five percent of the time, the Suck action fails to clean the floor if it is dirty
and deposits dirt onto the floor if the floor is clean. How is your agent program affected if the dirt
sensor gives the wrong answer 10% of the time?

b. Small children: At each time step, each clean square has a 10% chance of becoming dirty. Can
you come up with a rational agent design for this case?

Page 79 of 80
AI Student Guide final 28.10
Exercise 2:

Explore, explain and discuss (e.g. history, successes and algorithms) example of game playing
agents in the field of AI. Pick any two of the following games to discuss:

• Chess
• Reversi/Othello
• Checkers
• Backgammon
• Bridge
• Go
• Or any other games

Exercise 3:

Fill in each blank triangle with the proper minimax search value. What is MAX’s best move (write A
or B or C)?

Exercise 4:

Using the same tree as in Exercise 3. Cross out each leaf node that will be pruned by Alpha-Beta
Pruning. Mark “X” in each box corresponding to a pruned leaf node. What score does Max expect
to achieve?

End of guide

Page 80 of 80
AI Student Guide final 28.10