AI and its Application

Dr. Sachin Kumar Jain

PDPM IIITDM Jabalpur
skjain@iiitdmj.ac.in

Teachers open the door but you must enter by yourself

---Chinese Proverb
 Course Outline
Module-I
• What is AI? AI Concepts, Terminology, and Application Areas, AI: Issues,
Concerns and Ethical Considerations, The Future with AI, uninformed search,
Heuristic search.

Module II
• Uncertainty in AI, Uncertainty, Probability, Syntax and Semantics, Inference,
Independence and Bayes' Rule, Bayesian Network, Neural Networks, Support
Vector Machine.

Module III
• Classification & Regression, Supervised, Unsupervised and Reinforcement
Learning, Theory, concepts and applications.

Module IV
• Applications of AI.
 Grading
Evaluation/Part Weightage
Quiz 20%
End-Sem 60%
Class Performance$ 20%

$: Class performance includes your performance during the class

(interaction, answer to the questions, attendance, preparation of
class notes, etc.) and timely submission/completion of assignments.
 Reference Text
• Nilsson, N. J., Principles of Artificial Intelligence. Palo Alto,
CA: Tioga (1981).
• S. Rajasekaran and G.A.Vijaylakshmi Pai, Neural Networks
Fuzzy Logic, and Genetic Algorithms, Prentice Hall of India,
2003
• Nilsson, N. J. Artificial Intelligence - A Modern Synthesis.
Palo Alto: Morgan Kaufmann. (1998).
• T. Markiewicz and Josh Zheng, Getting Started with
Artificial Intelligence, O’Reilly Media. (2020).
 What I Expect…
You should:
• Be regular in class.
• Keep your mobile switched off / Silent in the class.
• Do not write anything while I am explaining something.
• Have discussions among yourselves on the subject. But not during the class.
• Never hesitate to ask any question during the class or after the class.

• Provide your feedback for any improvement without any fear as and when
it strikes to you.

• Maintain a notebook for this subject and note down key points. Attempt the
short/daily assignment problems in this notebook.

• In case you are not able to attend the class due to a valid reason, please drop
an email to me in advance. Please note that 75% attendance is mandatory in
this subject, the informed absence/leave will not be considered as present.
 Why this course ???
News clips
In the future, AI will be diffused into every aspect
of the economy.
—Nils J. Nilsson, founding researcher, Artificial
Intelligence & Computer Science, Stanford University

Courtesy: Google Images

 Why this course ???
News clips

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Why this course ???
News clips

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Why this course ???
News clips

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Why this course ???
Movies on AI

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Why this course ???
Movies on AI

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Why this course ???
AI in action today

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
Module-I
 Introduction
What is AI?
AI is an abbreviation of one of the most popular technologies
of recent time that means Artificial Intelligence.

Intelligence in psychology refers to the mental capacity to

learn from experiences, adapt to new situations,
understand and handle abstract concepts, and use
knowledge to manipulate one’s environment. It includes
skills such as problem-solving, critical thinking, learning
quickly, and understanding complex ideas.
 Introduction
What is AI?
AI is an abbreviation of one of the most popular technologies
of recent time that means Artificial Intelligence.

According to Wikipedia, Intelligence is defined as the

capacity for abstraction, logic, understanding, self-
awareness, learning, emotional knowledge, reasoning,
planning, creativity, critical thinking, and problem-solving.
It can be described as the ability to perceive or infer
information; and to retain it as knowledge to be applied to
adaptive behaviors within an environment or context.
 Introduction
What is AI?
AI is an abbreviation of one of the most popular technologies
of recent time that means Artificial Intelligence.

While there’s never really been an official, accepted definition,

a few high-level concepts shape and define what we mean by
AI.
It is the science and engineering of making intelligent
machines, especially intelligent computer programs. It is
related to the similar task of using computers to understand
human intelligence, but AI does not have to confine itself to
methods that are biologically observable.
 Introduction
What is AI?
 Introduction
What is AI?
 Introduction
What is AI?

To summarize, AI is something (a concept, a method, an

algorithm, a tool, a technique, a computation, … ) that has
the ability to:
• Perceive, reason and act.
• Solve problems, learn from data, discover meaning, and
mimic human.
• Make machines more productive with human like
decision-making.
 Introduction
AI’s official birth: Dartmouth, 1956

“An attempt will be made to find how to

make machines use language, form
abstractions and concepts, solve kinds of
problems now reserved for humans, and
improve themselves. We think that a
significant advance can be made if we
work on it together for a summer.”
John McCarthy and Claude Shannon
Dartmouth Workshop Proposal
 Introduction
A brief History of AI
• 1940-1950: Early days
– 1943: McCulloch & Pitts: Boolean circuit model of brain
– 1950: Turing's “Computing Machinery and Intelligence”
• 1950—70: Excitement: Look, Ma, no hands!
– 1950s: Early AI programs: chess, checkers (RL), theorem proving
– 1956: Dartmouth meeting: “Artificial Intelligence” adopted
– 1965: Robinson's complete algorithm for logical reasoning
• 1970—90: Knowledge-based approaches
– 1969—79: Early development of knowledge-based systems
– 1980—88: Expert systems industry booms
– 1988—93: Expert systems industry busts: “AI Winter”
• 1990— 2012: Statistical approaches + subfield expertise
– Resurgence of probability, focus on uncertainty
– General increase in technical depth
– Agents and learning systems… “AI Spring”?

• 2012— ___: Excitement: Look, Ma, no hands again?

– Big data, big compute, deep learning
– AI used in many industries
 Introduction
Key Areas/Directions/Topics in AI
• Search
• Knowledge
• Uncertainty
• Optimization
• Learning
• Neural Network
• Language
 Introduction
Terminology used in AI
• Agent
• State
• Actions
• Transition model
• State Space
• Solution
• Optimal Solution
 Terminology
Agent
• An Agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through actuators.

State
• A State is a representation of problem elements/AI
system at a given moment/current situation or
environment.
• This can be thought of as the "snapshot" of the
current situation of the system under consideration.
 Terminology
Actions
• Action is an activity, step, process, or function that
an agent performs given a specific state of the
environment. Actions is a description of the possible
actions that the agent can take.

Path Cost
• Path cost assigns a numerical cost to each path that
leads to the goal. The problem-solving agents choose
a cost function that matches its performance
measure.
 Terminology
Transition model
• Transition model describes what each action does.
It is specified by a function Result(s, a) that returns
the state that results from doing action a in state s.

State Space
• The initial state, actions, and transition model
together define the state space of a problem, a set of
all states reachable from the initial state by any
sequence of actions.
 Terminology
Goal Test
• The goal test determines whether a given state is a
goal state.

Solution
• A Solution to a problem is an action sequence that
leads from the initial state to a goal state.

Optimal solution
• An optimal solution is a solution that has the lowest
path cost of all the solutions.
 Agents and environments
Agent
• An Agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through actuators.
 Agents and environments

Agent Environment
Sensors
Percepts
?
Actuators
Actions

• An agent perceives its environment through sensors

and acts upon it through actuators (or effectors,
depending on whom you ask)

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agents and environments

Agent Environment
Sensors
Percepts
?
Actuators
Actions

• Are humans agents?

• Yes!
– Sensors = vision, audio, touch, smell, taste, proprioception
– Actuators = muscles, secretions, changing brain state

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agents and environments

Agent Environment
Sensors
Percepts
?
Actuators
Actions

• Are pocket calculators agents?

• Yes!
– Sensors = key state sensors
– Actuators = digit display

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agents and environments

Agent Environment
Sensors
Percepts
?
Actuators
Actions

• AI is more interested in agents with large

computational resources and environments that
require nontrivial decision making

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agent functions

• The agent function maps from percept histories to actions:

– f : P* → A
– I.e., the agent’s actual response to any sequence of percepts
NEXT NEXT NEXT NEXT
Percept

Action LEFT LEFT DROP RIGHT

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agent programs
• The agent program l runs on some machine M to
implement f :
– f = Agent(l,M)
– Real machines have limited speed and memory, introducing
delay, so agent function f depends on M as well as l
NEXT NEXT NEXT NEXT
Percept

Action NOOP NOOP NOOP LEFT

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Agent functions and agent programs

• Can every agent function be implemented by

some agent program?
• No! Consider chess with a slow PC

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Example: Vacuum world

A B

• Percepts: [location,status], e.g., [A,Dirty]

• Actions: Left, Right, Suck, NoOp

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Vacuum cleaner agent A B

Agent function Agent program

Percept sequence Action function Reflex-Vacuum-Agent([location,status])

[A,Clean] Right
returns an action
if status = Dirty then return Suck
[A,Dirty] Suck
else if location = A then return Right
[B,Clean] Left else if location = B then return Left
[B,Dirty] Suck
[A,Clean],[B,Clean] Left
[A,Clean],[B,Dirty] Suck What is the right agent function?
etc etc
Can it be implemented by a small agent program?

(Can we ask, “What is the right agent program?”)

Courtesy: Lecture notes from Prof. Stuart Russell and Dawn Song
 Rationality
• A rational agent does the right thing (what is this?)
• Rationality is not the same as perfection.
• Rationality maximizes expected performance, while
perfection maximizes actual performance.
• A fixed performance measure evaluates the
sequence of observed action effects on the
environment.
 Rationality A B

• Fixed performance measure evaluates the environment sequence

– one point per square cleaned up?
• NO! Rewards an agent who dumps dirt and cleans it up
– one point per clean square per time step, for t = 1,…,T
• A rational agent chooses whichever action maximizes the
expected value of the performance measure
– given the percept sequence to date and prior knowledge of
environment
Does Reflex-Vacuum-Agent implement a rational agent function?
Yes, if movement is free, or new dirt arrives frequently
 Rationality
• Are rational agents omniscient?
– No – they are limited by the available percepts
• Are rational agents clairvoyant?
– No – they may lack knowledge of the environment dynamics
• Do rational agents explore and learn?
– Yes – in unknown environments these are essential
• Do rational agents make mistakes?
– No – but their actions may be unsuccessful
• Are rational agents autonomous (i.e., transcend initial program)?
– Yes – as they learn, their behavior depends more on their own
experience
 PEAS
• Use PEAS to describe task environment
– Performance measure: Criterion for success of an
agent’s behaviour.
– Environment: Surrounding conditions/factors that
influence/affect an agent’s behaviour.
– Actuators: Mechanism to initiate action by an
agent.
– Sensors: Means to get perceptual inputs that
helps to perceive the environment.
 PEAS: Automated taxi
• Performance measure
– Income, happy customer, vehicle costs,
fines, insurance premiums
• Environment
– Streets, other drivers, customers,
weather, police…
• Actuators
– Steering, brake, gas, display/speaker
• Sensors
– Camera, radar, accelerometer, engine
sensors, microphone, GPS

Image: http://nypost.com/2014/06/21/how-google-
might-put-taxi-drivers-out-of-business/
 PEAS: Medical diagnosis system

• Performance measure
– Patient health, cost, reputation
• Environment
– Patients, medical staff, insurers,
courts
• Actuators
– Screen display, email
• Sensors
– Keyboard/mouse
 The task environment - PEAS

• Performance measure
– -1 per step; + 10 food; +500 win; -500 die;
+200 hit scared ghost
• Environment
– Pacman dynamics (incl ghost behavior)
• Actuators
– Left Right Up Down
• Sensors
– Entire state is visible (except power pellet
duration)
 Environment Properties
• Fully observable vs. partially observable
• Deterministic vs. stochastic / strategic
• Episodic vs. sequential
• Static vs. dynamic
• Discrete vs. continuous
• Single agent vs. multiagent
 Environment Properties
Fully observable vs. partially observable
• The task environment is said to be fully observable if
an agent’s sensors give it access to the complete
state of the environment at each point in time.

• A partially observable environment might have few

missing states from the sensor data, may be due
noisy, inaccurate or insufficient sensors.
 Environment Properties
Deterministic vs. stochastic / strategic
• If next state completely determined by current state
and action then task environment is deterministic,
otherwise stochastic.

• If an agent/environment is deterministic but the next

state depends on actions of other agents, then it is
strategic.
 Environment Properties
Episodic vs. sequential
• In Episodic task environment, action choice depends
only on current state. The next episode does not
depend on the actions taken in previous episodes.
• In Sequential, current action may affect future
actions/decisions.

Static vs. dynamic

• Dynamic environment can change while agent is
deliberating, otherwise it is static.
 Environment Properties
Discrete vs. continuous
• This distinction is applied to the state, to the way
time is handled, and to the percepts, or actions of
the agent.
• For example: Chess has finite number of distinct states
and has a discrete set of percepts and actions. Whereas,
taxi-driving is a continuous state problem.

Single agent vs. multiagent

• If other's behavior maximizes its performance
based on agent, then it is multiagent.
• For example: crossword puzzle is a single agent whereas
an agent playing chess is in a two-stage environment.
 Agent design
• The environment type largely determines the agent
design
– Partially observable => agent requires memory (internal
state)
– Stochastic => agent may have to prepare for contingencies
– Multi-agent => agent may need to behave randomly
– Static => agent has time to compute a rational decision
– Continuous time => continuously operating controller
– Unknown physics => need for exploration
– Unknown perf. measure => observe/interact with human
principal
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock

Chess without a clock

Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis Fully Determi Episodic Semi Discrete Single
nistic
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis Fully Determi Episodic Semi Discrete Single
nistic
Robot part picking
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis Fully Determi Episodic Semi Discrete Single
nistic
Robot part picking Fully Determi Episodic Semi Discrete Single
nistic
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis Fully Determi Episodic Semi Discrete Single
nistic
Robot part picking Fully Determi Episodic Semi Discrete Single
nistic
Interactive English
tutor
Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
Medical diagnosis Partial Stochast Episodic Static Continu Single
ic ous
Image analysis Fully Determi Episodic Semi Discrete Single
nistic
Robot part picking Fully Determi Episodic Semi Discrete Single
nistic
Interactive English Partial Stochast Sequential Dyna Discrete Multi
tutor ic mic
 Agent types
• In order of increasing generality and complexity,
the agent programs are classified as:
– Simple reflex agents
– Reflex agents with state
– Goal-based agents
– Utility-based agents
 Simple reflex agents
These agents select action on the basis of the current percept,
ignoring the rest of the percept history. For example: vacuum agent
that just checks the current location and dirt in it.
It follows condition-action rule, in which a condition triggers an
established connection in the agent program to the action.
Humans also have many Agent Sensors

such connections, some of

What the world
which are learned is like now
responses (if car-in-front-

Environment
Current internal states
is-braking then initiate-
braking) and some of
which are innate reflex
(blinking eye on sudden …). Condition-action rules What action I
should do now

Background information
Actuators
used in the process
 Simple reflex agents
The advantage of Simple reflex agents: Simple. But they turn out to
be of very limited intelligence.
A little bit of unobservability can cause serious trouble.

For example: the vacuum Agent Sensors

agent has only dirt sensor

What the world
working, then two possible is like now

percept will be: Dirty and

Environment
Current internal states
Clean. It will Suck in
response to Dirty, but it
will fail to move left or
What action I
right, in response to Clean. Condition-action rules should do now

Background information
Actuators
used in the process
 Reflex agents with state
These are known as model-based reflex agents. It maintains
internal state that depend on the percept history, and thereby
reflects at least some of the unobserved aspects of the current state.
Updating this internal state information requires two kinds of
knowledge to be encoded in the agent program.

1. Information about how Sensors

world evolves. State

How the world evolves What the world
is like now
2. Information about how

Environment
agent’s own action What my actions do Current
affect the world. internal states
Background information
This is implemented using used in the process
What action I
model of the world. Condition-action rules
should do now

Agent Actuators
 Goal-based agents
The state of the environment is not always enough to decide what
to do. The agent needs some sort of goal information that describes
situations that are desirable.
The agent program can combine this with information about the
results of possible actions in order to choose actions that achieve
the goal.
Sensors

Decision making of this kind State

What the world
is fundamentally different How the world evolves is like now

from the condition-action

Environment
What it will be like
What my actions do
rule, as it considers the if I do action A

future:- “what will happen if

I do such and such”, and
What action I
“will that make me happy”? Goals should do now

Agent Actuators
 Goal-based agents
The state of the environment is not always enough to decide what
to do. The agent needs some sort of goal information that describes
situations that are desirable.
The agent program can combine this with information about the
results of possible actions in order to choose actions that achieve
the goal.
Sensors

Pro: More flexible, use goal State

What the world
to index into actions that How the world evolves is like now

might achieve it, e.g.

Environment
What it will be like
What my actions do if I do action A
“Have milk” → “buy milk”
Con: Cannot handle tradeoffs
among goals, appears less Goals
What action I
should do now
efficient, failure probability
etc. Agent Actuators
 Utility-based agents
Goals alone are not really enough to generate high-quality behavior
in most environments. A utility function maps a state onto a real
number, which describes the associated degree of happiness. It
allows handling conflicting goals (e.g. speed/time vs safety in
drive). Utility is a numeric scale.
A complete specification
Sensors
of the utility function
State
allows rational decisions What the world
How the world evolves is like now
with multiple and/or

Environment
conflicting goals. What my actions do
What it will be like
if I do action A

Pro: can compute expected How happy I will be

Utility
values for actions, handle in such a state

tradeoffs and uncertainty. What action I

should do now
Con: cannot (easily) index
into actions. Agent Actuators
 Learning agents
• Learning element is responsible for making improvements.
• Performance element is responsible for selecting external
actions.
• The learning element uses feedback from critic on how agent is
doing and determines
how performance
element should be
modified to do better in
future.
• Problem generator is
responsible for
suggesting actions that
will lead to new and
informative experiences.
 Summary

• An agent interacts with an environment through sensors and

actuators
• The agent function, implemented by an agent program running
on a machine, describes what the agent does in all circumstances
• Rational agents choose actions that maximize their expected utility
• PEAS descriptions define task environments; precise PEAS
specifications are essential and strongly influence agent designs
• More difficult environments require more complex agent designs
and more sophisticated representations
 Assignment #1
1. Develop a PEAS description of the task environment:
A. English tutor
B. Automatic vehicle
2. Design all four types of AI agents for an automatic car
and compare their performance on different parameters
(e.g. simplicity, robustness (performance when any sensor
is having noise, etc.), goal, safety, comfort, etc.
 Search
Search: A problem-solving agent
• Searching is a step-by-step procedure to solve a problem in a
given search space.
• Search techniques are universal problem-solving methods.
• Rational agents or Problem-solving agents in AI mostly use
a search strategy or algorithm to solve a specific problem
and provide the best result.
• Problem-solving agents are a kind of goal-based agent.
• A search algorithm takes a problem as input and returns a
solution in the form of an action sequence.
 Search
Search: Terminology
The search methods are defined mostly on trees and graphs, so
let’s discuss some terminology for these structures:

• A tree is made up of nodes and links

(circles and lines) connected so that
there are no loops (cycles). Nodes are
sometimes referred to as vertices and
links as edges.
 Search
Search: Terminology
• A tree has a root node (where the tree "starts"). Every node
except the root has a single parent (aka direct ancestor).
More generally, an ancestor node is a node that can be
reached by repeatedly going to a parent node.
• Each node (except the terminal (aka leaf) nodes) has one or
more children (aka direct descendants). More generally, a
descendant node is a node that can be reached by repeatedly
going to a child node.
 Search
Search: Terminology
• A tree has a root node (where the tree "starts"). Every node
except the root has a single parent (aka direct ancestor).
More generally, an ancestor node is a node that can be
reached by repeatedly going to a parent node.
• Each node (except the terminal (aka leaf) nodes) has one or
more children (aka direct descendants). More generally, a
descendant node is a node that can be reached by repeatedly
going to a child node.
 Search
Search: Terminology

Root Node
Links
Parent of X

Node-X
Child of X

Terminal Node
 Search
Search: Terminology
• A graph is also a set of nodes connected by links but where
loops are allowed and a node can have multiple parents. We
have two kinds of graphs to deal with:
• Directed graphs, where the links have direction (akin to
one-way streets).
• Undirected graphs, where the links go both ways. You can
think of an undirected graph as shorthand for a graph with
directed links going each way between connected nodes.
 Search
Search: Terminology
• A graph is also a set of nodes connected by links but where
loops are allowed and a node can have multiple parents. We
have two kinds of graphs to deal with:
 Search
Search: Graph search vs Tree search
• Trees are a subclass of directed graphs, which don't have
cycles and every node has a single parent.
• Cycles are bad for searching, since, obviously, you don't
want to go round and round getting nowhere.
• A graph can be converted to an equivalent problem of
searching a tree by doing two things: turning undirected
links into two directed links; and, more importantly, making
sure we never consider a path with a loop or, even better, by
never visiting the same node twice.
 Search
Search: Graph search vs Tree search
• Example: We have a tree with S as the start point, and our
goal is to find a path to reach G.
 Search
Search: As a problem-solving agent
One general approach to problem solving in AI is to reduce
the problem to be solved to one of searching a graph. To use
this approach, we must specify what are the states, the
actions and the goal test.
Assumptions
The actions are deterministic, that is, we know exactly the
state after the action is performed.
The actions are discrete, so we don't have to represent what
happens while the action is happening.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S

Search space or state space represents a set of possible

solutions, which a system may have.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0

It is a state from where agent begins the search.

 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0
– Actions A(s) in each state N -9

E -9

Actions gives the description of all the available actions to the

agent.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0
– Actions A(s) in each state N -9

– Transition model Result(s,a) E -9

A description of what each action do, can be represented as a

transition model.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0
– Actions A(s) in each state N -9

– Transition model Result(s,a) E -9

– A goal test G(s)

• S has no dots left

Goal test is a function which observe the current state and

returns whether the goal state is achieved or not.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0
– Actions A(s) in each state N -9

– Transition model Result(s,a) E -9

– A goal test G(s)

• S has no dots left
– Action cost c(s,a,s’)
• +1 per step; -10 food; -500 win; +500 die; -200 eat ghost

Action/path cost is a function which assigns a numeric cost to

each action/path.
 Search
What a search problem should have?
• A search problem consists of:
– A state space S
– An initial state s0
– Actions A(s) in each state N -9

– Transition model Result(s,a) E -9

– A goal test G(s)

• S has no dots left
– Action cost c(s,a,s’)
• +1 per step; -10 food; -500 win; +500 die; -200 eat ghost
• A solution is an action sequence that reaches a goal state
• An optimal solution has least cost among all solutions
 Search
Search: Key point to remember
An important distinction that will help us keep things straight
is that between a state and a search node.
• A state is an arrangement of the real world (or at least our
model of it). We assume that there is an underlying "real"
state graph that we are searching (although it might not be
explicitly represented in the computer; it may be implicitly
defined by the actions).
• We assume that you can arrive at the same real-world state
by multiple routes, that is, by different sequences of
actions.
 Search
Search: Key point to remember
An important distinction that will help us keep things straight
is that between a state and a search node.
• A search node, on the other hand, is a data structure in the
search algorithm, which constructs an explicit tree of
nodes while searching.
• Each node refers to some state (of real-world), but not
uniquely (many nodes may refer to the same node).
• A node also corresponds to a path from the start state to the
state associated with the node. This follows from the fact
that the search algorithm is generating a tree. So, if we
return a node, we’re returning a path.
 Search
Classification
Based on the search problems we can classify the search
algorithms into uninformed (Blind search) search and
informed search (Heuristic search) algorithms.
The uninformed search does not contain any domain
knowledge such as closeness, the location of the goal. It
operates in a brute-force way as it only includes information
about how to traverse the tree and how to identify leaf and
goal nodes. Uninformed search applies a way in which search
tree is searched without any information about the search
space like initial state operators and test for the goal, so it is
also called blind search. It examines each node of the tree
until it achieves the goal node.
 Search
Classification
Based on the search problems we can classify the search
algorithms into uninformed (Blind search) search and
informed search (Heuristic search) algorithms.
Informed search algorithms use domain knowledge. In an
informed search, problem information is available which can
guide the search. Informed search strategies can find a
solution more efficiently than an uninformed search strategy.
Informed search is also called a Heuristic search.

A heuristic is a way which might not always be guaranteed

for best solutions but guaranteed to find a good solution in
reasonable time.
 Search
Classification

The simplest class is that of the uninformed, any-path algorithms. In

particular, we will look at depth-first and breadth-first search. Both
of these algorithms basically look at all the nodes in the search tree in
a specific order (independent of the goal) and stop when they find the
first path to a goal state.
 Search
Classification

The next class of methods are informed, any-path algorithms. The

key idea here is to exploit a task specific measure of goodness to try
to either reach the goal more quickly or find a more desirable goal
state.
 Search
Classification

Uninformed, optimal algorithms guarantee finding the "best" path

but do not use any information beyond what is in the graph definition.
 Search
Classification

Informed, optimal algorithms, also guarantee finding

the best path but these exploit heuristic ("rule of thumb")
information to find the path faster than the uninformed
methods.
 Search
General Tree Search
In a search process, data is often stored in a node, a data structure
that contains the following data:
• A state
• Its parent node, through which the current node was generated
• The action that was applied to the state of the parent to get to the
current node
• The path cost from the initial state to this node

Nodes contain information that makes them very useful for the
purposes of search algorithms. They contain a state, which can be
checked using the goal test to see if it is the final state.
 Search
General Tree Search
However, nodes are simply a data structure — they don’t search,
they hold information. To actually search, we use the frontier, the
mechanism that “manages” the nodes.
The frontier starts by containing an initial state and an empty set of
explored items, and then repeats the following actions until a
solution is reached:
 Search
General Tree Search

• Main variations:
– Which leaf node to expand next
– Whether to check for repeated states
– Data structures for frontier, expanded nodes
 Search
Measuring performance of Search Algorithms

We can evaluate a search algorithm’s performance using the

following four properties:
• Completeness: Is the algorithm guaranteed to find a solution
when there is one?

• Optimality: Does the strategy find the optimal solution?

• Time complexity: How long does it take to find a solution?

• Space complexity: How much memory is needed to perform

the search?
 Search
Measuring performance of Search Algorithms

• The typical measure for time and space complexity is the size
of the state space graph, |V | + |E|, where V is the set of
vertices (nodes) of the graph and E is the set of edges (links).

• Complexity is expressed in terms of three quantities:

o b - maximum branching factor of the search tree
o d - depth of the least-cost solution
o m - maximum depth of the state space (may be ∞).

• Time is often measured in terms of the number of nodes

generated during the search, and space in terms of the
maximum number of nodes stored in memory.
 Search
Depth-first Search

Depth-first search always expands deepest unexpanded node in

frontier of search tree.
• As nodes are expanded, they are dropped from frontier, so then
search “backs up” to next deepest node that still has unexplored
successors.

In Depth-first search, the frontier is managed as a stack data structure

that uses a LIFO queue.
• A LIFO queue means that the most recently generated node is
chosen for expansion.
• This must be the deepest unexpanded node because it is one
deeper than its parent.
 Search
Depth-first Search: Example
Strategy: expand a
Goal: Node G
a G
deepest node first b c
e
d f
Implementation: S h
Frontier is a LIFO stack p q r

d e p

b c e h r q

a a h r p q f

p q f q c G

q c G a

a Frontier:
Frontier: A,
D
B,
S C, E

We start with a frontier that contains the initial state.

 Search
Depth-first Search

d e p

b c e h r q

a a h r p q f

p q f q c G

q c G a

a
 Search
Depth-first Search

# Define the function that removes a node from the frontier and returns it.
def remove(self):
# Terminate the search if the frontier is empty, because this means that there is no solution.
if self.empty():
raise Exception("empty frontier")
else:
# Save the last item in the list (which is the newest node added)
node = self.frontier[-1]
# Save all the items on the list besides the last node (i.e. removing the last node)
self.frontier = self.frontier[:-1]
return node
 Search
Depth-first Search
• Complete: Guaranteed to find a solution if one exists?
• Optimal: Guaranteed to find the least cost path?
• Time complexity?
1 node
b
• Space complexity? … b nodes
b2 nodes
• Cartoon of search tree: m tiers
– b is the branching factor
– m is the maximum depth
– solutions at various depths bm nodes

• Number of nodes in entire tree?

– 1 + b + b2 + …. bm = O(bm)
 Search
Depth-first Search: Properties
• What nodes does DFS expand?
– Some left prefix of the tree down to depth b
1 node
m. … b nodes
– Could process the whole tree! b2 nodes
– If m is finite, takes time O(bm) m tiers

• How much space does the frontier take?

– Only has siblings on path to root, so O(bm)
bm nodes
• Is it complete?
– m could be infinite
– preventing cycles may help (more later)

• Is it optimal?
– No, it finds the “leftmost” solution,
regardless of depth or cost
 Search
Breadth-first Search

Breadth-first search is a simple strategy in which the root node is

expanded first, then all the successors of the root node are expanded
next, then their successors, and so on.
• All nodes are expanded at a given depth in search tree before
any nodes at next level are expanded.

In Breadth-first search, the frontier is managed as a queue data

structure that uses a FIFO queue.
• A FIFO queue means that the node added first is chosen for
expansion.
• This must be the deepest unexpanded node because it is one
deeper than its parent.
 Search
Breadth-first Search: Example
Goal: Node G
Strategy: expand a a G
b c
shallowest node first
e
d f
Implementation: Frontier S h
is a FIFO queue p q r

d e p
Search
b c e h r q
Tiers
a a h r p q f

p q f q c G

q c G a

a
 Search
Breadth-first Search: Pseudo Code
 Search
Breadth-first Search: Properties

• What nodes does BFS expand?

– Processes all nodes above shallowest b
1 node
solution … b nodes
s tiers
– Let depth of shallowest solution be s b2 nodes
– Search takes time O(bs)
bs nodes
• How much space does the frontier take?
– Has roughly the last tier, so O(bs)
bm nodes

• Is it complete?
– s must be finite if a solution exists, so yes!

• Is it optimal?
– If costs are equal (e.g., 1)
 Search
Breadth-first Search: Properties
 Search
Uniform Cost Search
 Search
Uniform Cost Search
2 a G
g(n) = cost from root to n c
b
1 8 2
Strategy: expand lowest g(n) 2 e
3 d f
9 2
S h 8 1
Frontier is a priority queue
1 p q r
sorted by g(n) 15

S 0

d 3 e 9 p 1

b 4 c e 5 h 17 r 11 q 16
11
Cost a 6 a h 13 r 7 p q f
contours
p q f 8 q c G

q 11 c G 10 a

a
 Search
Uniform Cost Search: Properties
• What nodes does UCS expand?
– Processes all nodes with cost less than cheapest solution!
– If that solution costs C* and arcs cost at least  , then the
“effective depth” is roughly C*/
– Takes time O(bC*/) (exponential in effective depth)

• How much space does the frontier take? b g1

…
– Has roughly the last tier, so O(bC*/) g2
C*/ “tiers”
g3
• Is it complete?
– Assuming C* is finite and  > 0, yes!

• Is it optimal?
– Yes! (Proof next lecture via A*)
 Search
Uninformed vs Informed Search

Guide search towards the goal instead of all over the place

Start Goal Start Goal

Informed Uninformed
 Search
Informed Search

• Uninformed search algorithms do not utilize any knowledge about

the problem that they did not acquire through their own
exploration. However, most often is the case that some knowledge
about the problem is, in fact, available.
• A type of algorithm that considers additional knowledge to try to
improve its performance is called an informed search algorithm.
• In such algorithms, a node is selected for expansion based on an
evaluation function, f(n). The node with the lowest evaluation is
selected for expansion.
• Popular algorithms under this category are: Greedy Best-first
search, A* search, Hill climbing, Beam search, etc.
 Search
Best-First Search

• There is a family of Best-first search algorithms with different

evaluation functions.
• Best-first search expands the node that is the closest to the goal,
as determined by a heuristic function h(n).
• The efficiency of the best-first algorithm depends on how good the
heuristic function is.
• As its name suggests, the function estimates how close to the goal
the next node is, but it can be mistaken.
• As with any heuristic, it can go wrong and lead the algorithm down
a slower path than it would have gone otherwise.
 Search
Heuristic function

• A heuristic function h(n) is the estimated cost of the cheapest path

from the state at node n to a goal state.
• Heuristic functions are the most common form in which additional
knowledge of the problem is imparted to the search algorithm.
• Heuristic functions are nonnegative, problem-specific functions,
with one constraint: if n is a goal node, then h(n)=0.
 Search
Greedy Best-First Search

• Greedy Best-first search expands the node that is the closest to the
goal, as determined by a heuristic function h(n).
• It evaluates nodes by using just the heuristic function: f(n) = h(n).

• Let’s consider an example of a route-finding problem from a city in

Romania to other.
• The heuristic function

is used an evaluation function.

 Search
Greedy Best-First Search: Example
Oradea
71
Neamt

Zerind 87
151
75
Iasi
Arad
140
92
Sibiu Fagaras
99
118
Vaslui
80
Rimnicu Vilcea
Timisoara
142 h(n) = straight-line distance to Bucharest
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu
 Search
Greedy Best-First Search
 Search
Greedy Best-First Search
 Search
Greedy Best-First Search
 Search
Greedy Best-First Search
 Search
Greedy Best-First Search: Properties

• Greedy best-first search resembles depth-first search (it prefers to

follow a single path all the say to the goal but will back up when it hits
a dead end).
 Search
Greedy Best-First Search: Properties
 Search
A* Search

• Expand a node n most likely to be on the optimal path

• Expand a node n s.t. the cost of the best solution through n is
optimal
• Expand a node n with lowest value of g(n) + h*(n)
– g(n) is the cost from root to n
– h*(n) is the optimal cost from n to the closest goal
• We seldom know h*(n) but might have a heuristic
approximation h(n)
• A* = tree search with priority queue ordered by g(n) + h(n)
 Search
A* Search: Example
Oradea
71
Neamt

Zerind 87
151
75
Iasi
Arad
140
92
Sibiu Fagaras
99
118
Vaslui
80
Rimnicu Vilcea
Timisoara
142 h(n) = straight-line distance to Bucharest
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu
Search
Search
Search
Search
Search
Search
Search
 Search
A* Search: Optimality ???
 Search
A* Search: Optimality ???

Assume:
• A is an optimal goal node …
• B is a suboptimal goal node
• h is admissible
A
Claim: B
• A will be chosen for expansion before B
 Search
A* Search: Optimality ???

Proof:
…
• Imagine B is on the frontier
• Some ancestor n of A is on the frontier, n
too (maybe A itself!)
• Claim: n will be expanded before B A
B
1. f(n) is less than or equal to f(A)

f(n) = g(n) + h(n) Definition of f-cost

f(n)  g(A) Admissibility of h
g(A) = f(A) h = 0 at a goal
 Search
A* Search: Optimality ???

Proof:
…
• Imagine B is on the frontier
• Some ancestor n of A is on the frontier, n
too (maybe A itself!)
• Claim: n will be expanded before B A
B
1. f(n) is less than or equal to f(A)
2. f(A) is less than f(B)

g(A) < g(B) Suboptimality of B

f(A) < f(B) h = 0 at a goal
 Search
A* Search: Optimality ???

Proof:
…
• Imagine B is on the frontier
• Some ancestor n of A is on the frontier,
n
too (maybe A itself!)
• Claim: n will be expanded before B A
1. f(n) is less than or equal to f(A) B
2. f(A) is less than f(B)
3. n is expanded before B
• All ancestors of A are expanded before B
• A is expanded before B f(n)  f(A) < f(B)
• A* tree search is optimal
 Search
A* Search: Properties
 Search
Creating Admissible Heuristics
Let us consider an example of 8-puzzle to learn how to create
admissible heuristics.

Start State Actions Goal State

• What are the states?

• How many states? • 322=3.1x1010. If repeated states are tracked, it is 9!/2=181,440.
• What are the actions? • Left, Right, Up, Down
• What are the step costs? • May be considered as 1 per step.
 Search
Creating Admissible Heuristics
• Heuristic-1: Number of tiles misplaced
• Why is it admissible?
• h(start) = 8

Start State Goal State

• Heuristic-2: Total Manhattan distance

• Why is it admissible?

• h(start) = 3 + 1 + 2 + … = 18
 Search
Local search algorithms
The uninformed and informed search algorithms that we have seen
are designed to explore search spaces systematically.
• They keep one or more paths in memory and by record which
alternatives have been explored at each point
along the path.
• When a goal is found, the path to that goal also constitutes a solution to
the problem.
• In many problems, however, the path to the goal is irrelevant.

If the path to the goal does not matter, we might consider a different
class of algorithms that do not worry about paths at all.
 Search
Local search algorithms
• Local search algorithms operate using a single current node and generally
move only to neighbors of that node.
• Local search algorithms ease up on completeness and optimality in the interest
of improving time and space complexity?
• Although local search algorithms are not systematic, they have two key
advantages:
▪ They use very little memory (usually a constant amount), and
▪ They can often find reasonable solutions in large or infinite (continuous) state spaces.
• In addition to finding goals, local search algorithms are useful for solving pure
optimization problems, in which the aim is to find the best state according to
an objective function.
▪ In optimization problems, the path to goal is irrelevant and the goal state itself is the
solution.
▪ In some optimization problems, the goal is not known and the aim is to find the best
state.
 Search
Local search algorithms: Hill Climbing

• Also known as: Steepest Ascent/Descent method

• Simple, general idea:
– Start wherever
– Repeat: move to the best neighboring state
– If no neighbors better than current, quit
 Search
Local search algorithms: Hill Climbing
 Search
Heuristic for n-queens problem
• Goal: n queens on board with no conflicts, i.e.,
no queen attacking another
• States: n queens on board, one per column
• Heuristic value function: number of conflicts
 Search
Local search algorithms: Hill Climbing

function HILL-CLIMBING(problem) returns a state

current ← make-node(problem.initial-state)
loop do
neighbor ← a highest-valued successor of current
if neighbor.value ≤ current.value then
return current.state
current ← neighbor

“Like climbing Everest in thick fog with amnesia”

 Search
Local search algorithms: Global and local maxima
• To understand local search, it is useful to consider the state-space landscape.
• The aim is to find the highest peak - a global maximum.
• Hill-climbing search modifies the current state to try to improve it.

Random restarts
– find global optimum
– duh

Random sideways
moves
– Escape from
shoulders
– Loop forever on flat
local maxima
 Search
Local search algorithms: Global and local maxima
• A local maximum is a peak that is higher than each of its neighboring states but
lower than the global maximum.
o Hill-climbing algorithms that reach the vicinity of a local maximum will be
drawn upward toward the peak but will then be stuck with nowhere else to go.
• A plateau is a flat area of the state-space landscape. It can be a flat local maximum,
from which no uphill exit exists, or a shoulder, from which progress is possible.

o A hill-climbing search might get lost

on the plateau.
o Random sideways moves can escape
from shoulders but they loop forever
on flat maxima.
 Search
Local search algorithms: Example
Hill-climbing on the 8-queens problem

• No sideways moves:
– Succeeds w/ prob. 0.14
– Average number of moves per trial:
• 4 when succeeding, 3 when getting stuck
– Expected total number of moves needed:
• 3(1-p)/p + 4 =~ 22 moves
• Allowing 100 sideways moves:
– Succeeds w/ prob. 0.94
– Average number of moves per trial:
• 21 when succeeding, 65 when getting stuck
– Expected total number of moves needed: Moral: algorithms with knobs
• 65(1-p)/p + 21 =~ 25 moves to twiddle are irritating