Intelligent Agents

Chapter-2
Outline
• Agents and environments
• Rationality
• PEAS (Performance measure, Environment, Actuators, Sensors)
• Environment types
• Agent types
Agents
• An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment
through actuators.
• An environment in artificial intelligence is the surrounding of the
agent. The agent takes input from the environment through
sensors and delivers the output to the environment through
actuators.
• Human agent: eyes, ears, and other organs for sensors; hands,
legs, mouth, and other body parts for actuators.
• Robotic agent: cameras and infrared range finders for sensors;
various motors for actuators.
Agents
• Software agent ( SoftBot : software robot ) :-
receives keystrokes, file contents, network packets etc as
sensory input.
A Network Packet is the fundamental unit of information
transmitted over a network or over a digital communication link
acts on the environment by displaying on the screen, writing
files, sending message packets.
 Agent perception and action
• Percept :- refers to an agents perceptual input at any given instant.
• Example of perceptual input:-information, objects, events, people
• Percept sequence :- refers to the complete history of what an agent has
perceived so far.
• Action:- an agent’s choice of action at any given instant can depend on the
entire percept sequence observed till that time.
• Agent function:- an agent’s behavior is described by the agent function that
maps any percept sequence to an action.
 Agent

Fig. 1 Agents interact with environments through sensors and actuators

Agents and environments

• The agent function maps from percept histories to actions: [f: P* A]

• The agent function is implemented by an agent program.
• The agent program runs on the physical architecture to produce f
• agent = architecture + program
 Vacuum-cleaner world

Fig. 2 A vacuum-cleaner world with just two locations

Percepts: location and contents , e.g.,[ A, Dirty]

The table shows the external characterstics of the the agent, whereas the internal behavior is carried out by the
agent program.
Actions: Left, Right, Suck, NoOp
Percept sequence : Action :
[A, Clean] right
[A, Dirty] suck
[B, Clean] left
[B, dirty] suck
[A, Dirty],[A, Clean] right
……………
Fig. 3 Partial tabulation of a simple agent function for the vacuum cleaner world
Rational agents

• An agent should strive to "do the right thing", based

on what it can perceive and the actions it can
perform. The right action is the one that will cause
the agent to be most successful.
• Performance measure: An objective criterion for
success of an agent's behavior.
• E.g., performance measure of a vacuum-cleaner agent
could be the amount of dirt cleaned up, the amount
of time taken, the amount of electricity consumed,
the amount of noise generated, etc .
RATIONALITY
Rationality at any given time depends on :
•Agent’s prior knowledge of the
environment.
•Actions that the agent can perform.
•Agent’s percept sequence till that moment.
•The performance measure that defines the
criterion of success.
Rational agents
•Rational Agent: For each possible percept sequence, a
rational agent should select an action that is expected to
maximize its performance measure, given the evidence
provided by the percept sequence and whatever built-in
knowledge the agent has.
•A self-driving car, for example, is rational if it can bring you
safely and comfortably where you need to go in the shortest
amount of time. It needs to follow road signs and directions
and avoid other vehicles, people, and other obstructions,
including traffic.
Requirements for a Rational agent

• Rationality ; which is distinct from omniscience

(all-knowing with infinite knowledge)
• Information gathering capability ; Agents can perform
actions in order to modify future percepts so as to
obtain useful information.
Learning ability.
• Ability to adapt.
• An agent is autonomous if its behavior is determined
by its own experience (with ability to learn and adapt)
 Building intelligent agents
• An agent is completely specified by the agent function which maps
percept sequences to action.
• An agent function is a map from the percept sequence(history of all
that an agent has perceived to date) to an action.
• An agent has some internal data structures that is updated as and
when new percepts arrive.
• The data structures are operated on by the agent’s decision making
procedures to generate an action choice, which is then passed to the
architecture to get executed.
 Building intelligent agents
•An agent consists of an architecture plus a program that
runs on that architecture.
•In designing intelligent systems , there are 4 main factors to
consider:-
•Percepts:- the inputs to our system.
•Actions:- the outputs of our system.
•Goals:- what the agent is expected to achieve.
•Environment:- what the agent is interacting with.
PEAS

• Task environments are problems to which rational agents

are the solutions. The task environment is specified by
PEAS.
• PEAS stands for :- Performance measure, Environment,
Actuators, Sensors.
• PEAS specifies the settings for an intelligent agent design.
PEAS

• Consider, e.g., the task of designing an automated taxi

driver:
• Performance measure: Safe, fast, legal, comfortable trip,
maximize profits.
• Environment: Roads, other traffic, pedestrians, customers.
• Actuators: Steering wheel, accelerator, brake, signal, horn.
• Sensors: Cameras, sonar, speedometer, GPS, odometer, engine
sensors, keyboard.
PEAS

• Agent: Medical diagnosis system

• Performance measure: Healthy patient, minimize costs,
lawsuits
• Environment: Patient, hospital, staff
• Actuators: Screen display (questions, tests, diagnoses,
treatments, referrals)
• Sensors: Keyboard (entry of symptoms, findings, patient's
answers)
PEAS

• Agent: Part-picking robot

• Performance measure: Percentage of parts in correct
bins
• Environment: Conveyor belt with parts, bins
• Actuators: Jointed arm and hand
• Sensors: Camera, joint angle sensors
PEAS

• Agent: Interactive English tutor

• Performance measure: Maximize student's score on
test
• Environment: Set of students
• Actuators: Screen display (exercises, suggestions,
corrections)
• Sensors: Keyboard
Environment types
Fully observable (vs. partially observable): An agent's sensors give
it access to the complete state of the environment at each point in
time.

Fully observable vs. Partially observable

If an agent’s sensors give it access to the complete state of the
environment at each point in time then the environment is
effectively fully observable
if the sensors detect all aspects
That are relevant to the choice of action
Environment Types
• Partially observable
An environment might be Partially observable because
of noisy and inaccurate sensors or because parts of
the state are simply missing from the sensor data.
Example:
● Only a local dirt sensor of the cleaner without a location detection sensor
cannot tell whether other squares are clean or not.
 Environment Types
• Deterministic vs. stochastic
● Next state of the environment Completely determined by the current state
and the actions executed by the agent, then the environment is
deterministic, otherwise, it is Stochastic.
● Strategic environment: deterministic except for actions of other agents
-Chess is:
deterministic as there would be only a few possible moves for a coin at the
current state and these moves can be determined.
-Cleaner and taxi driver are:
● Stochastic because of some unobservable aspects noise or unknown
 Environment Types
• In an Episodic task environment, each of the agent’s actions is divided into
atomic incidents or episodes. There is no dependency between current and
previous incidents. In each incident, an agent receives input from the environment
and then performs the corresponding action.
Example: Consider an example of Pick and Place robot, which is used to detect
defective parts from the conveyor belts. Here, every time robot(agent) will make
the decision on the current part i.e. there is no dependency between current and
previous decisions.
• In a Sequential environment, the previous decisions can affect all future
decisions. The next action of the agent depends on what action he has taken
previously and what action he is supposed to take in the future.
Example: Checkers- Where the previous move can affect all the following
moves.
Environment Types
• An agent is said to be in a competitive environment when it
competes against another agent to optimize the output.
• The game of chess is competitive as the agents compete with each
other to win the game which is the output.
• An agent is said to be in a collaborative environment when multiple
agents cooperate to produce the desired output.
• When multiple self-driving cars are found on the roads, they
cooperate with each other to avoid collisions and reach their
destination which is the output desired.
 Environment types
• Static (vs. dynamic): The environment is unchanged while an agent is
deliberating. (The environment is semi dynamic if the environment itself does
not change with the passage of time but the agent's performance score does)
A roller coaster ride is dynamic as it is set in motion and the environment keeps
changing every instant.
An empty house is static as there’s no change in the surroundings when an
agent enters.
• Discrete (vs. continuous): A limited number of distinct, clearly defined percepts
and actions.
The game of chess is discrete as it has only a finite number of moves. The
number of moves might vary with every game, but still, it’s finite.
Self-driving cars are an example of continuous environments as their actions are
driving, parking, etc. which cannot be numbered.
Environment types
• Single agent (vs. multiagent): An agent operating by itself in an
environment.
• A person left alone in a maze is an example of the single-agent system.
• The game of football is multi-agent as it involves 11 players in each
team.
• Known (vs. unknown): In a known environment, the output for all
probable actions is given.
• Obviously, in case of unknown environment, for an agent to make a
decision, it has to gain knowledge about how the environment works.
• A known environment can be partially observable, for example, in
solitare card games, i know the rules but am still unable to see the cards
that have not yet been turned over.
Environment Types
Environment types (Contd…)

• The environment type largely determines the agent design

• The real world is (of course) partially observable, stochastic, sequential, dynamic,
continuous, multi-agent
ENVIRONMENT CLASS
A number of environments are implemented, together with a general-purpose
environment simulator that places one or more agents in a simulated environment,
observes their behavior over time, and evaluates them according to a given
performance measure. Such experiments are often carried out not for a single
environment but for many environments drawn from an environment class.

For example, to evaluate a taxi driver in simulated traffic, we would want to run
many simulations with different traffic, lighting, and weather conditions. If we
designed the agent for a single scenario, we might be able to take advantage of
specific properties of the particular case but might not identify a good design for
driving in general.
ENVIRONMENT GENERATOR
An environment generator for each environment class that selects particular
environments (with certain likelihoods) in which to run the agent.

For example, the vacuum environment generator initializes the dirt pattern and
agent location randomly. We are then interested in the agent’s average
performance over the environment class. A rational agent for a given environment
class maximizes this average performance.
Agent functions and programs

• An agent is completely specified by the agent function

mapping percept sequences to actions.
• Agent program implements agent function using agent
architecture.
• Input to agent program is the current percept whereas
agent function takes the percept history as input.
Table Driven Agent

function TABLE-DRIVEN-AGENT(percept ) returns an action

persistent: percepts, a sequence, initially empty
table, a table of actions, indexed by percept sequences, initially fully
specified
append percept to the end of percepts
action ←LOOKUP(percepts, table)
return action
Examples of Table Driven Agent
•
Table Driven Agent: A Failure
•
Agent types

Four basic types of agents in order of increasing

sophistication:

• Simple reflex agents

• Model-based reflex agents
• Goal-based agents
• Utility-based agents
 Simple reflex agents
• It works by finding a rule whose condition matches the current
situation ( as defined by the percept ) and then doing the action
associated with that rule.
• A set of condition ~ action rules / situation ~ action rules /
productions / if~ then rules are defined.
e.g if the car – in - front is braking
then initiate – braking
• This agent function only succeeds when the environment is fully
observable.
• It only considers the current percept to make the correct decision.
A Simple Reflex Agent:
In the two-state vacuum environment

The agent program for a simple reflex agent in the two-state

vacuum environment:

function REFLEX-VACUUM-AGENT([location, status]) returns

an action
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left
A Simple Reflex Agent with Internal State:
In the two-state vacuum environment
function REFLEX-VACUUM-AGENT([location, status]) returns an action
static: remind, keeps track of clean status, initially zero
if status = Dirty then return Suck
else if (location = A and remind<1) then
{
remind=remind +1
return Right
}
else if (location = B and remind<1)then
{
remind=remind+1
return Left
}
else return NOP
DEMERITS OF Simple reflex agents
• Very limited intelligence.
• No knowledge of non-perceptual parts of the state.
• Usually too big to generate and store.
• If there occurs any change in the environment, then the collection
of rules need to be updated.
• For example, the braking rule given earlier assumes that the
condition car-in-fornt-is-braking can be determined from the current
percept- a single frame of video.
• This works if the car in front has a centrally mounted brake light.
• Unfortunately, the older models have different configurations of
taillights, brake lights, and it is not possible to tell form a single image
whether the car is braking.
• A simple refelx agent driving behind such a car would either brake
continously and unncessarily, orworse, never brake at all.
Simple reflex agents
 A Simple Reflex Agent in Nature
percepts
(size, motion)

RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP

needed for
completeness Action: SNAP or AVOID or
NOOP
Model-based reflex agents
• Model based agents can handle partially observable
environments.
• Its current state is stored inside the agent maintaining
some kind of structure which describes the part of
the world which can’t be seen. This behavior requires
information on how the world behaves and works.
This additional information completes the “world
view” model.
Model-based reflex agents
• A model – based reflex agent keeps track of the
current state of the world using an internal model. It
then chooses an action in the same way as the reflex
agent.
• The state changes whenever an action is performed
or something is perceived in the environment.
• For example,
The self-steering mobile vision, where it's necessary to
check the percept history to fully understand how the
world is evolving.
Model-based reflex agents
• For the world that is partially observable
● the agent has to keep track of an internal state
● That depends on the percept history
● Reflecting some of the unobserved aspects
● E.g., driving a car and changing lane
• Requiring two types of knowledge
● How the world evolves independently of the agent
● How the agent’s actions affect the world
• For example, that an overtaking car generally will be closer behind
than it was a moment ago.
• Second, we need some information about how the agent’s own
actions affect the world- for example, that when the agents turn the
steering wheel clockwise, then it turns to right.
• This knowlege about “how the world works”- whether implemented
in simple boolean circuits or in complete scientific theories- called
model of the world.
• An agent that uses such as model is called a model-based agent.
 Example Table Agent
With Internal State
IF THEN

Saw an object ahead, and turned Go straight

right, and it’s now clear ahead

Saw an object Ahead, turned right, Halt

and object ahead again

See no objects ahead Go straight

See an object ahead Turn randomly

 Example Reflex Agent With Internal State:
Wall-Following

start

Actions: left, right, straight, open-door

Rules:
1. If open(left) & open(right) and open(straight) then choose randomly between
right and left
2. If wall(left) and open(right) and open(straight) then straight
3. If wall(right) and open(left) and open(straight) then straight
4. If wall(right) and open(left) and wall(straight) then left
5. If wall(left) and open(right) and wall(straight) then right
6. If wall(left) and door(right) and wall(straight) then open-door
7. If wall(right) and wall(left) and open(straight) then straight.
8. (Default) Move randomly
Model-based reflex agents
Model-based Reflex Agents

The agent is with memory

• In model-based agent, the current percept is combined with old
internal state to generate the updated description of the current
state.
• The UPDATE_STATE function is responsible for creating the new
internal state description.
Goal based agents
• Knowing about the current state of the environment is not always
enough to decide what to do; additionally sort of goal information which
describes situations that are desirable is also required. This allows the
agent a way to choose among multiple possibilities , selecting the one
which reaches a goal state.
• It keeps track of the world state and a set of goals it is trying to achieve
and picks an action that will eventually lead to the achievement of its
goal.
• For example, at a road junction, the taxi can turn left, right, or go
straight on.
• The correct decision depends on where the taxi is trying to get to.
Goal-based agents
• Current state of the environment is always not
enough
• The goal is another issue to achieve
● Judgment of rationality / correctness
• Actions chosen goals, based on
● the current state
● the current percept
 Goal-based agents
• Conclusion
● Goal-based agents are less efficient
● but more flexible
● Agent Different goals different tasks
● Search and planning are two other sub-fields in AI
● to find out the action sequences to achieve its goal
Goal-based agents
 Utility based agents
• Goal based agents only distinguish between goal states and non-goal
states.
• It is possible to define a measure of how desirable a particular state
is. This measure can be obtained through the use of a utility function
which maps a state to a measure of the utility of the state. So, utility
function maps a state onto a real number, which describes the
associated degree of happiness.
• A complete specification of the utility function allows rational
decisions.
Utility-based agents

• Goals alone are not enough

● to generate high-quality behavior
● E.g. meals in Canteen, good or not ?
• Many action sequences the goals
● some are better and some worse
● If goal means success,
● then utility means the degree of success (how
successful it is)
Utility-based agents
• it is said state A has higher utility
● If state A is more preferred than others
• Utility is therefore a function
● that maps a state onto a real number
● the degree of success
Utility-based agents

• Utility has several advantages:

● When there are conflicting goals,
● Only some of the goals but not all can be achieved
● utility describes the appropriate trade-off
● When there are several goals
● None of them are achieved certainly
● utility provides a way for the decision-making
A model based-utility based agent
• It uses a model of the world along with a utility function that
measures its preferences among states of the world. Then it chooses
an action that leads to the best expected utility.
Utility-based agents
Learning Agents
• After an agent is programmed, can it work
immediately?
● No, it still need teaching
• In AI,
● Once an agent is done
● We teach it by giving it a set of examples
● Test it by using another set of examples
• We then say the agent learns
● A learning agent
Learning Agents
• Four conceptual components
● Learning element
● Making improvement
● Performance element
● Selecting external actions
● Critic
● Tells the Learning element how well the agent is doing with respect
to fixed performance standard.
(Feedback from user or examples, good or not?)
● Problem generator
● Suggest actions that will lead to new and informative experiences.
Learning agents

• Learning has an advantage that it allows the agents to

initially operate in unknown environments and to
become more competent than its initial knowledge
alone might allow.
Learning agents
Components:-
• Learning element:- responsible for making improvements.
• Performance elements:- responsible for selecting actions.
• Critics:- provides feedback and it is ncessary because the
percepts themselves provide no indication of the agent’s
success.
• Problem generator:- responsible for suggesting actions that
will lead to new experiences.
• For example, let us say a taxi makes a quick left turn across
three lanes of traffic, the critic observes the shocking
language used by the other drivers. From this experience
the learning element is able to formulate a new rule saying
this was a bad action, anf the performance element is
modified by installation of the new rule.
Learning agents
Representation of States
• In atomic representations each state of the world is invisible-it has no
internal structure.
• For example, the problem of finding a driving route from one end of a
country to the other via some sequence of cities.
• For the purpose of solving this problem, it is adequate to reduce the
state of the world to just the name of the city we are in-a single atom
of knowledge.
• Some exmamples are: algorithms used in search and game-playing,
Hidden markov models, and markov decision processes.
• A factored representations splits up each state into a fixed set of
variables or attributes, each of which can have a value.
• The uncertainty can be represented in factored representations.
• For example, the problem of finding a driving route from one end of a
country to the other via some sequence of cities, we might need to
pay attention to how much gas is in the tank, our current GPS
coordinates, whether or not the oil waring light is working, how much
spare change we have for toll crossings, and so on.
• Some important areas of AI are based on factored representations,
such as constraint satisfaction algorithms, proportional logic,
planning, and the machine learning algorithm.
• In a structured representations, the objects and their various and varying
relationship can be described explicitly.
• For example, we might notice that a large truck ahead of us is reversing
into the driveway of dairy farm but a cow has got loose and is blocking
the truck’s patch.
• A factored representations is unlikely to be pre-equipped with the
attribute TruckAheadBackingIntoDairyFarmDrivewayBlockedByLooseCow
with value true or false.
• Instead, a structured representations, in which objects such as cows and
trucks and their varying relationships can be described directly.
• Examples are: relational databases and first-order logic, first-order
probability models, and knowledge-based learning, and so on.