Reinforcement Learning

By,
Vishnu Sankar
Roll no 51
 Agenda
• What is Reinforcement Learning
• Why it is important
• Important terms used in RL
• How it works
• Types of Reinforcement Learning
• Reinforcement Learning Algorithms
• Learning Models of Reinforcement
• Reinforcement learning VS Deep learning
• Reinforcement learning VS Supervised Learning
• Characteristic
• Application
• Challenges
• Key takeaways
What is Reinforcement learning
• Reinforcement Learning is defined as a Machine
Learning method
• That is concerned with how software agents
should take actions in an environment.
• Reinforcement Learning is a part of the deep
learning method
• That helps you to maximize some portion of the
cumulative reward.
• The computer employs trial and error to come up
with a solution to the problem.
• To get the machine to do what the programmer
wants, the artificial intelligence gets either rewards
or penalties for the actions it performs
 Why It is important
• Reinforcement learning delivers decisions.

• By creating a simulation of an entire business or system

• It becomes possible for an intelligent system to test new actions or

approaches
• That change course when failures happen (or negative reinforcement)

• While building on successes (or positive reinforcement)

 Important terms used in RL
 Agent: It is an assumed entity which performs actions in an environment
to gain some reward.
 Environment (e): A scenario that an agent has to face.
 Reward (R): An immediate return given to an agent when he or she
performs specific action or task.
 State (s): State refers to the current situation returned by the
environment.
 Policy (π): It is a strategy which applies by the agent to decide the next
action based on the current state.
• Value (V): It is expected long-term return with discount, as
compared to the short-term reward.
 Value Function: It specifies the value of a state that is the total
amount of reward. It is an agent which should be expected
beginning from that state.
 Model of the environment: This mimics the behaviour of the
environment. It helps you to make inferences to be made and also
determine how the environment will behave.
 Model based methods: It is a method for solving
reinforcement learning problems which use model-based
methods.
 Q value or action value (Q): Q value is quite similar to value.
The only difference between the two is that it takes an
additional parameter as a current action.
 How Reinforcement learning works

• It is about taking suitable action to maximize reward.

• In reinforcement there is no answer key, but the agent decides
what to do to perform the task.
• Suppose we have an agent and a reward with many hurdles in
between.
• The agent is supposed to find the best possible path to reach the
reward.
• Lets take an example
• Your cat is an agent that is exposed to the environment (house).
• Our agent reacts by performing an action transition from one
"state" to another "state.“
• An example of a state could be your cat sitting, and you use a
specific word in for cat to walk.
• For example, your cat goes from sitting to walking.
• The reaction of an agent is an action, and the policy is a method of
selecting an action given a state in expectation of better
outcomes.
• After the transition, they may get a reward or penalty in return.
 Types of Reinforcement learning
Positive:
• Defined as an event that occurs because of specific behavior.
• It increases the strength and frequency.
• In positive reinforcement, a favorable stimulus is added
• Helps to maximize performance and sustain change for a longer
period.
• The stimuli act as a reward, for doing something.
• Strengthens or maintains the probability of recurrence of
response.
Negative:
• Defined as strengthening of behavior that occurs because of
negative condition.
• Helps to define minimum stand of performance.
• In negative reinforcement, an unfavorable stimulus is removed.
• In negative reinforcement, the stimuli act like a penalty, for not
doing something.
•  learns to get rid of nasty responses.
 Reinforcement Learning Algorithms

• There are three approaches to implement a Reinforcement Learning

algorithm.
Value-Based:
• In a value-based Reinforcement Learning method, you should try to
maximize a value function V(s).
• In this method, the agent is expecting a long-term return of the
current states under policy π.
Policy-based:
• In a policy-based RL method, you try to come up with such a policy that
the action performed in every state helps you to gain maximum reward
in the future.
Two types of policy-based methods are:
– Deterministic & Stochastic
Model-Based:
• In this method, you need to create a virtual model for each
environment. The agent learns to perform in that specific environment.
 Learning Models of Reinforcement

There are 2 types of Learning models for Reinforcement Learning

Markov Decision Process
The following parameters are used to get a solution:
– Set of actions- A
– Set of states -S
– Reward- R
– Policy- n
– Value- V
• In the problem, an agent is supposed to decide the best action to
select based on his current state.
• When this step is repeated, the problem is known as a Markov
Decision Process.
Q-Learning
• Q-learning is a values-based learning algorithm.
• Value based algorithms updates the value function based on an
equation.
• It uses Bellman equation to update the value function.
Reinforcement Learning VS Deep Learning
Reinforcement learning VS Supervised Learning
 Characteristics

• There is no supervisor, only a real number or reward signal.

• Sequential decision making.
• Time plays a crucial role in Reinforcement problems.
• Feedback is always delayed, not instantaneous.
• Agent's actions determine the subsequent data it receives.
 Applications

• Robotics for industrial automation.

• Business strategy planning.

• Machine learning and data processing.
• Aircraft control and robot motion control.
 Challenges of Reinforcement Learning

• Feature/reward design which should be very involved

• Parameters may affect the speed of learning.

• Realistic environments can have partial observability.

• Too much Reinforcement may lead to an overload of states

which can diminish the results.
• Realistic environments can be non-stationary.
 Key Takeaways
• Reinforcement Learning is a Machine Learning method.
• Helps you to discover which action yields the highest reward over the
longer period.
• Two types of reinforcement learning are 1) Positive 2) Negative.
• Application or reinforcement learning methods are: Robotics for
industrial automation and business strategy planning.
• You should not use this method when you have enough data to solve
the problem.
• The biggest challenge of this method is that parameters may affect the
speed of learning.
Thank You..