Planning and expert system

What is the Role of Planning in Artificial

Intelligence?
Artificial intelligence is an important technology in the future. Whether it is intelligent robots,
self-driving cars, or smart cities, they will all use different aspects of artificial intelligence!!! But
Planning is very important to make any such AI project.

Even Planning is an important part of Artificial Intelligence which deals with the tasks and
domains of a particular problem. Planning is considered the logical side of acting.

Everything we humans do is with a definite goal in mind, and all our actions are oriented towards
achieving our goal. Similarly, Planning is also done for Artificial Intelligence.

For example, Planning is required to reach a particular destination. It is necessary to find the
best route in Planning, but the tasks to be done at a particular time and why they are done are
also very important.

ADVERTISEMENT

That is why Planning is considered the logical side of acting. In other words, Planning is about
deciding the tasks to be performed by the artificial intelligence system and the system's
functioning under domain-independent conditions.

What is a Plan?
We require domain description, task specification, and goal description for any planning system.
A plan is considered a sequence of actions, and each action has its preconditions that must be
satisfied before it can act and some effects that can be positive or negative.

So, we have Forward State Space Planning (FSSP) and Backward State Space
Planning (BSSP) at the basic level.
1. Forward State Space Planning (FSSP)
FSSP behaves in the same way as forwarding state-space search. It says that given an initial state
S in any domain, we perform some necessary actions and obtain a new state S' (which also
contains some new terms), called a progression. It continues until we reach the target position.
Action should be taken in this matter.

o Disadvantage: Large branching factor

o Advantage: The algorithm is Sound

2. Backward State Space Planning (BSSP)

BSSP behaves similarly to backward state-space search. In this, we move from the target
state g to the sub-goal g, tracing the previous action to achieve that goal. This process is
called regression (going back to the previous goal or sub-goal). These sub-goals should
also be checked for consistency. The action should be relevant in this case.

o Disadvantages: not sound algorithm (sometimes inconsistency can be found)

o Advantage: Small branching factor (much smaller than FSSP)

So for an efficient planning system, we need to combine the features of FSSP and BSSP,
which gives rise to target stack planning which will be discussed in the next article.

What is planning in AI?

Planning in artificial intelligence is about decision-making actions performed by robots or
computer programs to achieve a specific goal.

Execution of the plan is about choosing a sequence of tasks with a high probability of
accomplishing a specific task.

Block-world planning problem

o The block-world problem is known as the Sussmann anomaly.

o The non-interlaced planners of the early 1970s were unable to solve this problem. Therefore it is
considered odd.

o When two sub-goals, G1 and G2, are given, a non-interleaved planner either produces a plan for
G1 that is combined with a plan for G2 or vice versa.
 o In the block-world problem, three blocks labeled 'A', 'B', and 'C' are allowed to rest on a flat surface.
The given condition is that only one block can be moved at a time to achieve the target.

The start position and target position are shown in the following diagram.

Components of the planning system

The plan includes the following important steps:

o Choose the best rule to apply the next rule based on the best available guess.
o Apply the chosen rule to calculate the new problem condition.
o Find out when a solution has been found.
o Detect dead ends so they can be discarded and direct system effort in more useful
directions.
o Find out when a near-perfect solution is found.

Target stack plan

o It is one of the most important planning algorithms used by STRIPS.
o Stacks are used in algorithms to capture the action and complete the target. A
knowledge base is used to hold the current situation and actions.
 o A target stack is similar to a node in a search tree, where branches are created with
a choice of action.

The important steps of the algorithm are mentioned below:

1. Start by pushing the original target onto the stack. Repeat this until the pile is
empty. If the stack top is a mixed target, push its unsatisfied sub-targets onto the
stack.
2. If the stack top is a single unsatisfied target, replace it with action and push the
action precondition to the stack to satisfy the condition.

iii. If the stack top is an action, pop it off the stack, execute it and replace the knowledge
base with the action's effect.

If the stack top is a satisfactory target, pop it off the stack.

Non-linear Planning
This Planning is used to set a goal stack and is included in the search space of all possible
sub-goal orderings. It handles the goal interactions by the interleaving method.

Advantages of non-Linear Planning

Non-linear Planning may be an optimal solution concerning planning length (depending

on the search strategy used).

Disadvantages of Nonlinear Planning

It takes a larger search space since all possible goal orderings are considered.

Complex algorithm to understand.

Algorithm

1. Choose a goal 'g' from the goal set

2. If 'g' does not match the state, then
o Choose an operator 'o' whose add-list matches goal g
o Push 'o' on the OpStack
 o Add the preconditions of 'o' to the goal set
3. While all preconditions of the operator on top of OpenStack are met in a state
o Pop operator o from top of opstack
o state = apply(o, state)
o plan = [plan; o]

o Different types of planning in AI

AI planning comes in different types, each suitable for a particular
situation. Popular different types of planning in ai include:
• Classical Planning: In this style of planning, a series of actions is created to
accomplish a goal in a predetermined setting. It assumes that everything is
static and predictable.
• Hierarchical planning: By dividing large problems into smaller ones,
hierarchical planning makes planning more effective. A hierarchy of plans
must be established, with higher-level plans supervising the execution of
lower-level plans.
• Temporal Planning: Planning for the future considers time restrictions and
interdependencies between actions. It ensures that the plan is workable
within a certain time limit by taking into account the duration of tasks.

Components of planning system in AI

A planning system in AI is made up of many crucial parts that cooperate to
produce successful plans. These components of planning system in ai consist of:
• Representation: The component that describes how the planning problem
is represented is called representation. The state space, actions, objectives,
and limitations must all be defined.
• Search: To locate a series of steps that will get you where you want to go,
the search component searches the state space. To locate the best plans, a
variety of search techniques, including depth-first search & A* search, can
be used.
• Heuristics: Heuristics are used to direct search efforts and gauge the
expense or benefit of certain actions. They aid in locating prospective routes
and enhancing the effectiveness of the planning process.

Benefits of AI Planning
Numerous advantages of AI planning contribute to the efficacy and efficiency of
artificial intelligence systems. Some key benefits include:
• Resource Allocation: With the help of AI planning, resources can be
distributed in the best way possible, ensuring that they are used effectively
to accomplish the desired objectives.
• Better Decision-Making: AI planning aids in making knowledgeable
judgments by taking a variety of aspects and restrictions into account. It
helps AI systems to weigh several possibilities and decide on the best
course of action.
• Automation of Complex Tasks: AI planning automates complicated tasks
that would otherwise need a lot of human work. It makes it possible for AI
systems to manage complex procedures and optimize them for better
results.

Applications of AI Planning
AI planning is used in many different fields, demonstrating its adaptability and
efficiency. A few significant applications are:
• Robotics: To enable autonomous robots to properly navigate their
surroundings, carry out activities, and achieve goals, planning is crucial.
• Gaming: AI planning is essential to the gaming industry because it enables
game characters to make thoughtful choices and design difficult and
interesting gameplay scenarios.
• Logistics: To optimize routes, timetables, and resource allocation and
achieve effective supply chain management, AI planning is widely utilized in
logistics.
• Healthcare: AI planning is used in the industry to better the quality and
effectiveness of healthcare services by scheduling patients, allocating
resources, and planning treatments.

Challenges in AI Planning
While AI planning has many advantages, many issues need to be resolved. Typical
challenges include:
• Complexity: Due to the wide state space, multiple possible actions, and
interdependencies between them, planning in complicated domains can be
difficult.
• Uncertainty: One of the biggest challenges in AI planning is overcoming
uncertainty. Actions' results might not always be anticipated, thus the
planning system needs to be able to deal with such ambiguous situations.
 • Scalability: Scalability becomes a significant barrier as the complexity and
scale of planning problems rise. Large-scale issues must be effectively
handled via planning systems.

Strategies for Mastering AI Planning

Adopting strategies that improve planning abilities is crucial if you want to master
AI planning. Here are some strategies to think about:
• Domain Knowledge: Learn everything there is to know about the planning
domain. Better strategies can be made if you are aware of the complexities
and limitations of the domain.
• Algorithm Selection: It is essential to choose the right planning algorithm
for the particular issue at hand. Choosing t
• Choosing the best algorithm can have a big impact on the planning process
because different algorithms have different strengths and disadvantages.
• Improvement through iteration: Planning is an iterative process, and
improvement is essential. Analyze the effectiveness of plans, pinpoint areas
that need improvement, and adjust the planning system as necessary.

Tools and Techniques for AI Planning

Various tools and strategies that support planning can be used to facilitate AI
planning. Techniques and tools that are commonly used include:
• Automated planners: Programmes that generate plans automatically
include STRIPS and PDDL, which offer a framework for specifying planning
problems.
• Constraint Programming: Using the strong technique of constraint
programming, complicated planning issues with a variety of constraints can
be modeled and solved.
• Machine Learning: Reinforcement learning is a machine learning technique
that can be used to enhance planning by learning from previous experiences
and refining plans in response to feedback.

Best Practices of AI Planning

Several important aspects are incorporated into best practices for AI planning:
• Formulation of the issue clearly: Clearly state your goals, restrictions, and
desired results.
• Optimal representation: Construct an appropriate representation of the
planning domain.
• Algorithm selection: Choose planning algorithms that strike a compromise
between complexity and optimality.
 • Iterative improvement: Keep the planning process under constant review
and improvement.
• Management of uncertainty: Include methods, such as probabilistic
modeling, to deal with uncertainty.
• Utilizing human expertise: Ask for and use feedback from others to make
sure your aims are in line with theirs.
• Benchmarking and evaluation: Continually assess performance and
evaluate against pertinent indicators.
• Collaboration: Encourage cooperative planning by incorporating pertinent
parties.
• Scalability: Design planning systems with scalability in mind to effectively
tackle big problems.
• Real-time responsiveness: Create systems that can instantly adjust and
replan in response to shifting circumstances.
• Ethical considerations: Address AI ethical issues, promote justice, and
ensure accountability in the planning procedures..
• Documentation: Keep thorough records of the planning process for future
reference.

A simple planning agent create and use plans based on the following assumptions:
• Atomic time: each action is indivisible
• No concurrent actions allowed
• Deterministic actions: result of each actions is completely determined by the
definition of the action, and there is no uncertainty in performing it in the
world.
• Agent is the sole cause of change in the world.
• Agent is omniscient: has complete knowledge of the state of the world

Means-Ends Analysis in Artificial Intelligence

o We have studied the strategies which can reason either in forward or backward,
but a mixture of the two directions is appropriate for solving a complex and large
problem. Such a mixed strategy, make it possible that first to solve the major part
of a problem and then go back and solve the small problems arise during
combining the big parts of the problem. Such a technique is called Means-Ends
Analysis.
o Means-Ends Analysis is problem-solving techniques used in Artificial intelligence
for limiting search in AI programs.
o It is a mixture of Backward and forward search technique.
 o The MEA technique was first introduced in 1961 by Allen Newell, and Herbert A.
Simon in their problem-solving computer program, which was named as General
Problem Solver (GPS).
o The MEA analysis process centered on the evaluation of the difference between
the current state and goal state.

How means-ends analysis Works:

The means-ends analysis process can be applied recursively for a problem. It is a strategy to
control search in problem-solving. Following are the main Steps which describes the working of
MEA technique for solving a problem.

a. First, evaluate the difference between Initial State and final State.
b. Select the various operators which can be applied for each difference.
c. Apply the operator at each difference, which reduces the difference between the current
state and goal state.

Operator Subgoaling
In the MEA process, we detect the differences between the current state and goal state. Once
these differences occur, then we can apply an operator to reduce the differences. But sometimes
it is possible that an operator cannot be applied to the current state. So we create the subproblem
of the current state, in which operator can be applied, such type of backward chaining in which
operators are selected, and then sub goals are set up to establish the preconditions of the operator
is called Operator Subgoaling.

Algorithm for Means-Ends Analysis:

Let's we take Current state as CURRENT and Goal State as GOAL, then following are the steps
for the MEA algorithm.

o Step 1: Compare CURRENT to GOAL, if there are no differences between both then
return Success and Exit.
o Step 2: Else, select the most significant difference and reduce it by doing the
following steps until the success or failure occurs.
a. Select a new operator O which is applicable for the current difference, and if there
is no such operator, then signal failure.
b. Attempt to apply operator O to CURRENT. Make a description of two states.
i) O-Start, a state in which O?s preconditions are satisfied.
ii) O-Result, the state that would result if O were applied In O-start.
c. If
(First-Part <------ MEA (CURRENT, O-START)
And
(LAST-Part <----- MEA (O-Result, GOAL), are successful, then signal Success and return
the result of combining FIRST-PART, O, and LAST-PART.

The above-discussed algorithm is more suitable for a simple problem and not adequate for
solving complex problems.

Example of Mean-Ends Analysis:

Let's take an example where we know the initial state and goal state as given below. In this
problem, we need to get the goal state by finding differences between the initial state and goal
state and applying operators.

Solution:
To solve the above problem, we will first find the differences between initial states and goal
states, and for each difference, we will generate a new state and will apply the operators. The
operators we have for this problem are:

o Move

o Delete

o Expand

1. Evaluating the initial state: In the first step, we will evaluate the initial state and will
compare the initial and Goal state to find the differences between both states.
2. Applying Delete operator: As we can check the first difference is that in goal state there is
no dot symbol which is present in the initial state, so, first we will apply the Delete operator to
remove this dot.

3. Applying Move Operator: After applying the Delete operator, the new state occurs which
we will again compare with goal state. After comparing these states, there is another difference
that is the square is outside the circle, so, we will apply the Move Operator.

4. Applying Expand Operator: Now a new state is generated in the third step, and we will
compare this state with the goal state. After comparing the states there is still one difference
which is the size of the square, so, we will apply Expand operator, and finally, it will generate
the goal state.
How does the Means-Ends Analysis work?
Let’s learn how it works.

• Measures the current state and identifies the problems and pain

points faced in the current state.

• Defines the to-be state (goal state) to be reached.

• Splits the goals into sub-goals and sub-goals into further sub-

sub-goals. To illustrate, we can divide long-term goals into short-

term goals and continue the subdivision process.

• The desired outcome requires identifying the necessary steps to

achieve it.

• Connect each sub-goal with executable actions.

• Includes all intermediate steps and relevant actions to address the

issues faced in the current state.

 • It makes these steps detectable and device ways and means to

track even small changes in the actual and to-be state.

Algorithm for Means-Ends Analysis

The algorithm provides the best possible solution to a problem and

contains the well-defined step-by-step correct resolution to a given

problem. One should blindly follow this mathematical template of steps

and expect it to produce the end results. We can use these algorithms as

input to develop computer programs and implement solutions. Each and

every step can have test cases built for it, and the program can undergo

thorough testing before implementation. When solving a problem,

logical steps guarantee a solution. In contrast, heuristics rely on trial and

error, past experience, and judgmental methods, which may not always

produce a guaranteed solution.

In general, we use algorithms where we expect accurate results and

where the time to complete the activity is not a significant criterion.

When completing tasks, people use heuristics to save time and can
tolerate a few errors. In planned activity (e.g., Organization Planning), we

deploy algorithms; in emergency activity (e.g., a Physician treating a

patient should decide on the treatment quickly), we use heuristics.

The algorithm for MEA consists of the following steps:

• Step 1: Measure the current state of things by doing as is the

study and capture the status at a macro level and a possible micro

level.

• Step 2: Capture the deficiency in the current state and avenues for

improvements (wish list) and define the goal state (to-be state).

Define the to-be state at a macro level and a possible micro level.

• Step 3: Compare the Current and Goal states; if they are at the

same level, the problem is resolved.

• Step 4: List the differences between the current and goal states at

macro and micro levels.

• Step 5: Convert the differences into deletions/modifications to the

current state and new additions.

 • Step 6: Define the action to implement the changes as defined in

step-5.

• Step 7: Implement the changes and measure the results with the

planned goals.

• Step 8: Do course correction and achieve the final goal.

Artificial Intelligence applications, General Management scenarios, and

phycology use cases to deploy the MEA algorithm step selectively and

recursively in achieving the end goal.

Types of Machine Learning

Machine learning is a subset of AI, which enables the machine to automatically
learn from data, improve performance from past experiences, and make
predictions. Machine learning contains a set of algorithms that work on a huge amount of data.
Data is fed to these algorithms to train them, and on the basis of training, they build the model &
perform a specific task.
These ML algorithms help to solve different business problems like Regression, Classification,
Forecasting, Clustering, and Associations, etc.

Based on the methods and way of learning, machine learning is divided into mainly four types,
which are:

1. Supervised Machine Learning

2. Unsupervised Machine Learning

3. Semi-Supervised Machine Learning

4. Reinforcement Learning
In this topic, we will provide a detailed description of the types of Machine Learning along with
their respective algorithms:

1. Supervised Machine Learning

As its name suggests, Supervised machine learning is based on supervision. It means in the
supervised learning technique, we train the machines using the "labelled" dataset, and based on
the training, the machine predicts the output. Here, the labelled data specifies that some of the
inputs are already mapped to the output. More preciously, we can say; first, we train the machine
with the input and corresponding output, and then we ask the machine to predict the output using
the test dataset.

Let's understand supervised learning with an example. Suppose we have an input dataset of cats
and dog images. So, first, we will provide the training to the machine to understand the images,
such as the shape & size of the tail of cat and dog, Shape of eyes, colour, height
(dogs are taller, cats are smaller), etc. After completion of training, we input the picture of
a cat and ask the machine to identify the object and predict the output. Now, the machine is well
trained, so it will check all the features of the object, such as height, shape, colour, eyes, ears,
tail, etc., and find that it's a cat. So, it will put it in the Cat category. This is the process of how
the machine identifies the objects in Supervised Learning.

The main goal of the supervised learning technique is to map the input variable(x)
with the output variable(y). Some real-world applications of supervised learning are Risk
Assessment, Fraud Detection, Spam filtering, etc.

Categories of Supervised Machine Learning

Supervised machine learning can be classified into two types of problems, which are given
below:

o Classification

o Regression

a) Classification

Classification algorithms are used to solve the classification problems in which the output
variable is categorical, such as "Yes" or No, Male or Female, Red or Blue, etc. The
classification algorithms predict the categories present in the dataset. Some real-world examples
of classification algorithms are Spam Detection, Email filtering, etc.

Some popular classification algorithms are given below:

o Random Forest Algorithm

 o Decision Tree Algorithm

o Logistic Regression Algorithm

o Support Vector Machine Algorithm

b) Regression

Regression algorithms are used to solve regression problems in which there is a linear
relationship between input and output variables. These are used to predict continuous output
variables, such as market trends, weather prediction, etc.

Some popular Regression algorithms are given below:

o Simple Linear Regression Algorithm

o Multivariate Regression Algorithm

o Decision Tree Algorithm

o Lasso Regression

Advantages and Disadvantages of Supervised Learning

Advantages:

o Since supervised learning work with the labelled dataset so we can have an exact idea about the
classes of objects.

o These algorithms are helpful in predicting the output on the basis of prior experience.

Disadvantages:

o These algorithms are not able to solve complex tasks.

o It may predict the wrong output if the test data is different from the training data.

o It requires lots of computational time to train the algorithm.

Applications of Supervised Learning

Some common applications of Supervised Learning are given below:

o Image Segmentation:
Supervised Learning algorithms are used in image segmentation. In this process, image
classification is performed on different image data with pre-defined labels.
 o Medical Diagnosis:
Supervised algorithms are also used in the medical field for diagnosis purposes. It is done by using
medical images and past labelled data with labels for disease conditions. With such a process, the
machine can identify a disease for the new patients.

o Fraud Detection - Supervised Learning classification algorithms are used for identifying fraud
transactions, fraud customers, etc. It is done by using historic data to identify the patterns that can
lead to possible fraud.

o Spam detection - In spam detection & filtering, classification algorithms are used. These
algorithms classify an email as spam or not spam. The spam emails are sent to the spam folder.

o Speech Recognition - Supervised learning algorithms are also used in speech recognition. The
algorithm is trained with voice data, and various identifications can be done using the same, such
as voice-activated passwords, voice commands, etc.

2. Unsupervised Machine Learning

Unsupervised learning is different from the Supervised learning technique; as its name suggests,
there is no need for supervision. It means, in unsupervised machine learning, the machine is
trained using the unlabeled dataset, and the machine predicts the output without any supervision.

In unsupervised learning, the models are trained with the data that is neither classified nor
labelled, and the model acts on that data without any supervision.

The main aim of the unsupervised learning algorithm is to group or categories the
unsorted dataset according to the similarities, patterns, and differences. Machines
are instructed to find the hidden patterns from the input dataset.

Let's take an example to understand it more preciously; suppose there is a basket of fruit images,
and we input it into the machine learning model. The images are totally unknown to the model,
and the task of the machine is to find the patterns and categories of the objects.

So, now the machine will discover its patterns and differences, such as colour difference,
shape difference, and predict the output when it is tested with the test dataset.

Categories of Unsupervised Machine Learning

Unsupervised Learning can be further classified into two types, which are given below:

o Clustering
 o Association

1) Clustering

The clustering technique is used when we want to find the inherent groups from the data.
It is a way to group the objects into a cluster such that the objects with the most
similarities remain in one group and have fewer or no similarities with the objects of other
groups. An example of the clustering algorithm is grouping the customers by their
purchasing behaviour.

Some of the popular clustering algorithms are given below:

o K-Means Clustering algorithm

o Mean-shift algorithm

o DBSCAN Algorithm

o Principal Component Analysis

o Independent Component Analysis

2) Association

Association rule learning is an unsupervised learning technique, which finds interesting relations
among variables within a large dataset. The main aim of this learning algorithm is to find the
dependency of one data item on another data item and map those variables accordingly so that it
can generate maximum profit. This algorithm is mainly applied in Market Basket analysis,
Web usage mining, continuous production, etc.

Some popular algorithms of Association rule learning are Apriori Algorithm, Eclat, FP-
growth algorithm.

Advantages and Disadvantages of Unsupervised Learning

Algorithm
Advantages:

o These algorithms can be used for complicated tasks compared to the supervised ones because
these algorithms work on the unlabeled dataset.

o Unsupervised algorithms are preferable for various tasks as getting the unlabeled dataset is easier
as compared to the labelled dataset.
Disadvantages:

o The output of an unsupervised algorithm can be less accurate as the dataset is not labelled, and
algorithms are not trained with the exact output in prior.

o Working with Unsupervised learning is more difficult as it works with the unlabelled dataset that
does not map with the output.

Applications of Unsupervised Learning

o Network Analysis: Unsupervised learning is used for identifying plagiarism and copyright in
document network analysis of text data for scholarly articles.

o Recommendation Systems: Recommendation systems widely use unsupervised learning

techniques for building recommendation applications for different web applications and e-
commerce websites.

o Anomaly Detection: Anomaly detection is a popular application of unsupervised learning, which

can identify unusual data points within the dataset. It is used to discover fraudulent transactions.

o Singular Value Decomposition: Singular Value Decomposition or SVD is used to extract particular
information from the database. For example, extracting information of each user located at a
particular location.

3. Semi-Supervised Learning
Semi-Supervised learning is a type of Machine Learning algorithm that lies
between Supervised and Unsupervised machine learning. It represents the intermediate
ground between Supervised (With Labelled training data) and Unsupervised learning (with no
labelled training data) algorithms and uses the combination of labelled and unlabeled datasets
during the training period.

Although Semi-supervised learning is the middle ground between supervised and unsupervised
learning and operates on the data that consists of a few labels, it mostly consists of unlabeled
data. As labels are costly, but for corporate purposes, they may have few labels. It is completely
different from supervised and unsupervised learning as they are based on the presence & absence
of labels.

To overcome the drawbacks of supervised learning and unsupervised learning

algorithms, the concept of Semi-supervised learning is introduced. The main aim
of semi-supervised learning is to effectively use all the available data, rather than only labelled
data like in supervised learning. Initially, similar data is clustered along with an unsupervised
learning algorithm, and further, it helps to label the unlabeled data into labelled data. It is
because labelled data is a comparatively more expensive acquisition than unlabeled data.

We can imagine these algorithms with an example. Supervised learning is where a student is
under the supervision of an instructor at home and college. Further, if that student is self-
analysing the same concept without any help from the instructor, it comes under unsupervised
learning. Under semi-supervised learning, the student has to revise himself after analyzing the
same concept under the guidance of an instructor at college.

Advantages and disadvantages of Semi-supervised Learning

Advantages:

o It is simple and easy to understand the algorithm.

o It is highly efficient.

o It is used to solve drawbacks of Supervised and Unsupervised Learning algorithms.

Disadvantages:

o Iterations results may not be stable.

o We cannot apply these algorithms to network-level data.

o Accuracy is low.

4. Reinforcement Learning
Reinforcement learning works on a feedback-based process, in which an AI agent
(A software component) automatically explore its surrounding by hitting & trail,
taking action, learning from experiences, and improving its performance. Agent gets
rewarded for each good action and get punished for each bad action; hence the goal of
reinforcement learning agent is to maximize the rewards.

In reinforcement learning, there is no labelled data like supervised learning, and agents learn
from their experiences only.

ADVERTISEMENT

The reinforcement learning process is similar to a human being; for example, a child learns
various things by experiences in his day-to-day life. An example of reinforcement learning is to
play a game, where the Game is the environment, moves of an agent at each step define states,
and the goal of the agent is to get a high score. Agent receives feedback in terms of punishment
and rewards.
Due to its way of working, reinforcement learning is employed in different fields such as Game
theory, Operation Research, Information theory, multi-agent systems.

A reinforcement learning problem can be formalized using Markov Decision

Process(MDP). In MDP, the agent constantly interacts with the environment and performs
actions; at each action, the environment responds and generates a new state.

Categories of Reinforcement Learning

Reinforcement learning is categorized mainly into two types of methods/algorithms:

o Positive Reinforcement Learning: Positive reinforcement learning specifies increasing the

tendency that the required behaviour would occur again by adding something. It enhances the
strength of the behaviour of the agent and positively impacts it.

o Negative Reinforcement Learning: Negative reinforcement learning works exactly opposite to

the positive RL. It increases the tendency that the specific behaviour would occur again by avoiding
the negative condition.

Real-world Use cases of Reinforcement Learning

o Video Games:
RL algorithms are much popular in gaming applications. It is used to gain super-human
performance. Some popular games that use RL algorithms are AlphaGO and AlphaGO Zero.

o Resource Management:
The "Resource Management with Deep Reinforcement Learning" paper showed that how to use RL
in computer to automatically learn and schedule resources to wait for different jobs in order to
minimize average job slowdown.

o Robotics:
RL is widely being used in Robotics applications. Robots are used in the industrial and
manufacturing area, and these robots are made more powerful with reinforcement learning. There
are different industries that have their vision of building intelligent robots using AI and Machine
learning technology.

o Text Mining
Text-mining, one of the great applications of NLP, is now being implemented with the help of
Reinforcement Learning by Salesforce company.
Advantages and Disadvantages of Reinforcement Learning
Advantages

o It helps in solving complex real-world problems which are difficult to be solved by general
techniques.

o The learning model of RL is similar to the learning of human beings; hence most accurate results
can be found.

o Helps in achieving long term results.

Disadvantage

o RL algorithms are not preferred for simple problems.

o RL algorithms require huge data and computations.
o Too much reinforcement learning can lead to an overload of states which can
weaken the results.

The curse of dimensionality limits reinforcement learning for real physical systems.

S.No. ARTIFICIAL INTELLIGENCE MACHINE LEARNING

The terminology “Artificial The terminology “Machine

Intelligence” was originally used Learning” was first used in 1952 by
1. by John McCarthy in 1956, who IBM computer scientist Arthur
also hosted the first AI Samuel, a pioneer in artificial
conference. intelligence and computer games.

AI stands for Artificial

ML stands for Machine Learning
intelligence, where intelligence
2. which is defined as the acquisition
is defined as the ability to
of knowledge or skill
acquire and apply knowledge.
 AI is the broader family
Machine Learning is the subset of
3. consisting of ML and DL as its
Artificial Intelligence.
components.

The aim is to increase the

The aim is to increase accuracy, but
4. chance of success and not
it does not care about; the success
accuracy.

AI is aiming to develop an Machine learning is attempting to

intelligent system capable of construct machines that can only
5.
performing a variety of complex accomplish the jobs for which they
jobs. decision-making have been trained.

It works as a computer program Here, the tasks systems machine

6.
that does smart work. takes data and learns from data.

The goal is to simulate natural The goal is to learn from data on

7. intelligence to solve complex certain tasks to maximize
problems. the performance on that task.

AI has a very broad variety of The scope of machine learning is

8.
applications. constrained.

ML allows systems to learn new

9. AI is decision-making.
things from data.

It is developing a system that

It involves creating self-learning
10. mimics humans to solve
algorithms.
problems.

AI is a broader family consisting

11. ML is a subset of AI.
of ML and DL as its components.
What is an Expert System?
An expert system is a computer program that is designed to solve complex problems and
to provide decision-making ability like a human expert. It performs this by extracting
knowledge from its knowledge base using the reasoning and inference rules according to
the user queries.

The expert system is a part of AI, and the first ES was developed in the year 1970, which
was the first successful approach of artificial intelligence. It solves the most complex issue
as an expert by extracting the knowledge stored in its knowledge base. The system helps
in decision making for compsex problems using both facts and heuristics like a human
expert. It is called so because it contains the expert knowledge of a specific domain and
can solve any complex problem of that particular domain. These systems are designed for
a specific domain, such as medicine, science, etc.

The performance of an expert system is based on the expert's knowledge stored in its
knowledge base. The more knowledge stored in the KB, the more that system improves
its performance. One of the common examples of an ES is a suggestion of spelling errors
while typing in the Google search box.

Below is the block diagram that represents the working of an expert system:
Note: It is important to remember that an expert system is not used to replace the
human experts; instead, it is used to assist the human in making a complex decision.
These systems do not have human capabilities of thinking and work on the basis of
the knowledge base of the particular domain.

Below are some popular examples of the Expert System:

o DENDRAL: It was an artificial intelligence project that was made as a chemical

analysis expert system. It was used in organic chemistry to detect unknown organic
molecules with the help of their mass spectra and knowledge base of chemistry.
o MYCIN: It was one of the earliest backward chaining expert systems that was
designed to find the bacteria causing infections like bacteraemia and meningitis. It
was also used for the recommendation of antibiotics and the diagnosis of blood
clotting diseases.
o PXDES: It is an expert system that is used to determine the type and level of lung
cancer. To determine the disease, it takes a picture from the upper body, which
looks like the shadow. This shadow identifies the type and degree of harm.
o CaDeT: The CaDet expert system is a diagnostic support system that can detect
cancer at early stages.

Characteristics of Expert System

o High Performance: The expert system provides high performance for solving any
type of complex problem of a specific domain with high efficiency and accuracy.
 o Understandable: It responds in a way that can be easily understandable by the
user. It can take input in human language and provides the output in the same
way.
o Reliable: It is much reliable for generating an efficient and accurate output.
o Highly responsive: ES provides the result for any complex query within a very
short period of time.

Components of Expert System

An expert system mainly consists of three components:

o User Interface
o Inference Engine
o Knowledge Base

1. User Interface

With the help of a user interface, the expert system interacts with the user, takes queries as an
input in a readable format, and passes it to the inference engine. After getting the response from
the inference engine, it displays the output to the user. In other words, it is an interface that
helps a non-expert user to communicate with the expert system to find a solution.

2. Inference Engine(Rules of Engine)

o The inference engine is known as the brain of the expert system as it is the main processing unit of
the system. It applies inference rules to the knowledge base to derive a conclusion or deduce new
information. It helps in deriving an error-free solution of queries asked by the user.

o With the help of an inference engine, the system extracts the knowledge from the knowledge base.

o There are two types of inference engine:

o Deterministic Inference engine: The conclusions drawn from this type of inference engine are
assumed to be true. It is based on facts and rules.

o Probabilistic Inference engine: This type of inference engine contains uncertainty in conclusions,
and based on the probability.

Inference engine uses the below modes to derive the solutions:

o Forward Chaining: It starts from the known facts and rules, and applies the inference rules to add
their conclusion to the known facts.

o Backward Chaining: It is a backward reasoning method that starts from the goal and works
backward to prove the known facts.

3. Knowledge Base
o The knowledgebase is a type of storage that stores knowledge acquired from the different experts
of the particular domain. It is considered as big storage of knowledge. The more the knowledge
base, the more precise will be the Expert System.

o It is similar to a database that contains information and rules of a particular domain or subject.

o One can also view the knowledge base as collections of objects and their attributes. Such as a Lion
is an object and its attributes are it is a mammal, it is not a domestic animal, etc.

Components of Knowledge Base

o Factual Knowledge: The knowledge which is based on facts and accepted by knowledge engineers
comes under factual knowledge.
 o Heuristic Knowledge: This knowledge is based on practice, the ability to guess, evaluation, and
experiences.

Knowledge Representation: It is used to formalize the knowledge stored in the

knowledge base using the If-else rules.

Knowledge Acquisitions: It is the process of extracting, organizing, and structuring the

domain knowledge, specifying the rules to acquire the knowledge from various experts,
and store that knowledge into the knowledge base.

Development of Expert System

Here, we will explain the working of an expert system by taking an example of MYCIN ES.
Below are some steps to build an MYCIN:

o Firstly, ES should be fed with expert knowledge. In the case of MYCIN, human experts specialized
in the medical field of bacterial infection, provide information about the causes, symptoms, and
other knowledge in that domain.

o The KB of the MYCIN is updated successfully. In order to test it, the doctor provides a new problem
to it. The problem is to identify the presence of the bacteria by inputting the details of a patient,
including the symptoms, current condition, and medical history.

o The ES will need a questionnaire to be filled by the patient to know the general information about
the patient, such as gender, age, etc.

o Now the system has collected all the information, so it will find the solution for the problem by
applying if-then rules using the inference engine and using the facts stored within the KB.

o In the end, it will provide a response to the patient by using the user interface.

Participants in the development of Expert System

There are three primary participants in the building of Expert System:

1. Expert: The success of an ES much depends on the knowledge provided by human experts. These
experts are those persons who are specialized in that specific domain.

2. Knowledge Engineer: Knowledge engineer is the person who gathers the knowledge from the
domain experts and then codifies that knowledge to the system according to the formalism.

3. End-User: This is a particular person or a group of people who may not be experts, and working
on the expert system needs the solution or advice for his queries, which are complex.
Why Expert System?

Before using any technology, we must have an idea about why to use that technology and hence
the same for the ES. Although we have human experts in every field, then what is the need to
develop a computer-based system. So below are the points that are describing the need of the ES:

1. No memory Limitations: It can store as much data as required and can memorize it at the time of its
application. But for human experts, there are some limitations to memorize all things at every time.

2. High Efficiency: If the knowledge base is updated with the correct knowledge, then it provides a highly
efficient output, which may not be possible for a human.

3. Expertise in a domain: There are lots of human experts in each domain, and they all have different
skills, different experiences, and different skills, so it is not easy to get a final output for the query. But if
we put the knowledge gained from human experts into the expert system, then it provides an efficient
output by mixing all the facts and knowledge

4. Not affected by emotions: These systems are not affected by human emotions such as fatigue, anger,
depression, anxiety, etc.. Hence the performance remains constant.

5. High security: These systems provide high security to resolve any query.
 6. Considers all the facts: To respond to any query, it checks and considers all the available facts and
provides the result accordingly. But it is possible that a human expert may not consider some facts due to
any reason.

7. Regular updates improve the performance: If there is an issue in the result provided by the expert
systems, we can improve the performance of the system by updating the knowledge base.

Capabilities of the Expert System

Below are some capabilities of an Expert System:

o Advising: It is capable of advising the human being for the query of any domain from the particular
ES.

o Provide decision-making capabilities: It provides the capability of decision making in any

domain, such as for making any financial decision, decisions in medical science, etc.

o Demonstrate a device: It is capable of demonstrating any new products such as its features,
specifications, how to use that product, etc.

o Problem-solving: It has problem-solving capabilities.

o Explaining a problem: It is also capable of providing a detailed description of an input problem.

o Interpreting the input: It is capable of interpreting the input given by the user.

o Predicting results: It can be used for the prediction of a result.

o Diagnosis: An ES designed for the medical field is capable of diagnosing a disease without using
multiple components as it already contains various inbuilt medical tools.

Advantages of Expert System

o These systems are highly reproducible.

o They can be used for risky places where the human presence is not safe.

o Error possibilities are less if the KB contains correct knowledge.

o The performance of these systems remains steady as it is not affected by emotions, tension, or
fatigue.

o They provide a very high speed to respond to a particular query.

Limitations of Expert System
o The response of the expert system may get wrong if the knowledge base contains the wrong
information.

o Like a human being, it cannot produce a creative output for different scenarios.

o Its maintenance and development costs are very high.

o Knowledge acquisition for designing is much difficult.

o For each domain, we require a specific ES, which is one of the big limitations.

o It cannot learn from itself and hence requires manual updates.

Applications of Expert System

o In designing and manufacturing domain
It can be broadly used for designing and manufacturing physical devices such as camera lenses and
automobiles.

o In the knowledge domain

These systems are primarily used for publishing the relevant knowledge to the users. The two
popular ES used for this domain is an advisor and a tax advisor.

o In the finance domain

In the finance industries, it is used to detect any type of possible fraud, suspicious activity, and
advise bankers that if they should provide loans for business or not.

o In the diagnosis and troubleshooting of devices

In medical diagnosis, the ES system is used, and it was the first area where these systems were used.

o Planning and Scheduling

The expert systems can also be used for planning and scheduling some particular tasks for achieving
the goal of that task

Architecture of Expert System in AI

Expert architecture is internally structure that represents to the
knowledge base has the certain domain knowledge that is
implemented by an expert to server conclusion from facts.
Explain Structure of Expert System:
• Knowledge Base – It is warehouse of special heuristics or rules,
which are used directly by knowledge, facts (productions). It has
knowledge that is needed for understanding, formulating, &
problem solving.
• Working Memory – It helps to describe the current running
problem and record intermediate output.
Records Intermediate Hypothesis & Decisions: 1. Plan, 2. Agenda, 3.
Solution
• Inference Engine – It is heart of expert system as well as helps
to manage entire structure of expert system, and it delivers to
different methodology for reasoning.
• Explanation System – It helps to trace responsibility and justify
the behavior of expert system by firing questions and answers,
such as Why, How, What, Where, When, Who.
• User Interface – It allows users to insert their queries with using
own Natural Language Processing otherwise menus & graphics.
 • Knowledge Engineer – Main objective of this engineer is to
design system for specific problem domain with using of expert
system shell.
• System Engineer – To design user interface and declarative
format of knowledge base as well as to build inference engine
• Users – They are non expert person who want to seek direct
advice.
Expert System Shell
Expert system shell contains the special software development
environment, and it has basic components of expert system such as –
Knowledge-based management system, Workplace, Explanation
facility, Reasoning capacity, Inference engine, user interface. This shell
is linked along with pre-defined method for designing different
applications through configuring of those components.

Examples for Shell:

• CLIPS (C Language Integrated Production System)
• OPS5, ART, JESS, and Eclipse
Components of Expert System
There are many components that are vital roles play in the structure of
expert system, because every component have own importance in the
heart of expert system.
 •
Knowledge-Based System
• Workplace
• Explanation facility
• Reasoning capacity
• Inference engine
• User interface
Knowledge-Based Management System
This is similar to the database management system in an information
system. Its main function Build knowledge base with knowledge and
rules.
Workplace
A work area or blackboard is a memory area used to describe the
current problem And archiving intermediate results.
Explanation Facility
Most expert systems have an explanation facility. It tells you how the
recommendations are Removed User can know how the expert system
reached the solution, why some options Disclaimed, why was asked for
some information, etc. The clarification feature answers these
Questions by reference to system targets, data input and decision rules.
For example, in the case of evaluation of loan proposal, the explanation
of the expert system will be clarified on the facility inquiry Why an
application was approved and why the other was rejected. In the case
of a medical specialist System such as Mycin, this feature produces
trust about the expert system and the user The solution provides this
problem.
Reasoning Capacity
The expert system has the ability to analyze whether its solution failed
or succeeded Methods to improve its solution.
Inference Engine
Intensive engine model works in the disassembly support system. this
Manipulate a series of rules using forward chewing and backward
chaining techniques. The following is a series of engine checking
ahead… Then check the situation. based on A special solution is
answered. In the techniques of backward chains, ingestion The engine
starts with the target and checks if the conditions that go to that goal
are present.
User Interface
The system provides an interface for users to interact with the system
to generate solutions. This decision is similar to the communication
feature in the support system. Artificial intelligence Technology tries to
provide users with a natural language interface.