Artificial Intelligence - Introduction : AI Course Lecture 1- 6, notes, slides Introduction

www.myreaders.info/ , RC Chakraborty, e-mail rcchak@gmail.com , June 01, 2010

www.myreaders.info/html/artificial_intelligence.html Artificial Intelligence
www.myreaders.info

Topics
(Lectures 01, 02, 03, 04, 05, 06 6 hours) Slides
Return to Website
1. Definitions 03-10
Introduction Artificial Intelligence, Intelligence, Intelligent behavior, Understanding
AI, Hard or Strong AI, Soft or Weak AI, Cognitive Science.
Artificial Intelligence
2. Goals of AI 11-12
Introduction to Artificial Intelligence, topics : Definitions, goals, General AI Goal, Engineering based AI Goal, Science based AI Goal.
approaches, techniques, and branches; Intelligent behavior,
3. AI Approaches 13-16
understanding AI, hard or strong AI, soft or weak AI, cognitive
science. General, engineering and science based AI Goals. AI Cognitive science, Laws of thought, Turing Test, Rational agent.

approaches - cognitive science, laws of thought, turing test, rational

4. AI Techniques 17-32
agent. AI Techniques that make system to behave as intelligent -
Techniques that make system to behave as Intelligent
describe and match, goal reduction, constraint satisfaction, tree
Describe and match, Goal reduction, Constraint satisfaction, Tree
searching, generate and test, rule based systems. Biology-inspired AI
Searching, Generate and test, Rule based systems,
techniques - neural networks, genetic algorithms, reinforcement
Biology-inspired AI techniques
learning. Branches of AI - logical AI, search in AI, pattern recognition,
Neural Networks, Genetic Algorithms, Reinforcement learning.
knowledge representation, inferencing, common sense knowledge and
reasoning, learning, planning, epistemology, ontology, heuristics, 5. Branches of AI 33-45
genetic programming. Applications of AI - game playing, speech Logical AI, Search in AI, Pattern Recognition, Knowledge Representation,
recognition, understanding natural language, computer vision, expert Inference, Common sense knowledge and reasoning, Learning, Planning,
systems.
Epistemology, Ontology, Heuristics, Genetic programming.

6. Applications of AI 46-50

Game playing, Speech Recognition, Understanding Natural Language,

Computer Vision, Expert Systems.

7. References 51

02
 AI - Definitions
Introduction 1. Definitions

1.1 Artificial Intelligence (AI)

What is Artificial Intelligence ?
The definitions of AI outlined in textbooks
• John McCarthy, who coined the term Artificial Intelligence in 1956,
(a) 'The exciting new effort to make (b) 'The study of mental faculties through
defines it as "the science and engineering of making intelligent computers think ... machines with the use of computational models'
machines", especially intelligent computer programs. minds, in the full and literal sense' (Charniak and McDermott, 1985)
(Haugeland, 1985)

• Artificial Intelligence (AI) is the intelligence of machines and the branch 'The automation of activities that we 'The study of the computations that
of computer science that aims to create it. associate with human thinking, make it possible to perceive, reason,
activities such as decision-making, and act' (Winston, 1992)
problem solving, learning ...'
• Intelligence is the computational part of the ability to achieve goals (Bellman, 1978)
in the world. Varying kinds and degrees of intelligence occur in
(C) 'The art of creating machines that (d) 'A field of study that seeks to explain
people, many animals and some machines. perform functions that require and emulate intelligent behavior in
intelligence when performed by terms of computational processes'
people' (Kurzweil, 1990) (Schalkoff, 1990)
• AI is the study of the mental faculties through the use of
computational models. 'The study of how to make computers 'The branch of computer science that
do things at which, at the moment, is concerned with the automation of
people are better' (Rich and Knight, intelligent behavior' (Luger and
• AI is the study of : How to make computers do things which, at the
1991) Stubblefield, 1993)
moment, people do better.

▪ The definitions on the top,

• AI is the study and design of intelligent agents, where an intelligent
(a) and (b) are concerned with reasoning , whereas those on the bottom,
agent is a system that perceives its environment and takes actions
(c) and (d) address behavior.
that maximize its chances of success.
03
▪ The definitions on the left,
(a) and (c) measure success in terms of human performance, whereas
those on the right,
(b) and (d) measure ideal concept of intelligence called rationality.

Note : A system is rational if it does the right thing.

04
 AI - Definitions AI - Definitions
1.2 Intelligence 1.3 Intelligent Behavior

 Relate to tasks involving higher mental processes.

 Perceiving one’s environment,
Examples:
creativity, solving problems, pattern recognition, classification, learning,
 Acting in complex environments,

induction, deduction, building analogies, optimization, language  Learning and understanding from experience,
processing, knowledge and many more.
 Reasoning to solve problems and discover hidden knowledge,

 Intelligence is the computational part of the ability to achieve goals.  Knowledge applying successfully in new situations,

05  Thinking abstractly, using analogies,

 Communicating with others, and more like

 Creativity, Ingenuity, Expressive-ness, Curiosity.

06
 AI - Definitions AI - Definitions
1.4 Understanding AI 1.5 Hard or Strong AI

 How knowledge is acquired, represented, and stored; ▪ Generally, artificial intelligence research aims to create AI that can
replicate human intelligence completely.
 How intelligent behavior is generated and learned;

 How motives, emotions, and priorities are developed and used; ▪ Strong AI refers to a machine that approaches or supersedes human
intelligence,
 How sensory signals are transformed into symbols;
◊ If it can do typically human tasks,
 How symbols are manipulated to perform logic, to reason about past, ◊ If it can apply a wide range of background knowledge and

and plan for future; ◊ If it has some degree of self-consciousness.

 How mechanisms of intelligence produce the phenomena of illusion, ▪ Strong AI aims to build machines whose overall intellectual ability is
belief, hope, fear, dreams, kindness and love.” indistinguishable from that of a human being.
08
07
 AI - Definitions AI - Definitions
1.6 Soft or Weak AI 1.7 Cognitive Science

▪ Weak AI refers to the use of software to study or accomplish specific ▪ Aims to develop, explore and evaluate theories of how the mind
problem solving or reasoning tasks that do not encompass the full works through the use of computational models.

range of human cognitive abilities.

▪ The important is not what is done but how it is done;
means intelligent behavior is not enough, the program must
▪ Example : a chess program such as Deep Blue.
operate in an intelligent manner.
▪ Weak AI does not achieve self-awareness; it demonstrates wide range
of human-level cognitive abilities; it is merely an intelligent, a specific ▪ Example : The Chess programs are successful, but say little about
problem-solver. the ways humans play chess.
09
10
 Goals of AI Goals of AI
2. Goals of AI • General AI Goal
 Replicate human intelligence : still a distant goal.
• The definitions of AI gives four possible goals to pursue :
 Solve knowledge intensive tasks.
1. Systems that think like humans. 2. Systems that think rationally.
 Make an intelligent connection between perception and action.
3. Systems that act like humans 4. Systems that act rationally  Enhance human-human, human-computer and computer to computer
interaction / communication.

• Traditionally, all four goals have been followed and the approaches were: Engineering based AI Goal

Human-like Rationally  Develop concepts, theory and practice of building intelligent machines
 Emphasis is on system building.
Think (1) Cognitive science Approach (2) Laws of thought Approach
Science based AI Goal

 Develop concepts, mechanisms and vocabulary to understand biological

Act (3) Turing Test Approach (4) Rational agent Approach intelligent behavior.
 Emphasis is on understanding intelligent behavior.
12

• Most of AI work falls into category (2) and (4).

11
 AI Approaches AI Approaches
3. AI Approaches 3.2 Laws of Thought : Think Rationally

The approaches followed are defined by choosing goals of the computational ▪ The study of mental faculties through the use of computational models;
model, and basis for evaluating performance of the system. that it is, the study of the computations that make it possible to
perceive, reason, and act.
3.1 Cognitive science : Think human-like
▪ Focus is on inference mechanisms that are provably correct and
▪ An exciting new effort to make computers think; that it is, the guarantee an optimal solution.
machines with minds, in the full and literal sense.
▪ Focus is not just on behavior and I/O, but looks at reasoning process. ▪ Develop systems of representation to allow inferences to be like
“Socrates is a man. All men are mortal. Therefore Socrates is mortal.”
▪ Computational model as to how results were obtained.
▪ Goal is to formalize the reasoning process as a system of logical
▪ Goal is not just to produce human-like behavior but to produce a
rules and procedures for inference.
sequence of steps of the reasoning process, similar to the steps followed
by a human in solving the same task. ▪ The issue is, not all problems can be solved just by reasoning
13 and inferences.
14
 AI Approaches AI Approaches
3.3 Turing Test : Act Human-like 3.4 Rational Agent : Act Rationally

▪ The art of creating machines that perform functions requiring ▪ Tries to explain and emulate intelligent behavior in terms of
intelligence when performed by people; that it is the study of, how to computational processes; that it is concerned with the automation of
make computers do things which at the moment people do better. intelligence.
▪ Focus is on action, and not intelligent behavior centered around
▪ Focus is on systems that act sufficiently if not optimally in all situations;
representation of the world.
▪ A Behaviorist approach, is not concerned with how to get results but ▪ It is passable to have imperfect reasoning if the job gets done.

to the similarity to what human results are. ▪ Goal is to develop systems that are rational and sufficient.
▪ Example : Turing Test 16
◊ 3 rooms contain: a person, a computer, and an interrogator.
◊ The interrogator can communicate with the other 2 by teletype (to
avoid the machine imitate the appearance or voice of the person).
◊ The interrogator tries to determine which is the person and which is
the machine.
◊ The machine tries to fool the interrogator to believe that it is the
human, and the person also tries to convince the interrogator that it
is the human.
◊ If the machine succeeds in fooling the interrogator, then conclude
that the machine is intelligent.

▪ Goal is to develop systems that are human-like.

15
 AI Techniques AI Techniques
4. AI Techniques 4.1 Techniques that make system to behave as "Intelligent"

Various techniques that have evolved, can be applied to a variety of AI tasks.

• Describe and Match
The techniques are concerned with how we represent, manipulate and reason
with knowledge in order to solve problems. ■ Model is a description of a system’s behavior.

■ Finite state model consists of a set of states, a set of input events

Example
and the relations between them. Given a current state and an
• Techniques, not all "intelligent" but used to behave as intelligent
input event you can determine the next current state of the model.
 Describe and match  Goal reduction
■ Computation model is a finite state machine. It includes of a set of
 Constraint satisfaction  Tree Searching

 Generate and test  Rule based systems states, a set of start states, an input alphabet, and a transition

function which maps input symbols and current states to a next

• Biology-inspired AI techniques are currently popular
state.
 Neural Networks  Genetic Algorithms
■ Representation of computational system include start and end
 Reinforcement learning
state descriptions and a set of possible transition rules that might be
17
applied. Problem is to find the appropriate transition rules.

■ Transition relation: If a pair of states (S, S') is such that one move
takes the system from S to S', then the transition relation is
represented by S => S’

■ State-transition system is called deterministic if every state has at

most one successor; it is called non-deterministic if at least one
state has more than one successor.

■ Examples of some possible transitions between states are shown for

the Towers of Hanoi puzzle.
18
 AI Techniques AI Techniques
■ Puzzle : Towers of Hanoi with only 2 disks • Goal Reduction

Solve the puzzle : ■ Goal-reduction procedures are a special case of the procedural
representations of knowledge in AI; an alternative to declarative, logic-
Initial state Goal state
based representations.

■ The process involves the hierarchical sub-division of goals into sub-

Move the disks from the leftmost post to the rightmost post while
goals, until the sub-goals which have an immediate solution are reached
never putting a larger disk on top of a smaller one;
and said “goal has been satisfied”.
move one disk at a time, from one peg to another;
middle post can be used for intermediate storage. ■ Goal-reduction process is illustrated in the form of AND/OR tree drawn
Play the game in the smallest number of moves possible. upside-down.

■ Possible state transitions in the Towers of Hanoi puzzle with 2 disks. ◊ Goal levels : Higher-level goals are higher in the tree, and lower-
level goals are lower in the tree.
[1, 2] [ ] []
◊ Arcs are directed from a higher-to-lower level node represents the
reduction of higher-level goal to lower-level sub-goal.
◊ Nodes at the bottom of the tree represent irreducible action goals.

[ 2] [1 ] [ ] [2] [ ] [1] ■ An AND-OR tree/graph structure can represent relations between goals
and sub-goals, alternative sub-goals and conjoint sub-goals.
[example in the next slide]

[ ] [1 ] [2] [ ] [2] [1] 20

[ ] [ ] [1, 2] [1] [ ] [2] [1] [2] [ ] [ ] [1, 2] [ ]

Towers of Hanoi puzzle

■ Shortest solution is the sequence of transitions from the top state

downward to the lower left.

19
 AI Techniques AI Techniques
■ Example Goal Reduction • Constraint Satisfaction Techniques
AND-OR tree/graph structure to represent facts such as “enjoyment”,
■ Constraint is a logical relation among variables. e.g. “circle is inside
“earning/save money”, “old age” etc.
the square” – The constraints relate objects without precisely specifying

Improve enjoyment of life

their positions; moving any one, the relation is still maintained.

OR ■ Constraint satisfaction is a process of finding a solution to a set of

constraints – the constraints express allowed values for variables and
Provide for old age Improve standard
of living finding solution is evaluation of these variables that satisfies all
AND Work less hard
constraints.

Save money Earn more money ■ Constraint Satisfaction Problem (CSP) and its solution
OR ◊ A Constraint Satisfaction Problem (CSP) consists of :
‡ Variables, a finite set X = {x1 , . . . , xn } ,
Go on strike Improve productivity
‡ Domain, a finite set Di of possible values which each variable xi
AND-OR tree/graph structure can take,

The above AND-OR tree/graph structure describes ‡ Constraints, a set of values that the variables can simultaneously
satisfy the constraint (e.g. D1 != D 2)
◊ Hierarchical relationships between goals and subgoals
The “going on strike” is a sub-goal of “earning more money”, is a ◊ A solution to a CSP is an assignment of a value from its domain to
sub-goal of “improving standard of living ”, is a sub-goal of every variable satisfying every constraint; that could be :
“improving enjoyment of life”. ‡ one solution, with no preference as to which one,
‡ all solutions,
◊ Alternative ways of trying to solve a goal
‡ an optimal, or a good solution - Constraint Optimization Problem
The “going on strike” and “increasing productivity” are alternative
ways of trying to “earn more money” (increase pay). (COP).
22
e.g.: “improving standard of living” and “working less hard” are
alternative ways of trying to “improve enjoyment of life”.

◊ Conjoint sub-goals
To “provide for old age”, not only need to “earn more money”, but
as well need to “save money”.
21
 AI Techniques AI Techniques
[continued from Constraint Satisfaction Techniques] ■ Example 2 : Map Coloring
■ Constraint satisfaction has application in Artificial Intelligence,
Problem : Given a map (graph) and a number of colors, the problem is
Programming Languages, Symbolic Computing, Computational Logic.
to assign colors to those areas in the map (nodes) satisfying the
■ Example 1 : N-Queens puzzle constraint that no adjacent nodes (areas) have the same color assigned

Problem : Given any integer N, place N queens on N*N chessboard to them.

satisfying constraint that no two queens threaten each other. Solution : To model this Map Coloring problem

(a queen threatens other queens on same row, column and diagonal). ◊ Label each node of the graph with a variable (domain corresponding

Solution : To model this problem to the set of colors);

◊ Assume that each queen is in different column; ◊ Introduce the non-equality constraint between two variables

◊ Assign a variable Ri (i = 1 to N) to the queen in the i-th column labeling adjacent nodes.

indicating the position of the queen in the row. ◊ A 4 – color map

◊ Apply "no-threatening" constraints between each couple Ri and Rj

of queens and evolve the algorithm.
◊ Example : 8 - Queens puzzle

a b c d e f g h a b c d e f g h
8 8 8 8 ◊ The "Four Color Theorem", states that 4 - colors are sufficient to
7 7 7 7
color any map so that regions sharing a common border receive
6 6 6 6
different colors.
5 5 5 5
24
4 4 4 4

3 3 3 3

2 2 2 2

1 1 1 1

a b c d e f g h a b c d e f g h
Unique solution 1 Unique solution 2

◊ The eight queens puzzle has 92 distinct solutions. If solutions that

differ only by symmetry operations (rotations and reflections) of the
board are counted as one. The puzzle has 12 unique solutions. Only
two solutions are presented above.
23
 AI Techniques
• Tree Searching
AI Techniques
■ Many problems (e.g. goal reduction, constraint networks) can be described in the form of a search tree. A ◊ Hill climbing ◊ Breadth-first search
solution to the problem is obtained by finding a path through this tree.
* Like depth first but involving * Look for a solution amongst
■ A search through the entire tree, until a satisfactory path is found, is called exhaustive search.
some quantitative decision on all nodes at a given level
■ Tree search strategies:
the "most likely" path to before proceeding to the next.
◊ Depth-first search
follow at each node.
* Assumes any one path is as good as any other path.
* At each node, pick an arbitrary path and work forward until a solution is found or a dead end is reached.
◊ Beam search ◊ Best-first search
* In the case of a dead end - backtrack to the last node in the tree where a previously unexplored path
branches of, and test this path. * Like breadth first (level by * Like beam search but only
* Backtracking can be of two types : level) but selecting only those proceeding from one "most
– Chronological backtracking : undo everything as we move back "up" the tree to a suitable node.
N nodes at each level that are likely" node at each level.
– Dependency directed backtracking : only withdraw choices that "matter" (ie those on which dead end
depends). "most likely" to lead to a

The four other types of search strategies are : solution.

25
 AI Techniques AI Techniques
• Generate and Test (GT) • Rule-Based Systems (RBSs)
■ Most algorithms for solving Constrain Satisfaction Problems (CSPs) ■ Rule-based systems are simple and successful AI technique.
search systematically through the possible assignments of values.
◊ Rules are of the form: IF <condition> THEN <action>.
◊ CSP algorithms guarantee to find a solution, if one exists, or to
◊ Rules are often arranged in hierarchies (“and/or” trees).
prove that the problem is unsolvable.
◊ the disadvantage is that they take a very long time to do so. ◊ When all conditions of a rule are satisfied the rule is triggered.

■ Generate-and-test method ■ RBS Components : Working Memory, Rule Base, Interpreter.

The method first guesses the solution and then tests whether this
solution is correct, means solution satisfies the constraints.
Change Interpreter
◊ This paradigm involves two processes:
* Generator to enumerate possible solutions (hypotheses). Conditions
* Test to evaluate each proposed solution Rules
Observed
Working
◊ The algorithm is Data Memory
Generate labels
Rule
Test satisfaction Base
Disadvantages Rule-Based System

◊ Not very efficient; generates many wrong assignments of values to Each of these 3 components are described in the next slide
27
variables which are rejected in the testing phase.
◊ Generator leaves conflicting instantiations and it generates other
assignments independently of the conflict.
◊ For better efficiency GT approach need to be supported by
backtracking approach.

Example: Opening a combination lock without knowing the combination.

26
 AI Techniques AI Techniques
■ RBS components - Description 4.2 Biology-Inspired AI Techniques

◊ Working Memory (WM) • Neural Networks (NN)

‡ Contains facts about the world observed or derived from a rule;
■ Neural Networks model a brain learning by example.
stored as a triplet < object, attribute, values >
■ Neural networks are structures trained to recognize input patterns.
e.g. < car, color, red > : “The color of my car is red”.
‡ Contains temporary knowledge about problem-solving session. ■ Neural networks typically take a vector of input values and produce a
‡ Can be modified by the rules. vector of output values; inside, they train weights of "neurons".

◊ Rule Base (RB) ■ A Perceptron is a model of a single `trainable' neuron, shown below :
‡ RB contains rules; each rule is a step in a problem solving.
◊ x1, x2, ..., xn are inputs as real
X1
‡ Rules are domain knowledge and modified only from outside. w1
numbers or boolean values depends
‡ Rule syntax is IF <condition> THEN <action> X2 w2
on problem.
e.g. IF <(temperature, over, 20> w3
X3
●
T Y ◊ w1 , w2, ..., wn are weights of the
THEN <add (ocean, swimable, yes)>
● wn edges and are real valued.
‡ If the conditions are matched to the working memory and if Xn ◊ T is the threshold and is real valued.
fulfilled then rule may be fired. Perceptron Model ◊ y is the output and is boolean.
‡ RB actions are :
If the net input which is w1 x1 + w2 x2 + ... + wn xn is greater than
“Add” fact(s) to WM;
the threshold T then output y is 1 else 0.
“Remove” fact(s) from WM;
“Modify” fact(s) in WM; ■ Neural networks use supervised learning, in which inputs and outputs

are known and the goal is to build a representation of a function that

◊ Interpreter
‡ It is the domain independent reasoning mechanism for RBS. will approximate the input to output mapping.
29
‡ It selects rule from Rule Base and applies by performing action.

‡ It operates on a cycle:

Retrieval - Finds the rules that matches the current WM;

Refinement - Prunes, reorders and resolves conflicts;

Execution - Executes the actions of the rules in the Conflict Set,

then applies the rule by performing action.
28
 AI Techniques AI Techniques
• Genetic Algorithms (GA) ■ Genetic Algorithm Steps

■ GAs are part of evolutionary computing, a rapidly growing area of AI. (1) [Start] Generate random population of n chromosomes (Encode

■ Genetic algorithms are implemented as a computer simulation, where

suitable solutions for the problem)
(2) [Fitness] Evaluate the fitness f(x) of each chromosome x in the
techniques are inspired by evolutionary biology.
population.
■ Mechanics of biological evolution
(3) [New population] Create a new population by repeating following

◊ Every organism has a set of rules, describing how that organism is steps until the new population is complete.
built, and encoded in the genes of an organism. (a) [Selection] Select two parent chromosomes from a population
◊ The genes are connected together into long strings called according to their fitness.
chromosomes. (b) [Crossover] With a crossover probability, cross over the parents
◊ Each gene represents a specific trait (feature) of the organism and to form new offspring (children). If no crossover was performed,
has several different settings, e.g. setting for a hair color gene may offspring is the exact copy of parents.
be black or brown. (c) [Mutation] With a mutation probability, mutate new offspring at
◊ The genes and their settings are referred as an organism's each locus (position in chromosome).
genotype. (d) [Accepting] Place new offspring in the new population.

◊ When two organisms mate they share their genes. The resultant (4) [Replace] Use new generated population for a further run of the
offspring may end up having half the genes from one parent and
algorithm.
half from the other. This process is called cross over.
(5) [Test] If the end condition is satisfied, stop, and return the best
◊ A gene may be mutated and expressed in the organism as a
solution in current population.
completely new trait.
(6) [Loop] Go to step 2.

■ Thus, Genetic Algorithms are a way of solving problems by mimicking ■ Genetic Algorithms does unsupervised learning – the right answer is
processes, the nature uses, Selection, Crosses over, Mutation and not known beforehand.
Accepting to evolve a solution to a problem. 31

30
 AI Techniques Branches of AI
• Reinforcement Learning (RL) 5. Branches of AI

■ RL is learning from interaction with an environment; from the • Logical AI

consequences of action, rather than from explicit teaching. ■ Logic is a language for reasoning; a collection of rules used while
■ RL is conducted within the mathematical framework of Markov decision
doing logical reasoning.
processes (MDPs). ■ Types of logic
■ The basic RL model consists : a set of “environment states - S ” ;
◊ Propositional logic - logic of sentences
a set of “actions - A”; and a set of scalar “rewards - R”.
◊ predicate logic - logic of objects
■ The decision-making agent interacts with its environment so as to
◊ logic involving uncertainties
maximize the cumulative reward it receives over time. The steps are:
◊ Fuzzy logic - dealing with fuzziness
◊ At each time t, the agent perceives its environment state st and
the set of possible actions A(st) .
◊ Temporal logic, etc
■ Propositional logic and Predicate logic are fundamental to all logic
◊ It chooses an action a ∈ A(st) and receives from the environment
◊ Propositional logic
the new state st+1 and a reward rt+1.
‡ Propositions are “Sentences”; either true or false but not both.
◊ Based on these interactions, the agent develops a policy  : S A
‡ A sentence is smallest unit in propositional logic
which maximizes the quantity R = r0+ r1+. . +rn for MDPs.
‡ If proposition is true, then truth value is "true"; else “false”
■ RL methods focus on the kind of learning and decision making ‡ Example : Sentence "Grass is green"; Truth value “ true”;
problems that people face in their normal, everyday lives. Proposition is “yes”
32
◊ Predicate logic
‡ Predicate is a function may be true or false for arguments
‡ Predicate logic are rules that govern quantifiers
‡ Predicate logic is propositional logic added with quantifiers
‡ Examples:
“The car Tom is driving is blue",
"The sky is blue",
"The cover of this book is blue"
Predicate is blue, give a name B ;

Sentence represented as B(x); read B(x) as "x is blue" ;

Object represented as x .
33
 Branches of AI Branches of AI
• Search in AI • Pattern Recognition (PR)
■ Search is a problem-solving technique that systematically consider ■ Definitions : from the literature

all possible action to find a path from initial state to target state.
◊ 'The assignment of a physical object or event to one of pre-specified
■ Search techniques are many; the most fundamental are categories' – Duda and Hart

◊ Depth first ◊ Hill climbing ◊ 'The science that concerns the description or classification
◊ Breadth first ◊ Least cost (recognition) of measurements' – Schalkoff
◊ 'The process of giving names Ω to observations X ' – Schürmann
■ Search components
◊ Pattern Recognition is concerned with answering the question 'What
◊ Initial state - First location
is this?' – Morse
◊ Available actions - Successor function : reachable states
◊ 'A problem of estimating density functions in a high-dimensional
◊ Goal test - Conditions for goal satisfaction
space and dividing the space into the regions of categories or classes'
◊ Path cost - Cost of sequence from initial state to reachable state
– Fukunaga
■ Search objective ■ Pattern recognition problems
◊ Transform initial state into goal state - find a sequence of actions.
◊ Machine vision - Visual inspection, ATR
■ Search solution
◊ Character recognition – Mail sorting, processing bank cheques
◊ Path from initial state to goal - optimal if lowest cost.
34 ◊ Computer aided diagnosis - Medical image/EEG/ECG signal analysis

◊ Speech recognition - Human Computer Interaction, access

■ Approaches for Pattern recognition

◊ Template Matching

◊ Statistical classification

◊ Syntactic or Structural matching

Each of these approaches are explained in the next slide.

35
 Branches of AI Branches of AI
[continued from previous slide Approaches for Pattern Recognition ]
• Knowledge Representation
Template Matching ■ How do we represent what we know?
Match with stored template considering translation, rotation and scale Knowledge is a collection of facts. To manipulate these facts by a
changes; measure similarity (correlation) based on training set. program, a suitable representation is required. A Good representation

Statistical classification facilitates problem solving.

Each pattern is represented in terms of d features (measurements) and ■ Knowledge representation formalisms (techniques)
viewed as a point in a d-dimensional space. Using training sets Different types of knowledge require different types of representation.
establish decision boundaries in the feature space - following decision ◊ Predicate logic :
theoretic or discriminant analysis approaches. Predicate is a function may be TRUE for some arguments, and
FALSE for others.
Syntactic or Structural matching
Complex pattern is composed of sub-patterns and the relations; they ◊ Semantic networks :
A semantic net is just a graph, where the nodes represent concepts,
themselves are built from simpler / elementary sub-patterns are called
and the arcs represent binary relationships between concepts.
primitives. The patterns are viewed as sentences belonging to a
language, primitives are viewed as the alphabet of the language. The
◊ Frames and scripts :
A frame is a data structure that typically consists of : Frame name,
sentences are generated according to a grammar. A large collection of
Slot-filler (relations target), Pointers (links) to other Frames,
complex patterns can be described by a small number of primitives and
Instantiation Procedure (inheritance, default, consistency). The
grammatical rules. The grammar for each pattern class are inferred from
Scripts are linked sentences using frame-like structures; e.g., a
the training samples.
record of sequence of events for a given type of occurrence.
Neural networks ◊ Production rules :
Neural networks are viewed as weighted directed graphs in which the Consists of a set of rules about behavior; a production consists two
nodes are artificial neurons and directed edges (with weights) are parts: a precondition (or IF) and an action (or THEN); if a
connections between neurons input-output. Neural networks have the production's precondition matches the current state of the world,
ability to learn complex nonlinear input-output relationships from the then the production is said to be triggered.
sequential training procedures, and adapt themselves to input data. 37

■ Applications requiring Pattern recognition

◊ Image Proc / Segmentation ◊ Seismic Analysis

◊ Computer Vision ◊ Industrial Inspection
◊ Medical Diagnosis ◊ Financial Forecast
◊ Man and Machine Diagnostics
36
 Branches of AI Branches of AI
• Inference • Common Sense Knowledge and Reasoning
Inference is the act or process of deriving a conclusion based solely on what Common sense is the mental skills that most people have.
one already knows; it is deduction of new facts from old ones; Logic
■ It is the ability to analyze a situation based on its context.
captures inference.
◊ People can think because the brain contain vast libraries of
■ Deductive Inference common sense knowledge and has means for organizing,

◊ It is never false; inference is true if premise is true. acquiring, and using such knowledge.
◊ Computer can not think; the computers programs do many things,
◊ A traditional logic is based on deduction; it is a method of exact
they can play chess at the level of best players but cannot match
inference; there is no possibility of mistake if rules are followed
capabilities of a 3 year old child at recognizing objects. Currently,
exactly.
computers lack common sense.
◊ The information required is complete, precise, and consistent .
■ Researchers have divided common sense capability into :
◊ A logic is monotonic, if the truth of a proposition does not
◊ Common sense knowledge and
change when new information are added to the system.
◊ Common sense reasoning.
■ Inductive Inference
■ Teaching computers common sense
◊ It may be correct or incorrect inference, because in real world the
◊ Project “OpenMind” at MIT - Here the goal is to teach a computer
information is incomplete, inexact, inconsistent;
things that human take them for granted; here the knowledge is
◊ A logic is inductive, also called Induction or Inductive
represented in the form of Semantic net, Probabilistic graphical
reasoning, if the process of reasoning in which the premises of
models, and Story scripts.
an argument are believed to support the conclusion but do not
◊ Project “Cyc” – It is an attempt to manually build a database
ensure it.
of human common sense knowledge; it has 1.5 million
◊ A logic is non-monotonic, if the truth of a proposition may change
collection of common sense facts, but still far away from
when a new information (axioms) is added to or an old information
several hundred million needed.
is deleted from the system. 39

◊ The reasoner draws conclusions tentatively reserving the right to

retract them in the light of further information.
◊ Example: When we hear of a bird, we human infer that bird
can fly, but this conclusion can be reversed when we hear that
it is a penguin; the bird penguin cannot fly.
38
 Branches of AI Branches of AI
• Learning • Planning
Programs learn from what the facts or the behaviors can represent. A plan is a representation of a course of action.

■ Definitions Planning is a problem solving technique.

◊ Herbert Simon 1983 – “Learning denotes changes in the system Planning is a reasonable series of actions to accomplish a goal.

that are adaptive in the sense that they enable the system to do the ■ Planning programs
same task or tasks more efficiently and more effectively the next Start with facts about the world, particularly
time.” ◊ facts about the effects of actions,
◊ Marvin Minsky 1986 – “Learning is making useful changes in the ◊ facts about the particular situation, and
working of our mind.” ◊ statement of a goal.
◊ Ryszard Michalski 1986 – "Learning is constructing or modifying
■ Benefits of planning
representations of what is being experienced."
◊ reducing search,
◊ Mitchell 1997 – “A computer program is said to learn from
◊ resolving goal conflicts, and
experience E with respect to some class of tasks T and performance
◊ providing a basis for error recovery.
measure P, if its performance at tasks in T, as measured by P,
improves with experience E.” ■ Strategy for planning

■ Major Paradigms of Machine Learning A strategy is just a sequence of actions. From facts the program
generate a strategy for achieving the goal.
◊ Rote : Learning by memorization; Saving knowledge so that it can 41

be used again.

◊ Induction : Learning by example; Process of learning by example

where a system tries to induce a general rule from a set of
observed instances.
◊ Analogy : Learning from similarities; Recognize similarities in
information already stored; can determine correspondence between
two different representations.
◊ Genetic Algorithms : Learning by mimicking processes nature
uses; Part of evolutionary computing, a way of solving problems by
mimicking processes, nature uses, selection, crosses over, mutation
and accepting to evolve a solution to a problem.
◊ Reinforcement : Learning from actions; Assign rewards, +ve or -
ve; at the end of a sequence of steps, it learns which actions are
good or bad.
40
 Branches of AI Branches of AI
• Epistemology • Ontology
Epistemology is the theory of knowledge. Ontology is concerned with existence; a study of the categories of things
that exist or may exist in some domain.
■ There are various kinds of knowledge:

◊ knowing how to do something (e.g., how to ride a bicycle), ■ Ontology is a data model, represents a domain and is used to reason

◊ knowing someone in person, and about the objects in that domain and the relations between them.
◊ knowing a place or a city.
■ Ontology is used in artificial intelligence, as a form of knowledge

■ Epistemology is the study of knowledge and justified belief. representation about the world or some part of it.

Propositions ■ Ontology generally describe:

◊ Individuals (instances): the basic or ground level objects

Know-
Truths ledge Beliefs ◊ Classes: sets, collections, or types of objects.

◊ Attributes: properties, features, characteristics, or parameters

that objects can have and share.
◊ Consider a knowledge of proposition - a schema ‘S knows that P’ ;
◊ Question asked - what are the necessary and sufficient conditions? ◊ Relations: ways the objects can be related to one another.

■ Ontology is a specification of a conceptualization.

S knows that P if and only if
43
P is true;

S believes that P is true; and

S is justified in believing that P is true.

Gettier claimed that the above account of knowledge is insufficient.

■ Epistemology is the study of the kinds of knowledge that are required

for solving problems in the world.

42
 Branches of AI Branches of AI
• Heuristics • Genetic programming (GP)
Heuristics are simple, efficient rules; Genetic programming is an automated method for creating program
from a high-level problem statement.
■ Heuristics are in common use as Rule of thumb;

In computer science, a heuristic is an algorithm with provably good run ■ GP starts from a high-level statement of the requirements of a problem

times and with provably good or optimal solution. and attempts to produce a computer program that solves the problem.

■ Heuristics are intended to gain computational performance or ■ The user (human) communicates the high-level statement of
conceptual simplicity, potentially at the cost of accuracy or precision. the problem to the GP system by performing certain well-defined
preparatory steps.
■ People use heuristics to make decisions, come to judgments, and solve

problems, when facing complex problems or incomplete information. ■ The major five preparatory steps, the human user require to specify

These rules work well under most circumstances. for the GP to be evolved are :
◊ Set of terminals (variables , functions, and constants)
■ In AI programs, the heuristic functions are :
◊ Set of primitive functions
◊ used to measure how far a node is from goal state.
◊ Fitness measure (for fitness of individuals in the population),
◊ used to compare two nodes, find if one is better than the other.
◊ Controlling parameters for the run, and
44

◊ Termination criterion for designating the result of the run.

■ The “Run” of genetic programming executes a series of well-defined

problem-independent steps (the flowchart).

45
 Applications of AI Applications of AI
6. Applications of AI • Speech Recognition
■ A process of converting a speech signal to a sequence of words;
• Game playing
■ Games are Interactive computer program, an emerging area in which ■ In 1990s, computer speech recognition reached a practical level for

the goals of human-level AI are pursued. limited purposes.

■ Games are made by creating human level artificially intelligent entities, ■ Using computers recognizing speech is quite convenient, but most users

e.g. enemies, partners, and support characters that act just like find the keyboard and the mouse still more convenient.
humans.
■ The typical usages are :

■ Game play is a search problem defined by: ◊ Voice dialing (Call home),
◊ Initial state - board ◊ Call routing (collect call),
◊ Expand function - build all successor states ◊ Data entry (credit card number).
◊ Cost function - payoff of the state ◊ Speaker recognition.

◊ Goal test - ultimate state with maximal payoff

■ The spoken language interface PEGASUS in the American Airlines'

■ Game playing is characterized by: EAASY SABRE reservation system, allows users to obtain flight
◊ "Unpredictable" opponent information and make reservations over the telephone.
47
◊ Need to specify move for every possible opponent reply.
◊ Time limits - games become boring if there is no action for too long
a time; opponents are unlikely to find goal, must approximate.

■ Computer Games

◊ Computers perform at champion level games, examples :

Checkers, Chess, Othello, Backgammon.
◊ Computers perform well games, example : Bridge
◊ Computers still do badly, example : Go, Hex

■ The Deep Blue Chess program won over world champion Gary
Kasparov.
46
 Applications of AI Applications of AI
• Understanding Natural Language • Computer Vision
Natural language processing (NLP) does automated generation and ■ It is a combination of concepts, techniques and ideas from : Digital

understanding of natural human languages. Image Processing, Pattern Recognition, Artificial Intelligence and
Computer Graphics.
■ Natural language generation system

Converts information from computer databases into normal-sounding ■ The world is composed of 3-D objects, but the inputs to the human eye
human language and computers' TV cameras are 2-D.

■ Natural language understanding system ■ Some useful programs can work solely in 2-D, but full computer vision
Converts samples of human language into more formal representations requires partial 3-D information that is not just a set of 2-D views.
that are easier for computer programs to manipulate.
■ At present there are only limited ways of representing 3-D information
■ Some major tasks in NLP
directly, and they are not as good as what humans evidently use.
◊ Text-to-Speech (TTS) system :
■ Examples
converts normal language text into speech.
◊ Speech recognition (SR) system : ◊ Face recognition :
process of converting a speech signal to a sequence of words; the programs in use by banks
◊ Machine translation (MT) system : ◊ Autonomous driving :
translate text or speech from one natural language to another. The ALVINN system, autonomously drove a van from Washington,
◊ Information retrieval (IR) system : D.C. to San Diego, averaging 63 mph day and night, and in all
search for information from databases such as Internet or World weather conditions.
Wide Web or Intranets. ◊ Other usages
48 Handwriting recognition, Baggage inspection, Manufacturing
inspection, Photo interpretation, etc .
49
 Applications of AI AI- Introduction - References
• Expert Systems 7. References : Textbooks

Systems in which human expertise is held in the form of rules 1. "Artificial Intelligence", by Elaine Rich and Kevin Knight, (2006), McGraw Hill
companies Inc., Chapter 1-22, page 1-613.
■ It enable the system to diagnose situations without the human expert
2. "Artificial Intelligence: A Modern Approach" by Stuart Russell and Peter Norvig,
being present. (2002), Prentice Hall, Chapter 1-27, page 1-1057.

■ A Man-machine system with specialized problem-solving expertise. 3. "Computational Intelligence: A Logical Approach", by David Poole, Alan Mackworth,
and Randy Goebel, (1998), Oxford University Press, Chapter 1-12, page 1-608.
The "expertise" consists of knowledge about a particular domain,
4. "Artificial Intelligence: Structures and Strategies for Complex Problem Solving", by
understanding of problems within that domain, and "skill" at solving George F. Luger, (2002), Addison-Wesley, Chapter 1- 16, page 1-743.
some of these problems.
5. "AI: A New Synthesis", by Nils J. Nilsson, (1998), Morgan Kaufmann Inc., Chapter
1-25, Page 1-493.
■ Knowledge base

A knowledge engineer interviews experts in a certain domain and tries 6. "Artificial Intelligence: Theory and Practice", by Thomas Dean, (1994),
Addison-Wesley, Chapter 1-10, Page 1-650.
to embody their knowledge in a computer program for carrying out
some task. 7. Related documents from open source, mainly internet. An exhaustive list is
being prepared for inclusion at a later date.

■ One of the first expert systems was MYCIN in 1974, which diagnosed 51

bacterial infections of the blood and suggested treatments.

■ Expert systems rely on knowledge of human experts, e.g.

◊ Diagnosis and Troubleshooting :

deduces faults and suggest corrective actions for a malfunctioning
device or process
◊ Planning and Scheduling :
analyzing a set of goals to determine and ordering a set of actions
taking into account the constraints; e.g. airline scheduling of
flights
◊ Financial Decision Making :
advisory programs assists bankers to make loans, Insurance
companies to assess the risk presented by the customer, etc.
◊ Process Monitoring and Control :
analyzes real-time data, noticing anomalies, predicting trends, and
controlling optimality and do failure correction.
50