Basic concepts of Pattern Recognition

Pattern: A pattern is the description of an object. More precisely it is the description of any
member of a category representing a pattern class.
Pattern Class: It is a category determined by some given common attributes.

Example: Alphanumeric character and Chinese characters are patterns and character is a
pattern class.

Pattern Recognition: Pattern recognition can be defined as the categorization of input data
into identifiable classes via the extraction of significant features or attributes of the data from
a background of irrelevant detail.

Some applications of Pattern Recognition

Task of Classification Input Data Output Response

Character Recognition Optical signals or strokes Name of character
Speech Recognition Acoustic waveforms Name of word
Speaker Recognition Voice Name of speaker
Weather Prediction Weather maps Weather forecast
Medical Symptoms Disease Diagnosis
Stock Market Financial news and Predicted market ups and downs.
Prediction charts

Fundamental Problems in Pattern Recognition System Design

 The first one is concerned with the representation of input data which can be
measured from the objects to be recognized. This is called sensing problem
 The second problem concerns the extraction of characteristic features or attributes
from the received input data and the reduction of the dimensionality of pattern
vectors. This is often referred to as the preprocessing and feature extraction problem.
 The third problem involves the determination of optimum decision procedures, which
are needed in the identification and classification process.
Block Diagram of an adaptive pattern recognition system

Contextual
Analysis

Preprocessing &
Objects Measurements Feature vectors
Sensing Feature Selection Categorization
or primitives

Estimation
Adaption Learning
Clustering Concept

 Cluster is defined as the branch of point lies close to one another.

 When the patterns of a class are vectors whose components are real numbers, a
pattern class can be characterized by its clustering properties in the pattern space.

Some Clustering Algorithm

 Cluster Seeking Algorithm

 Maximin Distance Algorithm
 K- means Algorithm

Cluster Seeking Algorithm

1. Suppose that we have set of N sample patterns {X1, X2........., XN }. Let the first cluster
center Z1, be equal to any one of the sample pattern and select one arbitrary non-
negative threshold T.
2. Let choose Z1=X1 and compute distance D21 from X2 to Z1.If it exceeds T, a new
cluster center is Z2=X2, is started otherwise we assign X2 to the domain of cluster
center Z1. Suppose that D21 > T so that Z2 is established
3. In next step, D31 and D32 from X3 to Z1 and Z2 are computed. If both D31 and D32 are
greater than T, a new cluster center Z3= X3 is created otherwise we assign X3 to the
domain of the cluster center to which it is closest.
4. In a similar fashion, the distance from each new pattern to every established cluster
center is computed and thresholded, and a new cluster center is created if all of these
distances exceed T. Otherwise, the pattern is assigned to the domain of the cluster
center to which it is closest
5. The process will continue until all the patterns fall into any group.

Maximin Distance Algorithm

Step-1: Suppose that we have a set of N sample patterns {X1, X2......., XN). Choose Z1 = Xi,
where X, can be any sample point.
Step-2: Determine the farthest sample from Z1 and call it cluster center Z2.
Step-3: For Z3 to ZK:
Step-4: Compute the distance from each remaining sample, Xi, to all Z's. Save the minimum
of these distances for each Xi.
MINi= min {|Xi-Zj|, j)}
Step-5: Consider the sample with the largest minimum distance.
MAX=max{MINi, i }
If that distance is an appreciable fraction of typical distances between existing Z’s, then the
sample is a new cluster center. Go to step 4. Otherwise go to step 6.
Step-6: Assign the remaining samples to the cluster of the nearest Zj.
K- means Algorithm
Step-1: Choose K initial cluster centers Zl (1), Z2 (1)....., ZK (1). These are arbitrary and are
usually selected as the K samples of the given sample set.
Step-2: At the Kth iterative step distribute the samples {X} among the K cluster domains,
using the relation:
X€ Sj (K) if |X-Zj (K) |< |X- Zi (K)|,
i= 1, 2, 3 ………, K and i≠j
Where Sj (k) denotes the set of samples whose cluster center is Zj (K).
Step-3: From the results of step 2, compute the new cluster centers Zj (K+l), j = 1, 2....., K,
such that the sum of the squared distances from all points in Sj(k) to the new cluster center is
minimized.
Therefore, the new cluster center is given by:
Zj (K+l) = ∑ ,
j=1, 2, 3……….K
Where Nj is the number of samples in Sj (K).
The name "K-Means" is derived from the manner in which cluster centers are sequentially
updated.
Step- 4: If Zj (K+1) =Zj (K) for j = 1, 2…... K, the algorithm has converged and the
procedure is terminated. Otherwise go to step 2.
 Problems

1. Apply cluster Seeking algorithm to given dataset:

(0, 0), (0, 1), (5, 4), (5, 5), (4, 5), (1, 0), (4, 2), (0, 4)
When T=2.

Solution:

Let,

X1= (0, 0), X2= (0, 1), X3= (5, 4), X4= (5, 5), X5= (4, 5), X6= (1, 0), X7= (4, 2), X8= (0, 4)

And Z1=X1= (0, 0)

D= √ ‫ݔ‬2 ‫ݔ‬ ‫ݕ‬2 ‫ݕ‬
D (Z1, X2) = √ = 1;

Similarly,

D (Z1, X3) = 6.40; which is greater than T. So, Z2= X3= (5, 4)

D (Z1, X4) = 7.07;

D (Z2, X4) = 1;

D (Z1, X5) =6.403;

D (Z2, X5) =1.41;

D (Z1, X6) = 1;

D (Z2, X6) = 5.66;

D (Z1, X7) = 4.472;

Both are greater than T. So, Z3=X7= (4,
D (Z2, X7) = 2.236;

D (Z1, X8) = 4;

D (Z2, X8) = 5; All are greater than T, So, Z4= X8= (0, 4)
D (Z3, X8) =4.472; X1 Z1

Hence, X2

S (Z1) = {X1, X2, X6} X3 Z2

S (Z2) = {X3, X4, X5} X4

S (Z3) = {X7} X5

S (Z4) = {X8} X6

X7 Z3

X8 Z4
2. Apply maximin distance algorithm on given dataset:
(0, 0), (0, 1), (5, 4), (5, 5), (4, 5), (1, 0), (4, 2), (0, 4)
Letting (0, 0) as first cluster and fraction= .

Solution:

Let,

X1= (0, 0), X2= (0, 1), X3= (5, 4), X4= (5, 5), X5= (4, 5), X6= (1, 0), X7= (4, 2), X8= (0, 4)

Here, Z1=X1= (0, 0)

D= √ ‫ݔ‬2 ‫ݔ‬ ‫ݕ‬2 ‫ݕ‬
D (Z1, X2) =1;

D (Z1, X3) = 6.40;

D (Z1, X4) = 7.07;

D (Z1, X5) = 6.403;

D (Z1, X6) = 1;

D (Z1, X7) = 4.472;

D (Z1, X8) =4;

Here, Max1= 7.0.7

Hence Z2= X4= (5, 5)

Now, D (Z2, X2) =6.40;

D (Z2, X3) = 1;

D (Z2, X5) = 1;

D (Z2, X6) = 6.403;

D (Z2, X7) = 3.162;

D (Z2, X8) =5.099;

Calculating Min1 [D (Z1, Xi), D (Z2, Xi)]

Min (X2) = 1 (Z1)

Min (X3) = 1 (Z2)

Min (X5) = 1 (Z2)

 Min(X6) = 1 (Z1)

Min(X7) = 3.162 (Z2)

Min(X8) = 4 (Z1)

Here, Max2= 4, 2

And, Max2>

4> 3.52
So, Z3= X8= (0, 4)

Now, D (Z3, X2) = 9;

D (Z3, X3) = 5;

D (Z3, X5) = 4.123;

D (Z3, X6) = 4.123;

D (Z3, X7) = 4.472;

Calculating Min2 [D (Z1, Xi), D (Z2, Xi), D (Z3, Xi)]

Min (X2) = 1 (Z1)

Min (X3) = 1 (Z2)

Min (X5) = 1 (Z2)

Min(X6) = 1 (Z1)

Min(X7) = 3.162 (Z2)

Here, Max3= 3.162, 2

And
Max3>

3.162> 2.76

So, Z4= X7= (4, 2)

Now, D (Z4, X2) = 4.123;

D (Z4, X3) = 2.236;

D (Z4, X5) = 3;

D (Z4, X6) = 3.61;

Calculating Min3 [D (Z1, Xi), D (Z2, Xi), D (Z3, Xi), D (Z4, Xi)]

Min (X2) = 1 (Z1)

Min (X3) = 1 (Z2)

Min (X5) = 1 (Z2)

Min(X6) = 1 (Z1)

Here Max4= 1, 2

And Max4
X1 Z1
So program will be terminated
X2
Hence X3
S (Z1) = {X1, X2, X6} X4 Z2

S (Z2) = {X3, X4, X5} X5

S (Z3) = {X8} X6

S (Z4) = {X7} X7 Z3

X8 Z4
3. Apply K- means algorithm to the data set given:
(0, 0), (0, 1), (5, 4), (5, 5), (4, 5), (1, 0), (4, 2), (0, 4)
Letting K=3 and the first three samples as first three centers.

Solution:
Let,
X1= (0, 0), X2= (0, 1), X3= (5, 4), X4= (5, 5), X5= (4, 5), X6= (1, 0), X7= (4, 2), X8= (0, 4)

Here, Z1 (1) = X1= (0, 0) X1 Z1

X2 Z2
Z2 (1) = X2= (0, 1) X3 Z3

Z3 (1) = X3= (5, 4)

D= √ ‫ݔ‬2 ‫ݔ‬ ‫ݕ‬2 ‫ݕ‬

D(Z1 (1),Xi) D(Z2 (1) ,Xi) D(Z3 (1) ,Xi)

D (Z1 (1), X4) = 7.07; D (Z2 (1), X4) = 6.403; D (Z3 (1), X4) = 1;
D (Z1 (1), X5) = 6.403; D (Z2 (1), X5) = 5.656; D (Z3 (1), X5) = 1.414;
D (Z1 (1), X6) = 1; D (Z2 (1), X6) = 1.414; D (Z3 (1), X6) = 5.656;
D (Z1 (1), X7) = 4.472; D (Z2 (1), X7) = 4.123; D (Z3 (1), X7) = 2.236;
D (Z1 (1), X8) =4; D (Z2 (1), X8) =3; D (Z3 (1), X8) =5;

S (Z1 (1)) = {X1, X6} S (Z2 (1)) = {X2, X8} S (Z3 (1)) = {X3, X4, X5, X7}

0 0 0 0 0
Z1 (2) = ( , ) Z2 (2) = ( , ) Z3 (2) = ( , )
= (0.5, 0) = (0, 2.5) = (4.5, 4)

D(Z1 (2),Xi) D(Z2 (2) ,Xi) D(Z3 (2) ,Xi)

D (Z1 (2), X1) = 0.5; D (Z2 (2), X1) = 2.5; D (Z3 (2), X1) = 6.02;
D (Z1 (2), X2) = 1.11; D (Z2 (2), X2) = 1.15; D (Z3 (2), X2) = 5.408;
D (Z1 (2), X3) = 6.02; D (Z2 (2), X3) = 5.22; D (Z3 (2), X3) = 0.5;
D (Z1 (2), X4) = 6.72; D (Z2 (2), X4) = 5.59; D (Z3 (2), X4) = 1.118;
D (Z1 (2), X5) = 5.33; D (Z2 (2), X5) = 4.71; D (Z3 (2), X5) = 1.118;
D (Z1 (2), X6) = 0.5; D (Z2 (2), X6) = 2.69; D (Z3 (2), X6) = 5.315;
D (Z1 (2), X7) = 4.03; D (Z2 (2), X7) = 4.03; D (Z3 (2), X7) = 2.06;
D (Z1 (2), X8) = 4; D (Z2 (2), X8) =1.5; D (Z3 (2), X8) =4.5;

S (Z1 (2)) = {X1, X2 X6} S (Z2 (2)) = {X8} S (Z3 (2)) = {X3, X4, X5, X7}

Z1 (3) = (
0 0
,
0 0
) Z2 (3) = (0, 4) Z3 (3) = ( , )
= (4.5, 4)
= (0.33, 0.33)
 D(Z1 (3),Xi) D(Z2 (3) ,Xi) D(Z3 (3) ,Xi)
D (Z1 (3), X1) = 0.467; D (Z2 (3), X1) = 4; D (Z3 (3), X1) = 6.02;
D (Z1 (3), X2) = 0.747; D (Z2 (3), X2) = 3; D (Z3 (3), X2) = 5.408;
D (Z1 (3), X3) = 5.94; D (Z2 (3), X3) = 5; D (Z3 (3), X3) = 0.5;
D (Z1 (3), X4) = 6.604; D (Z2 (3), X4) = 5.099; D (Z3 (3), X4) = 1.118;
D (Z1 (3), X5) = 5.94; D (Z2 (3), X5) = 4.123; D (Z3 (3), X5) = 1.118;
D (Z1 (3), X6) = 0.747; D (Z2 (3), X6) = 4.123; D (Z3 (3), X6) = 5.315;
D (Z1 (3), X7) = 4.03; D (Z2 (3), X7) = 4.47; D (Z3 (3), X7) = 2.06;
D (Z1 (3), X8) =3.68; D (Z2 (3), X8) =0; D (Z3 (3), X8) =4.5;

S (Z1 (3)) = {X1, X2 X6} S (Z2 (3)) = {X8} S (Z3 (3)) = {X3, X4, X5, X7}

Z1 (4) = (
0 0
,
0 0
)
Z2 (4) = (0, 4) Z3 (4) = ( , )
= (4.5, 4)
= (0.33, 0.33)

Here, Z1 (3) = Z1 (4)

Z2 (3) = Z2 (4)

Z3 (3) = Z3 (4)

Hence, the algorithm has converged and the procedure is terminated.