Scribd
Upload
71 views9 pages

Probabilistic Neural Network (PNN)

Probabilistic neural networks (PNNs) are a type of neural network used for classification tasks. PNNs consist of sub-networks that estimate the probability density function for each class using Parzen window estimation. A new data point is classified by determining which class's probability density function yields the highest value for that point. PNNs find the optimal decision boundary that minimizes misclassification error by solving the equation that sets the probability density functions of the two classes equal to each other.

Uploaded by

Amjad Hussain
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
71 views9 pages

Probabilistic Neural Network (PNN)

Probabilistic neural networks (PNNs) are a type of neural network used for classification tasks. PNNs consist of sub-networks that estimate the probability density function for each class using Parzen window estimation. A new data point is classified by determining which class's probability density function yields the highest value for that point. PNNs find the optimal decision boundary that minimizes misclassification error by solving the equation that sets the probability density functions of the two classes equal to each other.

Uploaded by

Amjad Hussain
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Lecture 3

Probabilistic neural network (PNN)


Consider the problem of multi-class classifica-
tion. We are given a set of data points from
each class. The objective is to classify any new
data sample into one of the classes.

x1 ,1

x1 ,n1−1
+
x

x1 ,n1
ArgMAX

x2 ,1

x x2 ,n2−1
+

x2 ,n2

Figure above: A schematic illustration of a


PNN.
1
Lecture 3

Probabilistic neural network (PNN) is closely


related to Parzen window pdf estimator. A
PNN consists of several sub-networks, each of
which is a Parzen window pdf estimator for
each of the classes.

The input nodes are the set of measurements.

The second layer consists of the Gaussian func-


tions formed using the given set of data points
as centers.

The third layer performs an average operation


of the outputs from the second layer for each
class.

The fourth layer performs a vote, selecting the


largest value. The associated class label is
then determined.

2
Lecture 3

Suppose that for class 1, there are five data


points x1,1 = 2, x1,2 = 2.5, x1,3 = 3, x1,4 = 1
and x1,5 = 6. For class 2, there are 3 data
points x2,1 = 6, x2,2 = 6.5, x2,3 = 7. Using
the Gaussian window function with σ = 1, the
Parzen pdf for class 1 and class 2 at x are
5 " #
1 ! 1 (x1,i − x)2
y1(x) = √ exp −
5 i=1 2π 2
and
3 " #
1 ! 1 (x2,i − x)2
y2(x) = √ exp −
3 i=1 2π 2
respectively. The PNN classifies a new x by
comparing the values of y1(x) and y2(x). If
y1(x) > y2(x), then x is assigned to class 1;
Otherwise class 2.

For this example y1(3) = 0.2103 (see Lecture


2).

3
Lecture 3

0.4

0.35

0.3

0.25
y (x)
2
y (x)
0.2 1

0.15

0.1

0.05

0
−15 −10 −5 0 5 10 15

Figure above, Parzen window pdf for two classes.


$ " #
1 (6 − 3)2
y2(3) = √ exp −
3 2π 2
" #
(6.5 − 3)2
+ exp −
2
" #%
(7 − 3)2
+ exp −
2

= 0.0011 < 0.2103 = y1(x)


so the sample x = 3 will be classified as class
1 using PNN.
4
Lecture 3

The decision boundary of the PNN is given by

y1(x) = y2(x).
So
$ " # " #
1 (6 − x)2 (6.5 − x)2
√ exp − + exp −
3 2π 2 2
" #%
(7 − x)2
+ exp −
2
$ " " ##
1 (2 − x)2 (2.5 − x)2
= √ exp − + exp −
5 2π 2 2
" # " #
(3 − x)2 (1 − x)2
+ exp − + exp −
2 2
" #%
(6 − x)2
+ exp −
2
The solution of x can be found numerically.
e.g. a grid search. This is an optimal solution,
minimizing misclassification rate.

5
Lecture 3

Figure above: Decision boundary and the error


probability of PNN (the shaded area).

Moving the decision boundary to either side


would increase the misclassification probability.

6
Lecture 3

Since the term √1 is a common factor in both


2π
y1(x) and y2(x), it can be dropped out without
changing the classification result. We can use
5 " #
1 ! (x1,i − x)2
y1(x) = exp −
5 i=1 2
and
3 " #
1 ! (x2,i − x)2
y2(x) = exp −
3 i=1 2

In general, a PNN for M classes is defined as


nj " #
1 ! (∥xj,i − x∥)2
yj (x) = exp −
nj i=1 2σ 2

j = 1, ..., M
where nj denotes the number of data points
in class j. The PNN assigns x into class k
if yk (x) > yj (x), j ∈ [1, ..., M ]. ∥xj,i − x∥2 is
calculated as the sum of squares. e.g. if xj,i =
[2, 4]T , x = [3, 1]T , then
∥xj,i − x∥2 = (2 − 3)2 + (4 − 1)2 = 10
7
Lecture 3

Example 1: Determine the class label for the


data sample x = [0.5, 0.5]T by using a two class
PNN classier, with σ = 1, based on the two
class data sets given in the following Table.

x1,1 1 0
x2,1 -1 0
x1,2 0 1
x2,2 0 -1
x1,3 1 1

class 1 class 2

Solution:
" #
1 (1 − 0.5)2 + (0 − 0.5)2
y1(x) = {exp −
3 2
" #
(0 − 0.5)2 + (1 − 0.5)2
+ exp −
2
" #
(1 − 0.5)2 + (1 − 0.5)2
+ exp − }
2

= 0.7788
8
Lecture 3

" #
1 (−1 − 0.5)2 + (0 − 0.5)2
y2(x) = {exp −
2 2
" #
(0 − 0.5)2 + (−1 − 0.5)2
+ exp −
2

= 0.4724
Because y2(x) < y1(x), so x = [0.5, 0.5]T is
classified as Class 1.
2
Class 1
Class 2
1.5 Class ?

0.5

−0.5

−1

−1.5

−2
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

Figure above, Data points in Example 1.


9

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy