CS 4104

APPLIED MACHINE LEARNING

Dr. Hashim Yasin

National University of Computer
and Emerging Sciences,
Faisalabad, Pakistan.
 CONVOLUTIONAL
NEURAL NETWORK
 Why CNN
3

Image Classification

Cat? (0/1)

Object detection
64x64x3

Text
detection

16 x 16 = 256

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Why CNN
4

x1 256 inputs
x2
Total weights = W = 256 x 1000
…
…

x256 𝑥1
16 x 16 = 256
𝑥2
Ink → 1
No ink → 0 ⋮ ⋮ ⋮ 𝑦ො
𝑥𝑛
1000 1000
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Why CNN
5

Image Classification

Cat? (0/1)

64x64x3

12,288 1000x1000x3 = 3 million

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Why CNN
6

Image Classification
3M inputs

𝑥1
𝑥2
⋮ ⋮ ⋮ 𝑦ො
𝑥𝑛
1000x1000x3 1000 1000
= 3 million
Total weights = W = 3M x 1000

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 CNN
7

 Neural Networks that use convolution in place of

general matrix multiplication in at least one layer

 There are three types of layers in the convolutional

network,

❑ Convolution layer (Conv)

❑ Pooling layer (Pool)
❑ Fully connected layer (FC)

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Cross-correlation
8

❑ Let 𝑓 be the image,

❑ 𝑤 be the kernel of size 𝑚 × 𝑛
o where 𝑚 = 2𝑎 + 1 and 𝑛 = 2𝑏 + 1), 𝑎 and 𝑏 are
the positive integers.
❑ g be the output image
a b
g ( x, y ) =   w(s, t ) f ( x + s, y + t )
s = − at = − b
This is called a cross-correlation operation:
g=𝑤⊗𝑓
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Cross-correlation
9

a b
g ( x, y ) =   w(s, t ) f ( x + s, y + t )
s = − at = − b
At any point (𝑥, 𝑦), the response g(𝑥, 𝑦) of the
filter is the sum of product of filter coefficient
and the image pixels
g ( x, y ) = w(−1,−1) f ( x − 1, y − 1) +
w(−1,0) f ( x − 1, y ) + ...
w(0,0) f ( x, y ) + ...
w(1,1) f ( x + 1, y + 1)
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 g ( x, y ) = w(−1,−1) f ( x − 1, y − 1) +
w(−1,0) f ( x − 1, y ) + ...
Cross-correlation w(0,0) f ( x, y ) + ...
10 w(1,1) f ( x + 1, y + 1)

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Convolution
11

 Same as cross-correlation, except that the kernel is

“flipped” (horizontally and vertically)
a b
g ( x, y ) =   w(s, t ) f ( x − s, y − t )
s = − at = − b

This is called a convolution operation:

g=𝑤∗𝑓

 Convolution is commutative and associative

Dr. Hashim Yasin Applied Machine Learning (CS4104)
 2D Spatial filtering
12

Input image f(x,y) Output image g(x,y)

(x,y) (x,y)


Image point
Filter mask point

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 2D Spatial filtering
13

10 11 10 0 0 1 X X X X X X

9 10 11 1 0 1 X 10 X

10 9 10 0 2 1 X X

11 10 9 9 11 X X
10
9 10 11 9 99 11 H X X

10 9 9 11 10 10 1 X X X X X X
1 1

F 1 1 1 G
1/9
1 1 1

1/9.(10x1 + 11x1 + 10x1 + 9x1 + 10x1 + 11x1 + 10x1 + 9x1 + 10x1) =

1/9.( 90) = 10
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Convolution
14

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Convolution Examples-Mean filtering
15

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

1
1
1
1
1
1
*
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

𝑤 ∗ 𝑓 = g
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Convolution Examples-Mean filtering
16

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

1
1
1
1
1
1
*
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

𝑤 ∗ 𝑓 = g
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Convolution Examples-Mean filtering
17

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

1
1
1
1
1
1
*
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

𝑤 ∗ 𝑓 = g
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Convolution Examples-Mean filtering
18

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

1
1
1
1
1
1
*
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

𝑤 ∗ 𝑓 = g
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Linear filters: examples
19

0 0 0

=
* 0
0
1
0
0
0

Original Identical image

Source: D. Lowe

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Linear filters: examples
20

0 0 0

=
* 1
0
0
0
0
0

Original Shifted left

By 1 pixel

Source: D. Lowe

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Smoothing Spatial Filters
21

Box Filter Weighted Average

the center is the most important and other

pixels are inversely weighted as a function of
their distance from the center of the mask
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Linear filters: examples
22

1 1 1

=
* 1
1
1
1
1
1

Original Blur (with a mean filter)

Source: D. Lowe

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Smoothing Filters
23

 (a) is the original image of size

500x500 pixel
 (b)-(f) results of smoothing with
square averaging filter masks of
size n = 3, 5, 9, 15 and 35,
respectively.
 Note:
 The big mask is used to eliminate small
objects from an image.
 The size of the mask establishes the
relative size of the objects that will be
blended with the background.

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Gradient Operator
24

 The first derivatives are implemented using the magnitude of

the gradient f
 
Gx   x 
f =   =  f 
G y   
 y 
1
M ( x, y ) = mag (f ) = [G + G ]
2
x
2
y
2

1
 f   f  
2 2 2

=   +   
 x   y  
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 Gradient Operator
25

1
M ( x, y ) = mag (f ) = [G + G ]
2
x
2
y
2

1
 f   f  
2 2 2

=   +   
 x   y  
commonly approx.

the magnitude
becomes nonlinear M ( x, y )  G x + G y

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 z1 z2 z3
Sobel Operators z4 z5 z6
26
z7 z8 z9
 Sobel operators, 3x3
Gx = ( z7 + 2 z8 + z9 ) − ( z1 + 2 z2 + z3 )
Gy = ( z3 + 2 z6 + z9 ) − ( z1 + 2 z4 + z7 )

M ( x, y )  G x + G y
the weight value 2 is to achieve
smoothing by giving more
important to the center point

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Sobel Operators
27

 The summation of coefficients in all masks equals 0,

indicating that they would give a response of 0 in
an area of constant gray level.

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Image Derivatives
28

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 CNN … Convolution Layer
29

Edge detection in this Features detection in Faces detection in

layer this layer this layer

vertical edges

horizontal edges
Dr. Hashim Yasin Applied Machine Learning (CS4104)
 CNN … Example
30

Dr. Hashim Yasin Applied Machine Learning (CS4104)

 Acknowledgements
31

Stuart J. Russell and Peter Norvig, Tom M.

Mitchell, Jiwon Jeong, Floydhub, Andrej
Karpathy

Dr. Hashim Yasin Applied Machine Learning (CS4104)

32

Dr. Hashim Yasin Applied Machine Learning (CS4104)