Tin

C 514 - Com r Vi on E D vi
Ap i 25, 2024

Dr. Fel da,

As i t Pr e s Com rS i c
https://uojai.github.io/tinyml
C
s
r
S
s
l
a
i
n
x
t
G
o
y
n
o
f
s
M
o
p
r
u
t
o
f
e
L
p
s
u
i
o
t
n
e
c
e
n
d
e
g
e
e
c
e
s
Exam Prep 1
Dr. Felix Gonda
 Convolutional Neural Network

Cat

Dog

Tiger

Cow

Lion

64 x 64 x 3
 Convolution Layer
The convolution layer (CONV) uses lters that perform convolution operations as it is scanning the input I with respect to its dimensions. Its
hyperparameters include the lter size and stride . The resulting output is called feature map or activation map.

F F F F

I I I I
Step 1 Step 2 Step 3 Step 4

F F F

I I
Step 5 Step 6 Last Step
𝑂
𝑂
𝑂
𝑂
𝑂
𝑂
𝑂
fi
fi
𝐹
𝑆
𝑂
 Max Pooling Layer
Each pooling operation selects the maximum value of the current view

Step 1 Step 2 Step 3 Step 4 Step 5

Step 6 Step 7 Step 8 Step 9

 Fully Connected Layer
The fully connected layer (FC) operates on a attened input where each input is connected to all neurons. If present, FC layers are usually
found towards the end of CNN architectures and can be used to optimize objectives such as class scores.

fl
 Q1

Convolutions
What is the dimension of the

=
feature map produced by the
following convolution operation?
Draw the output feature map and
write the width and height of the
feature map below the gure.
• Stride = 1
• Padding = 0
• Filter:

1 1 0
Input Output
1 1 1 ( 5 x 5) ( ____ x ____ )
1 0 1

LEGEND: Input Filter Output

fi
 A1

Convolution
What is the dimension of the

=
feature map produced by the
following convolution operation?
Draw the output feature map and
write the width and height of the
feature map below the gure.
• Stride = 1
• Padding = 0
• Filter:

1 1 0
Input Output
1 1 1 ( 5 x 5) (3x3)
1 0 1

LEGEND: Input Filter Output

fi
 Q2
5 2 1 1 0
Convolution
Compute the value of the rst cell 6 0 2 4 20
of the output feature map given
the convolution lter below:
2 1 4 1 9
• Stride = 1
• Padding = 0
2 6 3 2 4
• Filter:

1 1 0 3 0 6 1 2
1 1 1
Input Output
1 0 1
( 5 x 5) (3x3)

LEGEND: Input Filter Output

fi
fi
 = (5x1) + (2x1) + (1x0) +
(6x1) + (0x1) + (2x1) +
(2x1) + (1x0) + (4x1)
= 21
A2
5 2 1 1 0
Convolution 21

Compute the value of the rst cell 6 0 2 4 20

of the output feature map given
the convolution lter below:
2 1 4 1 9
• Stride = 1
• Padding = 0
2 6 3 2 4
• Filter:

1 1 0 3 0 6 1 2
1 1 1
Input Output
1 0 1
( 5 x 5) (3x3)

LEGEND: Input Filter Output

fi
fi
 Q3

Convolution
Draw the padded input using the
convolution settings below? Also

=
specify the dimension of the new
input?

Convolution Settings:
• Stride = 1
• Padding = 1

Input
( 5 x 5)
New Input
( ____ x ____)

LEGEND: Input (5 x 5) Padding

 A3

Convolution
Draw the padded input using the
convolution settings below? Also

=
specify the dimension of the new
input?

Convolution Settings:
• Stride = 1
• Padding = 1

Input
( 5 x 5)
New Input
( ____ x ____)

LEGEND: Input (5 x 5) Padding

 Q4

Convolutions
What is the dimension of the

=
feature map produced by the
following convolution operation?
Draw the output feature map and
write the width and height of the
feature map below the gure.
• Stride = 2
• Padding = 0
• Filter:

1 1 0

1 1 1 Input Output
1 0 1
( 5 x 5) ( ____ x ____ )

LEGEND: Input Filter Output

fi
 A4

Convolutions
What is the dimension of the

=
feature map produced by the
following convolution operation?
Draw the output feature map and
write the width and height of the
feature map below the gure.
• Stride = 2
• Padding = 0
• Filter:

1 1 0

1 1 1 Input Output
1 0 1
( 5 x 5) (2x2)

LEGEND: Input Filter Output

fi
 Q5

Convolutions
What is the dimension of the
feature map produced by the
following convolution operation?
Draw the output feature map and
specify the width and height of
=
the feature map.
• Stride = 1 Filter
Output
( 3 x 3 x 3)
• Padding = 0 ( ____ x ____ )
Input
( 6 x 6 x 3)
 A5

Convolutions
What is the dimension of the
feature map produced by the
following convolution operation?
Draw the output feature map and
specify the width and height of
=
the feature map.
• Stride = 1 Filter
Output
( 3 x 3 x 3)
• Padding = 0 (4x4)
Input
( 6 x 6 x 3)
 Q6
=
Convolutions Filter Feature Map 1
What is the dimension of the ( 3 x 3 x 3)
(4x4)
feature map produced by the
following convolution operation?
Draw the output feature map and
specify the width and height of

=
the feature map.
• Stride = 1 Output
• Padding = 0 (__ x__ x__)

Input Filter Feature Map 2

( 6 x 6 x 3) ( 3 x 3 x 3)
(4x4)
 A6
=
Convolutions Filter Feature Map 1
What is the dimension of the ( 3 x 3 x 3)
(4x4)
feature map produced by the
following convolution operation?
Draw the output feature map and
specify the width and height of

=
the feature map.
• Stride = 1 Output
• Padding = 0 (4x4x2)
Input Filter Feature Map 2
( 6 x 6 x 3) ( 3 x 3 x 3)
(4x4)
 Q7

Deconvolutions
What is the dimension of the feature
map produced by the following
deconvolution operation? Draw the
output feature map and write the
width and height of the feature map
below the gure.
=
• Stride = 1
• Padding = 0
• Filter:

1 1 0

1 1 1 Input Output
1 0 1
( 3 x 3) ( ____ x ____ )

LEGEND: Input Filter Output

fi
 A7

Deconvolutions
What is the dimension of the feature
map produced by the following
deconvolution operation? Draw the
output feature map and write the
width and height of the feature map
below the gure.
=
• Stride = 1
• Padding = 0
• Filter:

1 1 0

1 1 1 Input Output
1 0 1
( 3 x 3) (5x5)

LEGEND: Input Filter Output

fi
 Q8
0 0 0 0 0 0 0
Deconvolutions
0 0 0 0 0 0 0
Compute the value of the rst cell
of the output feature map 0 0 22 1 5 0 0
produced by the application of
the following deconvolution lter
to the input image:
0 0 33 7 3 0 0
=
0 0 5 10 9 0 0
• Stride = 1
• Padding = 0 0 0 0 0 0 0 0

• Filter:
0 0 0 0 0 0 0
1 1 0

1 1 1 Input Output
1 0 1
( 3 x 3) (5x5)

LEGEND: Input Filter Output

fi
fi
 = (0x1) + (0x1) + (0x0) +
(1x1) + (5x1) + (0x1) +
(7x1) + (3x0) + (0x1)
= 13
A8
0 0 0 0 0 0 0
Deconvolutions
0 0 0 0 0 0 0
Compute the value of the rst cell
13
of the output feature map 0 0 22 1 5 0 0
produced by the application of
the following deconvolution lter
to the input image:
0 0 33 7 3 0 0
=
0 0 5 10 9 0 0
• Stride = 1
• Padding = 0 0 0 0 0 0 0 0

• Filter:
0 0 0 0 0 0 0
1 1 0

1 1 1 Input Output
1 0 1
( 3 x 3) (5x5)

LEGEND: Input Filter Output

fi
fi
 Q9
Convolution Layer

Convolution
Layer
Computer the number of
parameters and memory size of
the convolution layer depicted in
the following.

Not o s
ci : nu r i p h el
co : nu r o p ha l
wi : in w h
hi : in h i t
wo : o t wi h
ho : o t he t
k: l e s Parameters = _________ Memory = _________
Par er = (ci x k + 1) x o
Mem = (wi x i x o)
fi
a
a
o
m
t
t
u
u
p
r
p
m
i
m
e
u
y
r
u
p
p
t
b
n
t
b
t
u
u
e
e
t
t
i
z
s
e
e
i
o
d
o
g
f
t
f
d
i
h
g
t
n
h
u
h
t
u
t
u
k
t
c
c
x
c
a
n
n
n
n
s
e
s
c
 A9
Convolution Layer

Convolution
Layer
Computer the number of
parameters and memory size of
the convolution layer depicted in
the following.

Not o s
ci : nu r i p h el
co : nu r o p ha l
wi : in w h
hi : in h i t
wo : o t wi h
ho : o t he t
k: l e s Parameters = (3 x 4 x 4 + 1) x 2 = 98 Memory = (6 x 6 x 2 ) = 72
Par er = (ci x k + 1) x o
Mem = (wi x i x o)
fi
a
a
o
m
t
t
u
u
p
r
p
m
i
m
e
u
y
r
u
p
p
t
b
n
t
b
t
u
u
e
e
t
t
i
z
s
e
e
i
o
d
o
g
f
t
f
d
i
h
g
t
n
h
u
h
t
u
t
u
k
t
c
c
x
c
a
n
n
n
n
s
e
s
c
 Q10

MaxPooling
Layer MaxPooling
Compute the output of the • Stride = 2
• Filter size = (2 x 2)
following max pooling operation.

Output
( 3 x 3 x 3)

Input
( 6 x 6 x 3)
 A10

MaxPooling
Layer MaxPooling
Compute the output of the • Stride = 2
• Filter size = (2 x 2)
following max pooling operation. 9 5 8

6 5 8

8 9 8

Output
( 3 x 3 x 3)

Input
( 6 x 6 x 3)
 Q11

Relu Activation
Layer
Compute the output of the
Relu Activation
following recti ed linear
activation operation.

Input
Output
( 4 x 4 x 3)
( 4 x 4 x 3)
fi
 A11

Relu Activation
Layer
Compute the output of the 1 0 2 0
following recti ed linear
activation operation. 0 0 4 3

0 0 0 0

0 4 4 2

Input
Output
( 4 x 4 x 3)
( 4 x 4 x 3)
fi