Pervasive computing

2024/2025

Lab 4. Classification

Aim. Assuming that you have already acquired an image, a sound or an ECG signal, after reading this
tutorial you will be able to realize pattern recognition using a very simple, rule-based classification
algorithm. Moreover, you will learn how to apply a more sophisticated machine learning algorithm,
called neural network, on an existing dataset. The material used in this tutorial is explained in Lectures
6-7-8. Classification and in the book Chapter 6.

Method. You will work in groups of three students, and will use MATLAB 2021 or higher, a Microsoft
Lifecam webcam with an embedded microphone, a Vernier ECG-BTA sensor for electrocardiogram
signals and Cricket, an inhouse data-acquisition module based on a National Instruments 6001-USB
board.
Your assignment has 3 tasks. For the first task you have to choose between Task 1 and Task 2 depending
on what you plan to do for the project. Your first two tasks are to write a MATLAB script to recognize
video/audio/ECG patterns using a rule-based classifier. For the third task you will use a MATLAB neural
network to recognize handwritten digits. You will experiment with different network hyperparameters
and evaluate the classification accuracy using a confusion matrix.

You need to study before coming to this lab. Therefore you must solve all the preparation exercises (PE)
published in this document, and hand in the answers on paper to your TA, at the start of the lab. These
answers will not be graded. However, you cannot start the practical session without handing in these
answers.
Your group has to hand in the answers to Exercise 1, 2 and 3 in Canvas. You should hand in a pdf file,
with the commented MATLAB code and the relevant figures along with a description of the parameters
you used for classification and a discussion of your results. Please include a complete listing of your
MATLAB code as an appendix at the end of your pdf file. Your answers will be graded. The deadline is on
Sunday, 24 November, 23:59.

1
Preparation steps

• The theoretical concepts necessary for this lab have been presented in Lecture 6-7-8.
Classification. Read the lecture slides and Chapter 6. Classification from the book. You will also
need the MATLAB code from Lab3.
• If you want to use your lab time in the most efficient way, you should read this tutorial in
advance. Solve the preparation exercises below, to make sure that you are well-prepared for
this lab. These answers have to be given on paper and not in MATLAB, and you have to hand
them in to your TA at the start of the lab.

Preparatory exercises

PE1. What is a rule-based classifier?

PE2. a) IMAGE What does BLOB mean? Define the terms “form factor” and “Euler number”.
a) SOUND Define the concept of frequency spectrum. Sketch a signal in time domain and its
frequency spectrum (as calculated with MATLAB's FFT function), and label the x-axis.
b) Read about ECG signals on Internet. Sketch an ECG signal in time domain. What is its period
approximately? How will this period change after an intense physical exercise?
PE3. What is an artificial neural network? Draw a neural network for automatic character recognition
where the images are black and white of the 3x3 format and we want to recognize only three letters A,
B or C.
PE4. What is a confusion matrix? Draw a possible confusion matrix for the PE3 problem and describe its
components.

2
You have to choose between Task1-Image or Task1-Sound

Task 1-Image. Pattern recognition in images using a rule-based

classifier.
In this exercise we will recognize three types of objects in an image, such as small coins, large coins and
drawing pins (Fig. 1).

Fig. 1. The objects we want to recognize.

Classes. We obviously have three classes, which are {pin, 10c and 2e}.
Features. Which features can we use to distinguish between these three classes? In order to
differentiate between the drawing pins and the coins, we can use the form factor f, defined as :
4πA
f =
P2

where A is the area and P is the perimeter, respectively. Its useful characteristic is that a circular object -
such as a coin - has a form factor equal to one, while a non-circular object, such as a drawing pin, will
have a smaller form factor.
Classifier. Knowing this information, we can use a simple rule-based classifier, with the following basic
rules:

If form factor < 1 then class = pin

If form factor == 1 then class = coin

However, in order to distinguish between the two coins, we will need an additional feature. For
example, we could use the area of the object.
This adds some more relevant rules to our classifier, such as:

If (form factor is close to 1) and (area is small) then class = 10c

If (form factor is close to 1) and (area is large) then class = 2e

Training. Now that we have the features, the classifier and its rules, we can start to train our classifier.

3
For this purpose, we will acquire individual images with each of the three objects, and we will determine
their features, form factor and area. We write these down in a table.

First, we have to acquire an image.

>> vid=videoinput('winvideo',1);
>> set(vid,'ReturnedColorSpace','rgb');
>> start(vid);
>> im=getsnapshot(vid);

Im2 im3 im6

We repeat the same steps as in Lab3, until we get a labeled image using connected components
algorithm.

>> im2 = imcrop(im,[300,200,400,400]);

>> imshow(im2);
>> im3=rgb2gray(im2);
>> imshow(im3);

>> im4=im2bw(im3,0.4);
>> imshow(im4);
>> im5=imcomplement(im4);
>> im6=imclose(im5,strel('disk',6));
>> imshow(im6);

>> [labels,numlabels]=bwlabel(im6);
>> disp(numlabels)
1

>> vislabels(labels); % display the labeled components

In this lab, we go further, and analyze the BLOB we have just obtained. MATLAB has a built-in function
that returns some geometrical properties of a singular BLOB, such as perimeter, area, Euler number, etc.
This function is called regioprops, and is designed to scan through a binary image and extract the
properties of each detected BLOB. The following commands will query all the properties of all BLOBs:

>> stats = regionprops(labels, 'all');

4
The result of this query, stats, is an array of structures, with one structure with features per BLOB.

We only have one BLOB, and we can query its properties with the following command:

>> stats = regionprops (labels,'all');

>> stats(1)

ans =

Area: 49773
Centroid: [176.4570 160.2347]
BoundingBox: [50.5000 33.5000 251 254]
SubarrayIdx: {[1x254 double] [1x251 double]}
MajorAxisLength: 253.6803
MinorAxisLength: 249.8654
Eccentricity: 0.1728
Orientation: -77.9834
ConvexHull: [236x2 double]
ConvexImage: [254x251 logical]
ConvexArea: 50155
Image: [254x251 logical]
FilledImage: [254x251 logical]
FilledArea: 49773
EulerNumber: 1
Extrema: [8x2 double]
EquivDiameter: 251.7398
Solidity: 0.9924
Extent: 0.7807
PixelIdxList: [49773x1 double]
PixelList: [49773x2 double]
Perimeter: 838.9504

We're only interested in Area: 49773, Perimeter: 838.9504.

We can now calculate the form factor as:

4πA
f =
P2

>> f=4*pi*stats(1).Area/((stats(1).Perimeter)^2)

f =

0.8887

5
You will have to repeat the same steps for a large coin (for example 2 euro) and for a drawing-pin, and
finally make a table with all their features. This table will guide you in designing the appropriate
discrimination rules for the classifier.

Area Perimeter Form factor

Small coin (10c) 49773 838.9504 0.8887
Large coin (2euros)
Drawing pin

Testing. Now you are ready to test your classifier - you can take a snapshot of an object, and ask your
classifier to decide which class this object belongs to.

Exercise 1.
a) Write a MATLAB script that can recognize objects from an image by using a rule-based classifier.
You don’t have to restrict yourself to the objects presented in this example; you could recognize
other types of objects, such as keys, pens, etc. You can also recognize characters, based on
geometrical features such as orientation, eccentricity and Euler number.
b) Write a document in which you specify which classes you used, as well as which features and
which rules you implemented in your classifier. Include the inputs you tested, and the
corresponding classifier’s decisions. Also, answer the following question: Do you see any
limitations of this method of classification?

You have to hand in your MATLAB script, as well as the document in which you describe your
classification experiment.

6
Task 1-Sound. Pattern recognition in sound using a rule-based classifier
In this task we will distinguish between two spoken vowels: a, pronounced like in ‘far’, and u,
pronounced like in ‘doomed’, according to the International Phonetic Alphabet (IPA). We will use the
frequency spectrum as the feature for classification.

We begin with the vowel a, and will record a sound waveform from the microphone. We will keep on
uttering ‘aaaah’ for 5 seconds and then cut a frame from the middle of the signal with 500 “nice”
samples. We can see in Fig. 2.1 that the signal is quite periodic.

>> Fs=8000 ;
>> rec = audiorecorder(8000,16,1);
>> recordblocking (rec, 5); <Enter>
>> y = getaudiodata (rec);
>> plot(y);
>> y=y(10000:10500);% plot a window of 500 samples
>> plot(y);

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8
0 100 200 300 400 500 600

Fig. 2.1. The time domain plot of 500 samples of the vowel a.

We calculate the frequency spectrum of this signal, by using the FFT with a Hamming window of length
500.

>> N=500;
>> f=fft(hamming(500) .* y(1:500));%compute FFT
>> m=abs(f); % the absolute value of f
>> x=(0:249)*Fs/N; ; % the frequency axis for the first half
>> plot(x(1:150),m(1:150)); %we plot only the first 150 coefficients

7
We plot this spectrum in Fig. 2.2.

35

30

25

20

15

10

0
0 500 1000 1500 2000 2500

Fig. 2.2. The frequency spectrum of the vowel a (the frequency in Hz).

And then we repeat the procedure, this time with the vowel u.
Again, we take 500 samples from the recorded waveform and we plot its time variation in Fig. 2.3.
0.4

0.3

0.2

0.1

-0.1

-0.2

-0.3
0 100 200 300 400 500 600

Fig. 2.3 The time domain variation of the vowel u.

The frequency spectrum of the vowel u is plotted in Fig. 2.4.

25

20

15

10

0
0 500 1000 1500 2000 2500
Fig.2.4. The frequency spectrum of the vowel u.
Frequency [Hz]

8
Let’s analyze these two spectra, shown in Fig. 2.5. We can see that the spectra look quite different, and
this is good news, because it means that the frequency spectrum is a good feature that can help us in
vowel recognition.
Try to identify what is specific for “a” but not for ”u”, and vice versa.

Fig. 2.5. The two spectra for “a” and “u”.

We see, for example, that the vowel “a” has the strongest peak at 1500Hz and the vowel “u” doesn’t –
instead, the vowel “u” has a strong peak at 400 Hz.

Classifier. In fact, just this information is enough to separate the two vowels. We choose to use a rule-
based classifier. This will decide which class a new vowel belongs to by applying the following
discrimination rules:

If the frequency of the largest peak in the spectrum is around 1500 Hz then the vowel is an “a”.

If this frequency is around 400Hz then it is an “u”.

Exercise 1. Write a MATLAB script that acquires a vowel and recognizes it based on the frequency
spectrum and a rule-based classifier. Write a document in which you describe your experiment: which
features you used, which inputs you tested, and what results you obtained. Also, answer the following
question: Do you see any limitations of this method of classification?

You have to hand in your MATLAB script, as well as the document in which you describe your
classification experiment.

9
Task-2-ECG. Physical activity recognition using ECG signals

In this experiment you will be analyze physiological signals to distinguish between different physical
activities. The physiological signal we will use is called ECG (electrocardiogram). An electrocardiogram is
a graphic tracing of the heart’s electrical activity. The principle is simple: the electrical activity from the
muscular contractions of the heart spreads throughout the body and can be detected on the surface of
our skin [1]. A “normal” ECG signal is periodic and varies in time like illustrated in the figure below. You
can see that prominent in an ECG pattern is the so-called QRS complex characterized by an R-peak.

You will measure live this signal, using a Vernier ECG sensor [2]. The output of the sensor is an analog
voltage that will be digitized using a National Instruments 6001-USB data acquisition board. You can
read in its datasheet [3] that it is a multifunctional I/O device with a maximum sampling rate of 20 KHz, a
resolution of 14 bits, 8 Inputs, 2 Outputs, in the range of ± 10 V.

In order to do this, you have to first build up the setup like shown in Fig. 2-1. Identify first the Vernier
sensor, which has 3 lead wires ending with a clamp connector, one black (ground), one red(+) and one
green (-).

10
Fig. 2-1. The ECG measurement chain used in this lab, consisting of an ECG sensor, the Cricket module
and a laptop.

Identify next the VU-Cricket, an intermediary module 1 built inhouse at the VU, that allows to measure
the signals coming from the Vernier ECG sensor using a laptop. The module is built around a National
Instruments NI-6001 USB data acquisition board, and contains also a few BTA and BNC connectors, a 5V
power supply for the sensor, a LED, a buzzer and a switch (se Fig. 2-2). The board accepts maximum four
analog input signals. Our experiment will use the first channel, labelled CH0 in the figure below. The
input signal can be connected to a BTA connector or a BNC connector. The input is available at the CH-
OUT BNC out connector to be visualized if needed using an oscilloscope.

Fig. 3-2. The VU-Cricket breakout board.

You can now plug in the white BTA connector from the Vernier ECG sensor into the BTA connector
labeled CH0-IN BTA on the Cricket board.

1
The Cricket data acquisition module was designed and built in 2023 by Niels Althuisius and Mario Molenaar from
VU Beta Electronica. Thanks to FMI workshop and Samuel Jonathan for their help, and all students and TAs from
November 2023 for testing it.

11
Next, you have to attach three disposable ECG electrodes (sticky patches) on the volunteer’s skin.

• You should use only new electrodes.

• Remove any watches and/or jewelry from the volunteer's wrists and ankles.

• Dead skin cells are not good conductors of electrical impulses. [Optional: Wash your skin with
abrasive dry soap]. Use some cotton pads and alcohol to clean your skin.

• There are many way you can attach the three electrodes. We recommend the following
configuration as shown in the figure below. The positive electrode should go to the left wrist, the
negative electrode to the right wrist, and the ground electrode to the right leg.

Next, hook the ECG sensor wires to the three corresponding electrode patches like shown in Fig. 2-3.

Fig. 2-3. The placement configuration for the ECG electrodes.

The last thing you have to do is connect the USB cable from the VU-Cricket module to the USB port of
your laptop. You should see a blue LED turning on the data acquisition board and three yellow and green
LEDS on the Cricket board. Now your laptop is connected to the patient and the measurement chain is
ready.

Start MATLAB 2021b. MATLAB does not know yet how to work with your data acquisition board. In
order to do this, you need to install some toolboxes. Go to Get Hardware Support Packages.
12
Search for National Instruments NI-DAQ.

Find and install the NI-DAQmx toolbox.

Now MATLAB should recognize your NI data acquisition board and communication will be possible.

Type in MATLAB:

>> d=daqlist("ni")
d =

1×4 table

DeviceID Description Model DeviceInfo

________ _________________________________________ ________________

_______________________

"Dev6" "National Instruments(TM) USB-6001 (OEM)" "USB-6001 (OEM)" [1×1

daq.ni.DeviceInfo]

13
We can see that the device is connected to the USB port called Dev6.

Now we can create an object dq connected to the data acquisition board, and set its sampling rate to
say 10000 Hz.

>> dq=daq("ni");

>> dq.Rate=10000;

We now program the AI0, first analog input port for measuring the voltage coming from the ECG sensor
and connect it in software to port Dev6.

>> addinput(dq,"Dev6","ai0,"Voltage");

We are ready to get data from the NI board. Let us record data for 2 seconds and immediately plot the
data against time.

The person has to sit comfortably and try not to move during the measurements.

>> data=read(dq, seconds(2));

Data is a table type of variable in MATLAB, containing the time vector stored in data.Time and the raw
voltage samples in data.Variables. We store them in separate vectors, ecg_vector and t.

>> ecg_vector= data.Variables ;

>> t = data.Time ;

>> plot(t,ecg_vector);

>> grid on;

>> xlabel("Time [s]");

You should obtain a plot similar with the one shown below.

14
Exercise 3. Acquire a live ECG signal and process it like you did in Lab3. Show the plot of the original
signal in time domain and its frequency spectrum and after each processing step. When the signal looks
good, write a MATLAB code for a rule-based classifier that recognizes three activities, for example
relaxed, a light exercise and a more intensive physical exercise, based on this ECG signal. Instead of a
physical exercise you can try a breathing exercise. Describe your experiment. Which feature did you use
to discriminate between the activities?

Bonus. Create a GUI in MATLAB for a heart monitor that continuously plots the ECG signal in time
domain and displays the heart rate. To make it more realistic, you can use the buzzer on the Cricket
board that is connected to digital output port 7 and the LED connected to the digital output port 6, to
make a sound and generate a light pulse for each R-peak. Hint: You will need to investigate how the
function addoutput() works.

Disclaimer. We are educators and not medical staff so we do not know how to correctly interpret an
ECG. If anyone encounters something on an ECG that appears like it might be suspicious or not normal,
the person should see a physician and have another ECG run under proper clinic or hospital diagnostic
conditions and read by someone who is fully trained in ECG interpretation

Troubleshooting.
• Allow the electrode tabs to stabilize with the skin of the subject for at least 2 minutes before
recording. Verify that the clips are firmly attached to the tabs of the electrodes.
• Electrode tabs should be fresh and can be used only once. Dry, old, or used electrode tabs will
be problematic.
• Make sure the subject does not move during the recording. For best results, make sure the
subject is sitting when recording ECGs.

15
Task 3-NN. Using an artificial neural network (NN) to recognize handwritten digits2
1. Network topology
The problem that you have to solve here is automatic handwritten digit recognition. We have 10 classes
labeled with ”1”, “2”, ”3”, … ”9”, ”0”. You already know how to solve this classification problem using a
rule-based classifier and some geometric features such as Euler number, orientation, etc. However,
since different people have different handwriting, the algorithm might not work with very high accuracy
in all cases. A solution can be to use a more complex classification algorithm, known to have a high
generalization power, called neural network (NN). In this task, we will be using a simple network called
multilayer perceptron, with one input layer, one output layer and in between a few hidden layers. Below
you can see our NN with only one hidden layer.

2
This part of the tutorial is based on the work of Sifra ter Wee, one of our TAs and on the blog by Loren Shure and
the user manual on the Mathworks site [6,7].

16
In the figure above, the neural network is not completely specified. We do not know for example how
many input nodes and how many output nodes one needs.
The number of input neurons depends on the dataset used. For this lab, we will use the MNIST data set
[1]. MNIST ("Modified National Institute of Standards and Technology") is the de facto “hello world”
dataset of computer vision. Since its release in 1999, this classic dataset of handwritten images has
served as the basis for benchmarking classification algorithms. Kaggle is using it for a worldwide
competition [2]. Each image in this database is a matrix of 28 x 28 pixels. Below you can see an excerpt
from the MNIST dataset.

As inputs to the NN we can use different features of these images, such as the Euler number,
orientation, eccentricity, or the seven invariant moments of Hu, but we can also use the whole gray-
scale image as input. As each sample in the dataset is a gray scale image, the pixel colours are numbers
between 0 and 255. If we unfold this 28x28 image matrix into a one dimensional array or vector to be
fed at the input of the NN, we obtain 28x28 = 784 elements. This means that our neural network will
have 784 input neurons.

The number of output neurons is equal to the number of classes we have, in our case 10. We need to
setup a convention on how to interpret the output data coming from the neural network. We can use
the following convention, illustrated in the table below. An output 1000000000 means that the
predicted class is ”1”. An output equal to 0100000000 means the predicted class is ”2”, etc, and an
output equal to 0000000001 indicates the class ”0”.

17
 Predicted “1” “2” “3” “4” “5” “6” “7” “8” “9” “0”
Class
NN
Output
O1 1 0 0 0 0 .. 0
O2 0 1 0 0 0 .. 0
O3 0 0 1 0 0 .. 0
O4 0 0 0 1 0 .. 0
,,, .. … … .. … .. .. ..
Q10 0 0 0 0 0 0 0 0 0 1

We also have to decide the activation function. We can use for example first a sigmoid function.

2. Loading the MNIST dataset into MATLAB

First, we need to import the MNIST data into MATLAB. Make a new folder for this assignment. The
starting point are two MNIST data files, train.csv and test.csv containing gray-scale images of
hand-drawn digits, from zero through nine. We downloaded them from the kaggle.com digit recognizer
competition site [2] and posted them on Canvas in the folder Practical.
You can find the archived csv files in Canvas. Download them and extract the csv files in your directory.
You should see two excel files, test.csv and train.csv. The files are structured as follows. Each sample is a
row. Each row has 785 elements, where the first element is the sample’s label (a number from 0 to 9)
and the rest of 784 elements contain the colours of the pixels of the digit’s image (a number from 0 to
255). The file train.csv contains 42.000 images together with their labels. The file test.csv contains
20.000 images and their labels.
We will use only the first dataset called train.csv.
You have to load these two files into two arrays in MATLAB. We named the files MNIST_train for training
the NN and MNIST_test for testing it.

>> MNIST_train=csvread('train.csv',1, 0);

>> MNIST_test = csvread('test.csv', 1, 0);

Let’s look more carefully at the dataset we have just imported, represented by the matrix MNIST_train.
In the MATLAB Wokspace window you can see that it is a 42000 x 785 array of double integers. This

18
means that it has 42.000 rows with each row containing a data sample, consisting of a label and the
corresponding 784 pixel values of an image.

Let us for example visualize the first image in this data set, which is stored in the first row of this matrix.
The pixels of the image can be found in positions 2 to 785. To visualize the image we have to reshape it
first from a 784 long vector to a 28x28 matrix, in fact a grayscale image that can be displayed as we
already know, using the function imshow().

>> digit = reshape (MNIST_train(1, 2:end), [28,28])’; % reshapes the image vector from position 2 to the
end, into a 28x28 matrix and transposes it
>> imshow (digit); % display digit, a gray scale image

You can see that the first image in the dataset is the hand-written digit “1”.

We can also display its label, given by the first element of the first row MNIST_train (1,1).
>> num2str(MNIST_train(1,1))

ans =

'1'

19
>>
If you want to see the second item, then you can use this code to visualize the image.

>> digit = reshape (MNIST_train(2, 2:end), [28,28])';

>> imshow(digit);
>>
Its label is according to MATLAB “0”, which is what we already expected.

>> num2str(MNIST_train(2,1))

ans =

'0'

>>

Now that we have control over our training data imported in MATLAB, we can process it and use it to
train our NN.

3. Preparing the training data for the neural network

We have at this moment in MATLAB MNIST_train, a matrix containing training data. Each row in this
matrix is a training sample, which is a 785 long vector. It consists of a label in the first position and the
image pixels, stored in positions 2 to 785. The first element is the correct output of the NN and the rest
of the elements represent the corresponding inputs in the NN. Below you see one row from the training
matrix.

20
From the MATLAB tutorial for neural networks [4] that can be found in Canvas, we discover that a neural
network in MATLAB needs two matrices, one containing the input samples and one with the
corresponding correct outputs.
The input matrix has as many columns as samples. The number of rows is the number of input features
which is in our case 784, the number of pixels in an image. If we use all the samples, we will have 42.000
columns (see figure below).

Also, NN needs a target matrix containing the correct outputs. This matrix has as many rows as classes,
10 in our case, and as many columns as samples. If we use all the samples in MNIST_train, we have
42.000 columns (see figure).

21
What we have to do now is to split the training matrix MNIST_train in two – the labels for the outputs
and images for the inputs. After that, we have to transpose the newly created matrices (inverse rows
with columns) to fit into the NN conventions.

22
We will make two matrices in MATLAB. One is called targets, containing the correct output vectors. The
contents of each vector depends on the label. For example if the label is 1, then the output vector is as
follows:

1
0
0
0
0
0
0
0
0
0

A special case is label 0, that we will change to 10, in order to follow the same mapping algorithm. The
output vector corresponding to label 0 is then:
0
0
0
0
0
0
0
0
0
1
Now we will transform the first column containing the labels, in a matrix that contains the output
of the NN for all samples. Each row in this matrix is the correct NN output for an input image
sample. For this purpose we use the function dummyvar () from the Statistics and machine
learning toolbox.

23
 Label 1 2 3 4 5 6 7 8 9 0
Output
O1 1 0 0 0 0 .. 0
O2 0 1 0 0 0 .. 0
O3 0 0 1 0 0 .. 0
O4 0 0 0 1 0 .. 0
,,, .. … … .. … .. .. ..
Q10 0 0 0 0 0 0 0 0 0 1

The first column in MNIST_train matrix containing all the labels

24
The matrix targets, containing all the corresponding NN outputs with 10 columns, before
transpose operation

labels=MNIST_train(:,1); % this is the first column containing the

labels
labels(labels==0)=10; % change the label 0 to 10
targets=dummyvar(labels); make the matrix targets

The inputs matrix is a matrix containing on each row the corresponding images for each training sample,
in fact the rest of the train matrix.

inputs = MNIST_train (:,2:end); % contains all the image pixels

Now the last thing we have to do is to transpose the inputs and targets matrices to fit the MATLAB NN
conventions.

inputs = inputs'; % transpose input matrix

targets = targets'; % transpose the target matrix

25
Now we have all the data prepared to train the neural network in MATLAB.
We can check for the last time that all the training matrices have the correct sizes. In the Workspace you
can see the matrix inputs, with 784 rows and 42.000 columns and the matrix targets with 10 rows and
42.000 columns.

4. Training the MATLAB neural network

Type the following commands in MATLAB command line to create a new neural net with 10 hidden
layers and visualize it.

>> net = patternnet (10); % one layer with 10 hidden neurons

>> view(net);

We as you see there is nothing yet at their inputs and output, so the neural network knows nothing as it
is not trained. With the following commands we will tell the net where to search for training data, with
which inputs and which outputs.

26
This command will start training and visualizing the network’s progress.

>> net = train (net, inputs, targets);

You can click on Performance during the training to see how the error decreases in time.
When the training is finished, we can plot the final classification performance by clicking on
Performance.

27
You can see that the training took 212 iterations or epochs and that it stopped when the error dropped
under a certain threshold.
Now let us check whether the trained neural network works. We take the first image from the training
set for example, which we already know that is an image of the digit ”1” and we ask the net which class
it belongs to.

>> test_input = MNIST_train(1, 2:end); % take the first image in the training set
>> test_input = test_input'; % transpose it
>> test_output = net (test_input) % ask the net which class it belongs to

test_output =

0.9830
0.0011
0.0001
0.0001
0.0002
0.0002

28
 0.0000
0.0153
0.0000
0.0000

>>
You can see that the output given by the neural net is a vector with 10 elements, O1 to O10, where the
first element clearly is higher than the rest, approaching 1, while the rest is close to zero. According to
our convention this means predicted class ”1”, which is correct.

This was just a test with one input. It is interesting now to perform a large classification experiment on
the whole training set and plot the confusion matrix. This will say more about the accuracy of our
experiment.

>> predicted = net (inputs);

These are all the predicted classes for all the training set.

>> plotconfusion(targets,predicted);

This command will plot a confusion matrix.

29
On the confusion matrix plot, the rows correspond to the predicted class (Output Class) and the columns
correspond to the true class (Target Class). The diagonal cells correspond to observations that are
correctly classified. The off-diagonal cells correspond to incorrectly classified observations. Both the
number of observations and the percentage of the total number of observations are shown in each cell.

The column on the far right of the plot shows the percentages of all the examples predicted to belong to
each class that are correctly and incorrectly classified. These metrics are often called the precision (or
positive predictive value) and false discovery rate, respectively. The row at the bottom of the plot shows
the percentages of all the examples belonging to each class that are correctly and incorrectly classified.
These metrics are often called the recall (or true positive rate) and false negative rate, respectively. The
cell in the bottom right of the plot shows the overall accuracy.

If we look at our confusion matrix, we see that the overall accuracy is 92.9%. If we start with the target
(true) class 1, we can say for example that digit 1 is quite accurately classified as 1, in 97.2% of total
cases. Digit “2” is less accurately classified, 92.9% only. For example it is quite often classified as 3 and 8.
Digit “3” is 90.3% accurately classified, it is often wrongly classified as 2, 5 and 8.

Note: However, probably you were wondering why are the results so good. This happens because we
used as test set the training set. If you curious you could run the experiment again using the test set
imported from MNIST at the beginning and see how the results change.

5. Experiments

Now that you know how NN works, we will perform some more experiments. We can run experiments
faster by using the nprtool pattern recognition app.

In MATLAB, go to APPS and then select Neural Net Pattern Recognition.

30
Click Next.

Now you will get a screen where you have specify the matrix to use as input and the matrix to use as
output. We can tell the NN that the two matrices it needs are called inputs and targets and that the
samples are columns.

31
Next we can decide how much of the training set we use for training, for validation and testing.

32
Now we have to decide how many hidden layers we have. We start with 10.

33
Let’s take 10 hidden layers and train the network. The training will stop when the error does not
improve anymore. You can visualize the performance to see how the error reduces.

34
Click Plot Confusion to see all kinds of confusion matrices that can say something about how accurate
the experiment worked. Take a look at the all confusion matrix.

35
Exercise 3.

a) Run an experiment to investigate the effect of the number of hidden layers on the
classification accuracy. Change the number of hidden layers to a low number, for
example 1, and describe what happens. Next, change the number of hidden layers to a
high number, such as 100 and describe what happens. Show the confusion matrix for
each number of hidden layers.

b) Run an experiment to investigate the effect of the size of training data set on the
classification accuracy. For example, take just a few samples for training and describe
what happens. Make a small graph showing the accuracy vs training set size. Describe
what you discovered. Is the graph linear? Can you notice a saturation trend? At what size
did you notice almost no improvement in accuracy? Add also the MATLAB code you
used. Show the confusion matrix for each size of the training set.

References.
1. https://www.ndsu.edu/pubweb/~grier/eheart.html
2. https://www.vernier.com/manuals/ekg-bta/
3. https://www.ni.com/nl-nl/shop/model/usb-6001.html
4. https://www.adinstruments.com/support/documentation/settings-sampling-panels-lt-ecg-and-
heart-rate
5. https://en.wikipedia.org/wiki/MNIST_database
6. https://www.kaggle.com/c/digit-recognizer
7. Loren Shure, https://blogs.mathworks.com/loren/2015/08/04/artificial-neural-networks-for-
beginners/
8. MATLAB Neural Networks Toolbox, User Manual

36