Ecole Nationale D'informatique et

d'Analyse des Systèmes

Iot Project
Real-time Smart parking system based on
Deep Learning

Created by: Supervised by:

- AIT HADDOU Mohammed - Dr. ABDELLATIF Kobbane
- MELLOUKI Abderrahim - Dr. ELKADMIRI Omar
- LBAZRI Soufiane
- HEIBALLA Abdellahi

Année universitaire 2019/2020

 Acknowledgment IoT Report

Acknowledgment
I would like to express my deepest appreciation to all those who
provided me the possibility to complete this report. A special
gratitude I give to our Professor and Supervisor Dr. ABDELLATIF
Kobbane, whose contribution in stimulating suggestions and
encouragement, helped me to coordinate our project especially in
writing this report.

Furthermore I would also like to acknowledge with much appreciation

the crucial role of the staff of Heliantha Enterprise, who gave the
permission to use all required equipment and the necessary material
to complete the prototyping task. Last but not least, many thanks go to
the Internql supervisor Mr El Kadmiri Omar whose have invested his
full effort in guiding the team in achieving the goal. I have to
appreciate the guidance given by other supervisor as well as the panels
especially in our project presentation that has improved our
presentation skills thanks to their comment and advices.

IOSM 2019/2020 2 Oct 2019

 Abstract IoT Report

Abstract

This project consists of the design of an integrated smart parking management system
for easier and quicker parking of vehicles, and as a consequence: a better saving of resources
such asfuel, time and space. The project is creating a mobile app to first inform drivers of the
status of particular parkinglot and show them the nearest way to it.Anotherfunctionality is to
give users the capability to book idle parking slots on the road to the parking. The state of a
parking is detected in real-time using a machine learning model from frames captured by a
limited resources camera and then sent to a real-time database which can be consulted and
used to reserve idle parking slots.

The main goal of this project is the detection of car parking occupancy in real-time using
a Convolutional Neural Network (CNN) classifier. The models has been trained on the
CNRPark+EXT dataset, a dataset that takes in consideration light condition changes, presence
of shadows and partial occlusions. The evaluation of the models we created showed a very
effective performance in multiple conditions.

Key words: IoT, Smart Parking, Deep Learning, Machine Learning

IOSM 2019/2020 3 Oct 2019

 Résumé IoT Report

Résumé

Ce projet consiste en la conception d'un système intégré de gestion du stationnement

intelligent permettant un stationnement plus facile et plus rapide des véhicules et, par
conséquent, une meilleure économie de ressources telles que le carburant, le temps et l'espace.
Le projet est la création d’une application mobile pour informer en premier lieu les
conducteurs de l’état de chaque parking et leur indiquer le chemin le plus proche. Une autre
fonctionnalité consiste à donner aux utilisateurs la possibilité de réserver des places de
stationnement libre sur la route menant au parking. L'état d'un parking est détecté en temps
réel à l'aide d'un modèle d'apprentissage automatique à partir de trames capturées par une
caméra à ressources limitées, puis envoyé à une base de données en temps réel qui peut être
consultée et utilisée pour réserver des places de stationnement libres.

L'objectif principal de ce projet est la détection de l'occupation des parkings en temps

réel à l'aide d'un classificateur CNN (Convolution Neural Network). Les modèles ont été
entrainés sur le dataset CNRPark + EXT, un dataset prenant en compte les changements de
condition d'éclairage, la présence d'ombres et d'occlusions partielles. L’évaluation des
modèles que nous avons créés a montré une performance très efficace dans de multiples
conditions.
Mots clés: IoT, Smart Parking, Deep Learning, Machine Learning

IOSM 2019/2020 4 Oct 2019

 <Table of Content IoT Report

Table of Content

Abstract ...................................................................................................................................... 2
Résumé ....................................................................................................................................... 4
Table of Content ......................................................................................................................... 5
Table of figures .......................................................................................................................... 7
Introduction ................................................................................................................................ 8
Chapter 1: General Context ........................................................................................................ 9
1.1 Introduction ................................................................................................................. 9
1.2 Parking problem ........................................................................................................ 10
1.3 General solutions ....................................................................................................... 10
1.4 Conclusion ................................................................................................................. 12
Chapter 2: Preliminary study.................................................................................................... 13
2.1 Introduction ............................................................................................................... 13
2.2 Insight into the 2017 and 2018 IOSM smart parking projects .................................. 14
2.3 Our solution ............................................................................................................... 17
2.4 Conclusion ................................................................................................................. 17
Chapter 3: Specification ........................................................................................................... 18
3.1 Introduction ............................................................................................................... 18
3.2 Functional specifications ........................................................................................... 19
3.3 Nonfunctional specifications ..................................................................................... 19
3.4 Use case Diagram ...................................................................................................... 19
3.5 Project Management .................................................................................................. 20
3.6 Project planning ......................................................................................................... 21
3.7 Conclusion ................................................................................................................. 22
Chapter 4: Machine Learning ................................................................................................... 23
4.1 Introduction ............................................................................................................... 23
4.2 Used deep learning architectures ............................................................................... 24
4.3 Dataset ....................................................................................................................... 27
4.4 Training ..................................................................................................................... 29
4.5 Evaluating .................................................................................................................. 29
4.6 Tuning the model for the prototype ........................................................................... 30
4.7 Deploying .................................................................................................................. 32

IOSM 2019/2020 5 Oct 2019

 <Table of Content IoT Report

4.8 Conclusion ................................................................................................................. 33

Chapter 5: Implementation ....................................................................................................... 34
5.1 Introduction ............................................................................................................... 34
5.2 Prototype .................................................................................................................... 35
5.3 Technologies .............................................................................................................. 35
5.4 Project architecture .................................................................................................... 38
5.5 Application and website Presentation........................................................................ 38
5.6 Conclusion ................................................................................................................. 41
General conclusion ................................................................................................................... 42
References ................................................................................................................................ 43
Appendix A: Machine learning code snippets ......................................................................... 44
a: Defining Densenet using PyTorch .................................................................................... 44
b: Defining SqueezeNet ........................................................................................................ 44
c: Initiate the dataset loaders ................................................................................................ 45

IOSM 2019/2020 6 Oct 2019

 Table of figures IoT Report

Table of figures

Figure 1: the parking problem .................................................................................................. 10

Figure 2: Barrier and Infrared sensor ....................................................................................... 11
Figure 3: Ground Sensors ......................................................................................................... 11
Figure 4: Impractical solutions ................................................................................................. 12
Figure 5: 2017 smat parking project........................................................................................ 14
Figure 6: 2018 smart parking project ....................................................................................... 15
Figure 7: Snow covering anon-street parking space. ............................................................... 16
Figure 8: Parking spaces with multiple visual conditions ........................................................ 17
Figure 9: Our project use case diagram .................................................................................... 20
Figure 10: Developement life cycle ......................................................................................... 20
Figure 11: The project realization Gantt diagram .................................................................... 21
Figure 12: Support vector machine .......................................................................................... 23
Figure 13: Image classification ................................................................................................ 24
Figure 14: DenseNet ................................................................................................................. 25
Figure 15: DenseNet vs ResNet. Validation error in function of the number parameters and
the number of floating point operations per second ................................................................. 25
Figure 16: SqueezeNet fire model ............................................................................................ 26
Figure 17: SqueezeNet archetecture ......................................................................................... 27
Figure 18: CNRPark+EXT dataset samples ............................................................................. 28
Figure 19: Loss in function of the number of batches .............................................................. 29
Figure 20: Accuracy for real world parking situations ............................................................ 30
Figure 21: Accuracy in the case of the prototype ..................................................................... 30
Figure 22: Sample of idle parking spaces ................................................................................ 31
Figure 23: Samples of occupied parking spaces ...................................................................... 31
Figure 24: Deployment ............................................................................................................. 33
Figure 25: the project's prototype ............................................................................................. 35
Figure 26: Project architecture ................................................................................................. 38
Figure 27: login in and signup interfaces ................................................................................. 39
Figure 28: showing a parking on the map with his state .......................................................... 39
Figure 29: showing the user's parking historic and balance ..................................................... 40
Figure 30: dashboard with the parking lots availabale ............................................................. 40
Figure 31: adding a new parking lot to the system .................................................................. 41
Figure 32: Defining Densenet using PyTorch .......................................................................... 44
Figure 33: Define SqueezeNet ................................................................................................. 44
Figure 34: Initiate the dataset loaders....................................................................................... 45

IOSM 2019/2020 7 Oct 2019

 Introduction IoT Report

Introduction

Parking can be a daily struggle for many people, as in attempting to find a nearby yet
affordable space to park for work in an office building. It’s also a problem during particular
occasions, as hundreds gather around a few blocks or streets for a mall or a festival.The
problem could be seen as the way parking spaces are managed and not necessarilythe
insufficiency of the space allowed to park cars itself. Inefficiencies like drivers hovering over
to find a free space for their cars can be frustrating to drivers and lead to even more traffic
congestions.By bouncing between lots and garages that are full due to lack of immediate
awareness of where spaces are open, drivers become restless and may choose to park illegally
or leave altogether, creating a loss for the city in more ways than one.

The straightforward solution for a such a problem is to offer the driver a reliable, real-
time data that allows him/her to check whether on-street parking, surface lots or garages are
available or not.These solutions (smart parking) bring to light the problem with existing
infrastructure. The need is to establish a highly functional and efficient parking management
solution that ensures resident satisfaction and utilizes the existing parking lots and on-street
parking thought-out the city.

In this report and for the next chapters and sections we’ll talk with further details about
the parking problematic and how to deploy an optimal solution that it’s costly cheap and
efficient. We’ll talk also about how we’ve arrived to train our models and we’ll do a
comparison with different machine learning solutions. We’ll have sections dedicated to the
app design, database and web functionalities and how everything is integrated to a singular
parking management service.

IOSM 2019/2020 8 Oct 2019

 Chapter 1: General Context IoT Report

Chapter 1: General Context

1.1 Introduction

Before proceeding to details and the technical stuff it’s better to start by drawing some
context, that what this chapter is for: underlining systematically and authentically the problem
and its questions is halfway through the solution.

The parking management is a wide and complex field. To solve the parking problem
and its impact there is a huge toolset of engineering solutions proposed by different respective
scientific fields, in this report although, we’ll only focus on the use of computers, information
science and mainly IoT to tackle this problem.

In this chapter we’ll be putting our project into its context, talking about the parking
problematic and some proposed solutions to it, mostly in a general view.

IOSM 2019/2020 9 Oct 2019

 Chapter 1: General Context IoT Report

1.2 Parking problem

Let’s start by dropping in some context, before exploring some solutions it’s better to
have a clearer view to the parking management problem, and this is what this section is for.
Then we’ll show a variety of solutions that are destined to solve it.

Figure 1: the parking problem

Nowadays, the parking problems have been one of the most discussed topics by the
general public. In many metropolises, the parking problems are becoming increasingly
important. Following the rapid incensement of traffic demand, the imbalance between parking
supply and parking demand has been considered as the main reason for metropolis parking
problems; the second main problem is the lack of a better management of the existing parking
facilities. Moreover, the parking system plays a key role in the metropolitan traffic system,
and the parking problems show closed relation with traffic congestion, traffic accidents, and
environment pollution.

Following the goal of this project we’ll try to address the problem as it’s a management
problem more than it’s a scarce parking facilities one. Hence the lack of information that
whether a parking lot is full or not or is a specific parking slot available or not, the lack of
such an information for the drivers results in a major resources losses and management issues.

1.3 General solutions

The parking problem has been for decades so a variety of solutions have been designed
to address it. In this sub-section we’ll try to site some.

IOSM 2019/2020 10 Oct 2019

 Chapter 1: General Context IoT Report

Barrier and Infrared sensor

Figure 2: Barrier and Infrared sensor

The first solution is using a barriers or infra red sensors to control the entrance of a
parking. This solution is effective if you want to count the number of parked cars deducing
whether the parking is full or not. The problem here is these same barriers in most of the cases
are used for both cars that park and cars that just need to enter which makes it difficult to infer
the status of the parking. There are other scenarios when this solution doesn’t work as well as
in the cases of dealing with Bicycles motorcycles and Pedestrians.

Ground Sensors

Figure 3: Ground Sensors

To begin with this is a very expensive solution, yet it’s the most reliable. Talking about
price, the fact that you are going to be charged 800Dhs per sensor and adding the installation
and maintenance cost for each parking slot makes it highly impractical for a lot of
applications.

Impractical solutions

Here is other impractical for the masses solutions that start from the point of view that
the parking problem is an insufficient parking spaces problem.

IOSM 2019/2020 11 Oct 2019

 Chapter 1: General Context IoT Report

Figure 4: Impractical solutions

1.4 Conclusion

We saw in the first chapter how parking can be a daily struggle for a lot and how technology
was applied to solve it, we defined the context of our project and that we’ll help us todevice a
solution for it.

IOSM 2019/2020 12 Oct 2019

 Chapter 2: Preliminary study IoT Report

Chapter 2: Preliminary study

2.1 Introduction

Before starting to work on our solution, it’s better to start studying and analyzing some
previous solutions of the problem, here in this chapter we do reviewing the smart parking
projects of the last two promotion of our master formation.

IOSM 2019/2020 13 Oct 2019

 Chapter 2: Preliminary study IoT Report

2.2 Insight into the 2017 and 2018 IOSM smart parking projects

We are the third promotion that is working on a smart parking project following the
work of the previous two promotions, of course each group has to come up with an idea and
improvement that set its parking management system apart from the previous one, but before
talking about our point of view we need first to give glimpse to the previous works.

2017 project

This was the first IOSM parking smart parking project where the students MOTIQ
Fatima-ezzahra, JERNANE Salma and REDA Oumaim created a prototype of what a smart
management of parking would look like and to demonstrate the benefits of such a system.

The model they used is strong and simple to implement get footage from limited
resources camera, use image processing to determine whether a slot is empty or not, then
update a real time database.

For the image processing part the idea that they came up with was drawing circles on an
idle parking slot, so when the circle is covered you can say with high certainty that a car is
parked.

To do that they need some basic image processing functionalities to determine the
presentation of a shape like a circle.

Figure 5: 2017 smat parking project

We are going to see the weakness of this solution later in the current.

2018 project

Last year the students KHARBOUCH Manal andKENNOUCHE Taoufiqused a quiet

similar method, the overall project’s architecture is the same a surveillance camera taking
parking frames that are processed by a computing unity to detect the full slots from the free

IOSM 2019/2020 14 Oct 2019

 Chapter 2: Preliminary study IoT Report

ones, then these informations are then used to update a real-time database so that app users are
informed of the parking state, allowing them to reserve slots etc…

The point of interest for as is the method used to classify a slot, whether, is empty or
not. The group of last year started from a more authentic question which is how to make a
computer fully understands what an empty slot is.? Once a computer understands that we can
easily classify slots by a simple comparison.

This idea is strong, although, the group last year only created a simple implementation
of it, certainly for prototype and demonstration reasons.

Figure 6: 2018 smart parking project

After receiving a frame that frame is converted to the gray scale and blurred, doing that
remove a lot of the secondary features from the image and what is lift is to compare a cropped
box, taken from a slot, to the case when that slot was empty(also a converted to gray and
blurred image).

The comparison between a free and a potentially full parking slot could be done using a
binary comparison or based on correlation function where the similarity can be determined by
how big the correlation between the two images is hence is the slot empty or not.

But also this method has its pitfalls.

IOSM 2019/2020 15 Oct 2019

 Chapter 2: Preliminary study IoT Report

Weaknesses of the previous solutions

What makes the strong point about the previous solutions is their ease in
implementation; this is true for the first solution and also for the second one in its simplest
form.

Speaking of weaknesses it’s clear that the first solution start from the assumption that
everything that covers a circle is a car which is not true in a lot of the cases, examples when
slots covered with snow or sands, and when there is a shadow that makes seeing parts of the
circle difficult, that makes the first solution quiet unpractical in exterior parking lots.

Figure 7: Snow covering anon-street parking space.

For the second solution, comparing a slot with the case when it was empty, it’s not weak
in its own but there is a problem with the difficulty it generates, to begin with a parking lot
with its slots are projected to different visual variables, lightening, shadows, occlusions,
environmental changes, namely snow and sands… we can add also different camera angles to
survey parking lots all these factors makes it very difficult to design a reliable system based
on this solution. Clearly because you have to do a study of the environment every time you
want to deploy it somewhere and after that you need to do some hardcore engineering to
create a customized solution for that particular parking lot. You can see that this generates a
lot of problems ranging from the time needed to deploy a solution to the reusability problem.

IOSM 2019/2020 16 Oct 2019

 Chapter 2: Preliminary study IoT Report

Figure 8: Parking spaces with multiple visual conditions

From what we can see the best IT solution to the smart parking problem must be based
on the learning process that is the ability of the machine to learn from data to build
generalized model that is reliable in different environments. This is the solution that we are
trying to build in project and we‘ll be using the deep learning method with its deep neural
nets.

Speaking of pitfalls and weaknesses the deep learning solution also has some, it has
only one and it’s getting data.

2.3 Our solution

Our solution is the use the process of learning to get over the complexities of manually
hard code the features of a full and empty slot for a model to understand. With learning using
deep learning, we only must provide a model with a significant amount of data and let learn
from it.

2.4 Conclusion

In this chapter we saw the proposed solution by the two previous groups of our master
formation, we saw why they can be considered weak solutions and which situation those
solutions full down, hence we have explained the motivation for making a solution based on
the ML technology.

IOSM 2019/2020 17 Oct 2019

 Chapter 3: Specification IoT Report

Chapter 3: Specification

3.1 Introduction
It a must, or at least very important, in a project to specify your specifications first, that
is the final goals and the use cases intended.

In this chapter we’ll try to list those specifications that must be respected and
accomplished by the end of the project. Here we are talking about, first, the functional
specificationsmeans the ones that must be delivered to the end user and the nonfunctional
ones: the underlying rules and technologies that govern our system but yet aren’t exposed to
the end user.

IOSM 2019/2020 18 Oct 2019

 Chapter 3: Specification IoT Report

3.2 Functional specifications

The functional specifications are:

 Log in and sign up

 Localization of parking lots in the vicinity of a client.
 Inform the client of the status of a parking lot.
 Booking parking spaces.
 Management of parking payments and fees.

3.3 Nonfunctional specifications

The nonfunctional specifications of our system are:

 The designed system should have little or no down time. It should always be up and
running.
 The system should have a fast response time. System should not take more than 30
seconds minus loading.
 The system should be secure. User should fill in his/her email address and password to
be authenticated to the system.
The system Should allow the customer to park without making a reservation.
 The system should be scalable. Even with an increasing number of users, system
should be able to perform effectively.
 The system should be user friendly with ability to show users where they are in the
system and guide them on some processes through programmed controls.

 The system should be reliable. In case of system failure, the system should be able to
recover quickly and continue working normally.

3.4 Use case Diagram

Here we represent the use cases of our system using the UML use case diagram.

IOSM 2019/2020 19 Oct 2019

 Chapter 3: Specification IoT Report

Figure 9: Our project use case diagram

3.5 Project Management

In this project we a simple cascade life cycle as it’s straightforward and we were sure that we
would have integration problems.

Figure 10: Developement life cycle

IOSM 2019/2020 20 Oct 2019

 Chapter 3: Specification IoT Report

3.6 Project planning

Here is the planning we tried to follow during this project represented using a Gantt diagram.

Figure 11: The project realization Gantt diagram

IOSM 2019/2020 21 Oct 2019

 Chapter 3: Specification IoT Report

3.7 Conclusion
Throughout this chapter we specify the requirement to respect while creating our
system, we saw the planning and the project’s life cycle that we’ll try to follow. We saw the
functional and non functional requirement that must be done and we represented the user’s
use cases using a UML use case diagram. Now we are finished specifying the rules we must
follow and we’ll start to build our real-time parking management system.

IOSM 2019/2020 22 Oct 2019

 Chapter 4: Machine Learning IoT Report

Chapter 4: Machine Learning

4.1 Introduction

Here we are going to talk about the application of machine learning in the field of
parking management. The reasonable type of machine learning to use in our case is
supervised learning that is providing a data to the model to learn. The classical supervised
learning to use for cases like parking slot occupancy detection is the use of SVM or Support
Vector Machine. SVM is a discriminative classifier formally defined by a separating
hyperplane. In other words, given labeled training data the algorithm outputs an optimal
hyperplane which categorizes new examples (1).

Figure 12: Support vector machine

IOSM 2019/2020 23 Oct 2019

 Chapter 4: Machine Learning IoT Report

Other solution is the use of the Haar cascade which is trained by superimposing the
positive image over a set of negative images.Haar can also be used to get features of an
images to get classified using SVM, so can be done using HOG.

Figure 13: Image classification

The method we’ll be using in this project is the famous deep learning using neural
networks.

4.2 Used deep learning architectures

One of the most important hyperparameters is the architecture to use basically it is the
neural network that holds the weights that are going to be learned. Is defined by the number of
convolutional layers you want to use, the number of classifier layers. You also need to define
the size of filters at each convolutional layer and other important parameters like the pooling
size and type.

Building a neural network from scratch is quiet a difficult endeavor so it’s better to use
a neural network that is already built and tuned through a lot of experiments on multiple
datasets. Doing this will let us focus on the learning process and defining the small amount of
hypreparameters.

DenseNet.

In this project we started by an architecture called Densenet (2).

A DenseNet is a stack of dense blocks followed by transition layers. Each block consists
of a series of units. Each unit packs two convolutions, each preceded by Batch Normalization
and ReLU activation. Besides, each unit outputs a fixed number of feature vectors. This
parameter, described as the growth rate, controls how much new information the layers allow
to pass through.

IOSM 2019/2020 24 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 14: DenseNet

Like constructing a deep neural network from scratch, teaching convolutional layers is
as much a difficult problem especially on a dense neural net, so what the best practices is, is
to take trained convolutional layers and freeze them and only do learning for the classification
layers. The convolutional layers are generally trained on big datasets like ImageNet and for a
dozen of hourse to several days that ensure that they are great at extracting images features.
The following figure shows DenseNet converges faster and need fewer parameters compared
to the other famous ReseNets on the ImageNet dataset.

Figure 15: DenseNet vs ResNet. Validation error in function of the number parameters and the number of
floating point operations per second

Check the a section of Appendix A to find how we define our DenseNet network.

SqueezeNet.

The second neural net we used was SqueezeNet (3), as the name suggest it’s powerful
point is being a lightweight model giving it the ability to make predictions in a shorter time,
thus it can give you a great performance on a CPU and other hardware with limited memory,
and this was highly beneficial for us because you would want to deploy the model on a
limited resources computer and you would like to have a real time performance in that case.

Squeezenet is deep neural network for computer vision that was released in 2016 When
it was developed by researchers at DeepScale, University of California, Berkeley, and
Stanford University and the research was submitted under the title AlexNet-Level Accuracy
with 50x Fewer Parameters.

IOSM 2019/2020 25 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 16: SqueezeNet fire model

To understand the Squeezenet architecture we must first understand what a Fire module
is:

A fire module is comprised of: a squeeze convolution layer(which has only 1x1 filters
instead of the typical 3x3), feeding into an expand layer that has a mix of 1x1 and 3x3
convolution filters.

There are three tunable dimensions (hyperparameters) in a Fire module:

 s1x1 the number of 1x1 in squeeze layer.

 e1x1 the number of 1x1 in expand layer.
 e3x3 the number of 3x3 in expand layer.

When we use fire modules we set s1x1 to be less than (e1x1 + e3x3), so the squeeze
layer helps to limit the number of input channels to the 3x3 filters.

Now we talk about the SqueezeNet architecture, the left figure shows a typical
SqueezeNet neural net: it begins with a standalone convolution layer (conv1), followed by 8
Fire modules (fire 2-9), ending with a final conv layer (conv10).

The number of filters per fire module is gradually increased from the beginning to the
end of the network.

Max-pooling with a stride of 2 is performed after layers conv1, fire4, fire8, and conv10.

IOSM 2019/2020 26 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 17: SqueezeNet archetecture

Check the b section of Appendix A for the code of how we defined our Squeezenet.

4.3 Dataset

The most crucial resource in deep learning is the data, in this section we’ll talk about
how did we managed our dataset and preprocess it.

IOSM 2019/2020 27 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 18: CNRPark+EXT dataset samples

To create our models we used the CNRPark+EXT (4) a dataset for visual occupancy
detection of parking lots of roughly 150,000 labeled images (patches) of vacant and occupied
parking spaces, built on a parking lot of 164 parking spaces.

CNRPark+Ext extends CNRPark, a preliminary dataset composed by 12,000 images

collected in different days of July 2015 from 2 cameras.

The additional subset, called CNR-EXT, is composed by images collected from

November 2015 to February 2016 under various weather conditions by 9 cameras with
different perspectives and angles of view. CNR-EXT captures different situations of light
conditions, and it includes partial occlusion patterns due to obstacles (trees, lampposts, other
cars) and partial or global shadowed cars.

After getting the dataset it’s time to import it to our project, we used Pytorch a deep
learning framework to train our model so we used its functionalities to import our dataset.

Importing a dataset with its labels is pretty easy with Pytorch: you only need to arrange
things on directories so that each directory includes only images of a single class.

We’ll create two directories on for training and one of evaluation. Then under each
directory we’ll create two sub-directories denoting the two classes: full and empty slot.

After we have loaded our data we do some preprocessing. What we did is that we
ensured that every image must be a 150x150 in width and height and we did a normalization
of images so we can get rid of a lot of secondary features, although not each time we want to
train a model did we normalize images for experiment searching for the best models you have
to test multiple hyperparameters like with normalization and without but generally
normalization is a great preprocessing step to prevent a potential over-fitting.

Pytorch allows you to define the number of instances randomly selected at each mini-
batch every iteration. Check c section of Appendix A for the code below of how we did
dataset loading and preprocessing.

IOSM 2019/2020 28 Oct 2019

 Chapter 4: Machine Learning IoT Report

4.4 Training

Training a model in deep learning training consumes a lot of the computing power and
you generally want to benefit from a GPU concurrent computation, in this project we used a
Tesla K80 with 12 GB GDDR5 VRAM.

Before training you should first define an optimizer that is how the learning rate will
evolve during training responding to multiple parameters like the loss and the number of
iterations.

To monitor your model’s training you should watch how the loss function evolves with
iterations. Throughout the training we used a lot of hyper parameters combinations, in the
chart below we show the evolution of the loss function in the case of the squeezenet with
0.001 initial learning rate and exponential scheduler that it will decay our learning rate decay
buy a factor of, in our case, 0.1 every couple of epochs in our case 7, for the optimizer
function we used the Stochastic Gradient Decent and the cross entropy loss as a criterion to
calculate the loss every batch iteration.

60

50

40

30

20

10

-10 0 500 1000 1500 2000 2500 3000 3500

Figure 19: Loss in function of the number of batches

4.5 Evaluating

You can’t judge your model is great before you put it under test, in the ML it’s a well
known best practice to use a evaluation dataset with instances different than the ones used in
the training, and most importantly they should be unbiased and should be diversified to test
your model ‘s generalization and detect every possible over-fitting.

In our case we randomly separate our dataset in two folders one for training and one for
testing so that we have a 75% of data destined to training and the remaining 25% randomly
selected instances for test.

For our evaluation we got a nice 98.9% precision on the overall test dataset which we
consider as a height precision.

IOSM 2019/2020 29 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 20: Accuracy for real world parking situations

The problem we faced is that at the end the model must be deployed on a prototype with
a black cover instead of the road and car-toys instead of cars. So to test our model in that case
we created a different dataset generated from our prototype and the results were that the
efficiency dropped to 76% and in the next section we’ll see how we tuned our model for the
prototype case.

Figure 21: Accuracy in the case of the prototype

4.6 Tuning the model for the prototype

We can overcome the problem above by including in the training a dataset generated
from our prototype so that the final model gets better at more parking space cases i.e. for our
prototype.

The problem here is how much data a small prototype can provide and how to augment
that data so the training worth the effort?

For free slots (i.e. the not occupied class) we can easily generate thousands of images by
first: expose the empty parking prototype to different lightening conditions and then crop with
stride each parking picture taken so that we can generate hundreds of patches for free slots.

IOSM 2019/2020 30 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 22: Sample of idle parking spaces

For the full class It’s a little bit trickier, supposedly you have a dozen of toy cars how
much data can you generate? In this case also we put the cars on their positions in the parking
and exposed the parking to different lightening conditions. Next we take some crops form the
pictures taken so that the car takes the most of the cropped picture. Having those pictures, we
could generate up to one thousand images by cropping the images again with stride. But this
is still not enough, and we can easily generate more images by augmenting our data for full
parking space. The augmenting methods we used are motion blurring the images vertically
and horizontally, tuning the gamma function for the images so that we make them darker, and
we flipped them horizontally and vertically. Doing this give us the possibility to generate
seven thousand images for the ‘occupied’ parking space class and four thousand images for
the ‘free’ parking space class.

Figure 23: Samples of occupied parking spaces

After splitting this dataset randomly between both the training and evaluation sets we
add them to the previous CNRPark+EXT dataset and we trained our model as we have done
earlier except this time we changes the size of our batches to 75x75 pixels images so that it
matches the classification boxes we use on our prototype. After training we test the model
using our evaluation set and we got a precision of 98.96% which makes our model usable
again.

IOSM 2019/2020 31 Oct 2019

 Chapter 4: Machine Learning IoT Report

4.7 Deploying

After training and evaluating we’ll use Pytorch again to export our trained model.
Pytorch allows for multiple types of doing export. One of them is to export a model for
prediction uses, this means the resulted file contain the weights plus the architecture in a way
that is easy to use the model for inference and you only need to import it again by the Pytroch
load function. The other two methods one of them allows you to save an optimized version for
weights only, so that you need to define the architecture every time you want load those
weight. The third option for you is to save and load your weights and optimizer at a specific
checkpoint so you can continue your model training from where you left it.

For prediction you need first to resize the image to a 150x150 pixels in width and height
and depending on the model you’ll need also to do normalization before feeding the image to
predict the right class.

Now you have a system that takes an image of a parking space and gives you the
occupancy status, and now is time to find uses for such functionality. Our model is optimized
so that it can give you a real time performance on a CPU that means you can install the
classifier on a small computing unity like Rasberry Pi attached to a camera. Now we can
connect our computing unity to the Internet so that we can update a real-time database of the
status of our parking. And then use can use that piece of information to allow every possible
type of service: ranging from informing an app user of the parking status to giving him the
capability to book parking spaces to parking payment services.

The group of scientist that worked on the CNRPark+EXT dataset also worked on an
integrated device that just did the functionalities we talked about above and they called their
device ‘Smart camera’ (5), we are not going to talk about the name, but we can agree that it’s a
beautiful way to deploy such a service.

You can find more about the deployment and the use of our ML model on the
implementation section of this report.

IOSM 2019/2020 32 Oct 2019

 Chapter 4: Machine Learning IoT Report

Figure 24: Deployment

4.8 Conclusion

In this chapter we looked at the essential elements of constructing our deep learning
model that is, first, preparing a dataset and neural network architecture, second the training.
Because the training is an experimental endeavor we needed to try multiple hyper parameters
like the learning rate the loss function and the neural network itself. The third element we
talked about was testing, because it’s the only tool we can use to ensure that our model
generalize fine and don’t suffer from over-fitting. The last element we talked about in this
chapter is deploying, we saw who the machine learning framework we used (PyTorch) make
this task pretty easy. Now we have our machine learning model working and classifying
parking spaces, it’s time to create a Web interface and an App to it so users can benefit from
what it has to offer.

IOSM 2019/2020 33 Oct 2019

 Chapter 5: Implementation IoT Report

Chapter 5: Implementation

5.1 Introduction

After a frame is taken by the camera and we extract the needed information from it, we
use that information to update a real-time database, then we can consult the database to allow
a lot of types of application ranging from informing a client of parking at his vicinity. To
allow these types of services we need to lay out an architecture of software that can handle
users and administrators’ operations and interactions.

In this chapter we will see the different technologies used and how they were used, next
will talk about the project architecture and then will finish by presenting the final result of our
work displaying all the features our App and Website has to offer.

IOSM 2019/2020 34 Oct 2019

 Chapter 5: Implementation IoT Report

5.2 Prototype

During this project we had to physically test our system so we needed a prototype. The
prototype we used is a representation of a parking with a camera at the top so that we can
track the events at the parking in real-time.

Figure 25: the project's prototype

5.3 Technologies

In this section we’ll see the software technologies we used during the creation of this project.

Node.js

Node.js is an open-source, cross-platform, JavaScript run-time

environment that executes JavaScript code outside of a browser.
Node.js lets developers use JavaScript to write command line
tools and for server-side scripting—running scripts server-side
to produce dynamic web page content before the page is sent to
the user's web browser.

Node.js allows the creation of Web servers and networking

tools using JavaScript and a collection of "modules" that handle
various core functionalities. These Modules are provided for file
system I/O, networking (DNS, HTTP, TCP, TLS/SSL, or UDP),
binary data (buffers), cryptography functions, data streams, and

IOSM 2019/2020 35 Oct 2019

 Chapter 5: Implementation IoT Report

other core functions. Node.js's modules use an API designed to

reduce the complexity of writing server applications.

React

React is a JavaScript library for building user interfaces. It is

maintained by Facebook and a community of individual
developers and companies.

React can be used as a base in the development of single-

page or a mobile application, as it is optimal for fetching rapidly
changing data that needs to be recorded. However, fetching data
is only the beginning of what happens on a web page, which is
why complex React applications usually require the use of
additional libraries for state management, routing, and
interaction with an API.

React Native

React Native is an open-source mobile application

framework created by Facebook. It is used to develop
applications for Android, IOS, Web and UWP by enabling
developers to use React along with native platform capabilities.

In context of our project we used this along React to develop

our mobile application that will give the client accessibility to
check parking slots availability in Real-Time and Reserve one
of them.

PyTorch

PyTorch is an open source machine learning library based on

the Torch library, used for applications such as computer
vision and natural language processing. It is primarily
developed by Facebook's artificial intelligence research
group. It is free and open-source software released under
the Modified BSD license.

Although the Python interface is more polished and the primary

focus of development, PyTorch also has a

IOSM 2019/2020 36 Oct 2019

 Chapter 5: Implementation IoT Report

C++ frontend. Furthermore, Uber's Pyro probabilistic

programming language software uses PyTorch as a backend.

PyTorch provides two high-level features:

 Tensor computing (like NumPy) with strong

acceleration via graphics processing units (GPU)
 Deep neural networks built on a tape-
based autodiff system

Python/OpenCV

OpenCV (Open source computer vision) is a library of

programming functions mainly aimed at real-time computer
vision. Originally developed by Intel, it was later supported
by Willow Garage then Itseez (which was later acquired by
Intel). The library is cross-platform and free for use under
the open-source BSD license.

OpenCV supports the deep learning frameworks such as the one

we used: Pytorch. It binds to python which is
an interpreted, high-level, general-purpose programming
language. Python's design philosophy emphasizes code
readability with its notable use of significant whitespace. Its
language constructs and object-oriented approach aim to help
programmers write clear, logical code for small and large-scale
projects.

This was mainly used in the detection and the recognition of all
the necessary information about our parking lot available
through our camera in order to process them later.

Cloud Firestore

Cloud Firestore is a flexible, scalable database for mobile, web,

and server development from Firebase and Google Cloud
Platform. Like Firebase Realtime Database, it keeps your data in
sync across client apps through realtime listeners and offers offline
support for mobile and web so you can build responsive apps that
work regardless of network latency or Internet connectivity. Cloud
Firestore also offers seamless integration with other Firebase and
Google Cloud Platform products, including Cloud Functions.

IOSM 2019/2020 37 Oct 2019

 Chapter 5: Implementation IoT Report

Firebase was heavily used to store and mange data. Needless to

say how important data for such a project.

5.4 Project architecture

We talked a lot about the project architecture throughout this report until it became
unnecessary and just redundant to dedicate a section for it. But we will do it anyway. Well!
The architecture is so much straightforward: a limited resources camera surveils the parking
lot (we are talking about the prototype case for a typical parking lot you would need more
than one camera). The next step is classify parking space images with a DL neural network
and then update the Firestore database. The data is used to manage and offer multiple services
for the users of the App so users can be informed about parking in their vicinity and reserve,
park and pay for their parking.

We’d like also to add that Google API was used to map parking lots and to indicate roads to
users.

Figure 26: Project architecture

5.5 Application and website Presentation

Finally we get to see the result of our work.

IOSM 2019/2020 38 Oct 2019

 Chapter 5: Implementation IoT Report

In this section full of screenshots we tried to show you how the App and dashboard look
like for certain use cases.

Figure 27: login in and signup interfaces

Figure 28: showing a parking on the map with his state

IOSM 2019/2020 39 Oct 2019

 Chapter 5: Implementation IoT Report

Figure 29: showing the user's parking historic and balance

Figure 30: dashboard with the parking lots availabale

IOSM 2019/2020 40 Oct 2019

 Chapter 5: Implementation IoT Report

Figure 31: adding a new parking lot to the system

5.6 Conclusion

In this final chapter we saw some technical stuff of which technologies were used and
how. We saw the architecture of the project and we demonstrate the look of our App and
Website so that we resume our work.

IOSM 2019/2020 41 Oct 2019

 General conclusion IoT Report

General conclusion

Here is time to summarize what we did during this project. So this project was to build a
real-time smart parking system based on deep learning. The system is supposed to inform
clients about the parking lots around them, allow them to reserve and then calculate payment
depending on the client parking time. At the end of the allowed time for us to finalize our
project the objectives of the project are met, a client can now charge his account and use his
balance to park and pay for the parking lots covered by our service.

For the deep learning models evaluating using the evaluation dataset shows a great
performance for different lightning conditions, and to preserve the models performance for
the case of our prototype we tuned our model using a dataset directly generated from the
prototype.

We can develop our service so that it includes the capability to guide a driver to a
particular parking slot, generally finding a vacant parking space is difficult even if our
application says that, for example, the slot number 36 is empty, to guide a user to a specific
parking space a localization method can be used so to show him the way to a particular slot.
For localization we can use the GPS, but that can generate errors by so much interesting
meters, as a better solution if the parking area is covered with Wi-Fi we can use advanced
techniques to narrow the position of the driver (6) to a higher precision. Another problem
rises is how to verify that a user parked in a reserved slot is the same user how made the
reserve, to respond to this question we can use another time localization so that we make sure
and only charge user parking fees we he really parks.

Another question rises is how to create a reliable payment system on top of a not
necessarily unreliable DL model. DL models could be 99.99% accurate but they can never be
100%, to minimize the error for such system a voting method could be used independent ML
process can predict classes for parking spaces from different perspectives. This way you can
make the error more unlikely to happen.

IOSM 2019/2020 42 Oct 2019

 <References IoT Report

References

1. SVM- and MRF-Based Method for Accurate Classification of Hyperspectral Images.

Tarabalka, Yuliya, et al. s.l. : IEEE, 2010.

2. Densely Connected Convolutional Networks. Gao Huang, Zhuang Liu, Laurens van der
Maaten, Kilian Q. Weinberger. 2016.

3. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size.
Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally,
Kurt Keutzer. 2016.

4. Deep learning for decentralized parking lot occupancy detection. G Amato, F Carrara, F
Falchi, C Gennaro, C Meghini, C Vairo. s.l. : Science Direct, 2016.

5. Car parking occupancy detection using smart camera networks and Deep Learning.
Amato, Giuseppe, et al. s.l. : IEEE, 2016.

6. Deep Learning in Indoor Localization Using WiFi. Turgut, Zeynep, et al. s.l. : Springer,
2018.

IOSM 2019/2020 43 Oct 2019

 Appendix A: Machine learning code snippets IoT Report

Appendix A: Machine learning code snippets

a: Defining Densenet using PyTorch

Figure 32: Defining Densenet using PyTorch

b: Defining SqueezeNet

Figure 33: Define SqueezeNet

IOSM 2019/2020 44 Oct 2019

 Appendix A: Machine learning code snippets IoT Report

c: Initiate the dataset loaders

Figure 34: Initiate the dataset loaders

IOSM 2019/2020 45 Oct 2019