A Project Report

on

VEHICLE LICENSE PLATE IDENTIFICATION

carried out as part of the Deep Learning Lab project
Submitted

by

Anirudh Bhardwaj
229309049

Data Science and Engineering

Under the Guidance of

Dr. Anju Susan
 LIST OF FIGURES
Figure No Figure Title Page No
1 Workflow of the proposed method 5
2 Images from “License Plate Detection 6
3 Output YOLO 7
4, 5 Image Processing, Determine the frame of characters in the number plate 8
6 CNN architecture 9
7, 8, 9 Result, Loss function & Accuracy in train and validation 10
 TABLE OF CONTENTS

INTRODUCTION 05
1.1 Motivation
1.2 Problem Statement
1.3 Objective & Research Method

SYSTEM DESIGN & METHODOLOGY 06

1.1 Datasets
1.2 License Plate Detection
1.3 License Plate Segmentation
1.4 License Plate Recognition

IMPLEMENTATION & RESULT 10

CONCLUSION & FUTURE PLAN 11
REFERENCES 12
 ABSTRACT
With the popularity of personal vehicles today, the management of traffic activities is becoming
increasingly difficult. Since then, the detection of the right people, the right errors to sanction
have become an important challenge to be solved. Therefore, we propose to develop a license
plate recognition algorithm to partially solve this problem. The original algorithm will only
recognize one license plate per frame. However, as we go further, we will aim to detect multiple
license plates at the same time in the frame.
Our aim is to apply deep learning algorithms to identify and recognize a license plate in the
frame. The proposed method here consists of three main parts: License Plate Detection, License
Plate Segmentation and License Plate Recognition. We are using both traditional processing
methods and deep learning. This is the first step in implementing the entire identification
system including software and hardware in real applications.
 INTRODUCTION
1.1 Motivation
The development of automated information technology system integration in modern life has
resulted in moving cars becoming an enormously valuable resource in the field of vehicle
monitoring. This system is commonly used to control vehicles entering a parking lot, identify
vehicles breaking traffic safety laws, follow suspicious vehicles on the road, and so on. Data
gathering is necessary since an automated system is useless without it. This can be
achieved by humans, such as traffic police or human agents. However, with the expansion in
the number of vehicles today, the difficulty in monitoring has increased, as have the expenses
for personnel, while resources are limited. As a result, automatic methods for detecting and
recognizing license plates are required. The license plate image is automatically taken,
recognized, and processed to detect characters and convert from photos to text using our
camera, computer system, and deep learning algorithms. From there, managing and
extracting data becomes easier.
1.2 Problem Statement
Automatic license plate recognition technology has been around for a long time and is widely
applied in life. However, the classical methods can only work well in limited areas where
license plates are clearly displayed or displayed alone. When applied to larger areas, much
affected by external conditions. The presence of multiple license plates in the same frame
also makes it more difficult to detect the correct number plate. Therefore, the street
application of these automated systems is still limited.
1.3 Objective & Research Method
Our main method will be to apply deep learning algorithms to identify license plates. The road
map of this project is divided into three sections as Fig.1: License Plates Detection, License
Plates Segmentation, License Plates Recognition. We propose different approaches that are
appropriate for purpose and can be very accurate for each part of this project. The ‘You Only
Look Once’ (YOLO) method has our highest recommendation for detection process due to its
demonstrated speed and accuracy across a wide range of datasets. For segmentation process,
we use thresholds and contours to segment characters in the clearest way. The output of this
method will be fed into the CNN model we constructed to begin the training process.

Fig.1 Workflow of the proposed method

 SYSTEM DESIGN & METHODOLOGY
1.1 Datasets
The dataset we use here was obtained on the Kaggle platform with the titles "Car License Plate
Detection" [1] and "Vietnamese Plate Dataset" [2] from GitHub. The use of available data sets is
intentional because these data ensure the clarity and completeness of our problem.
These datasets are professionally recorded and under real-world conditions. Specimens also
had differences in the angle and orientation of the number plates as shown in Fig.2. It is this
that increases the challenge for our algorithm.

Fig.2 Images from “License Plate Detection

1.2 License Plate Detection

Detecting the license plates under consideration is a typical object detection problem with only
one main class of license plates. And this project proposes the algorithm 'You Only Look Once'
(YOLO) because of its advantages in this problem. This architecture was introduced by Joseph
Redmon, Ali Farhadi, Ross Girshick and Santosh Divvala first version in 2015 and later version
2, 3 and here we used version 4, published in April 2020 by Alexey Bochkovsky. The YOLO
algorithm employs convolutional neural networks (CNN) to detect, recognize, and classify
objects. It includes convolutional layers and fully – connected layers. In which the
Convolutional layers will extract the features of the image, and the Full – connected layers will
predict probability and the coordinates of the object. As the name suggests, the algorithm
requires only a single forward propagation through a neural network to detect objects. This
means that prediction in the entire image is done in a single algorithm run. This makes its speed
and real-time performance so appreciated.
YOLO techniques. The algorithm yolo uses follows three techniques: residual block, bounding
box regression and Intersection Over Union, respectively. It will first divide the image into
grids. Each of these nets will be responsible for detecting the object whose center lies within
it. A single bounding box regression then predicts the height, width, and center of the object to
help locate the object in the image. Finally, the Intersection over union ensures that the
predicted bounding boxes are equal to the real boxes of the objects. The final detection will
consist of unique bounding boxes that fit the objects perfectly.
Loss function. The error function in yolo includes the error of the following 3 components:
Classification loss, Localization loss, Confidence loss. Classification loss - the error of
predicting the object's label type, this error function only calculates on the squares where the
object appears. Localization loss is an error function used to calculate the error value for the
predicted boundary box including the coordinates of the center, width, and height relative to
the actual position from the model's training data. Confidence loss is the error between the
prediction that the boundary box contains the object and the actual label in that square. This
error is calculated on both squares containing objects and not containing objects.
The output of this process as shown in Fig.3 will be the input to the following process. So, this
is an important step if you want this entire project to be successful.

Fig.3 Output YOLO

1.3 License Plate Segmentation
The input is the image of the plate, we will have to be able to extract each character image. The
result of this step, being used as input to the recognition phase, is of great importance.
Therefore, Segmentation is one of the most important processes for the automatic identification
of license plates, because any other step after this is based on it. If the segmentation fails, the
recognition phase will not be correct. The process of this step is described as Fig.4.
Convert to grayscale. Color plays a negligible role in understanding the license plate; thus, we
can remove it to optimize computational power.
Blur image. The Blur technique is performed to remove noise and irrelevant information. In
the example code, I used Gaussian Blur with kernel size of (7,7) but this value can be tuned
depending on your image. The higher the kernel size, the less noise but more information is
lost.
Image thresholding. We set a threshold so that any smaller pixel value than it would be
converted to 255 and vice versa. This type of thresholding is called inverse binary thresholding.
Segmentation. We use bounding boxes and compare its height/width against a certain threshold
to distinguish each character.
 Fig.4 Image Processing

1.4 License Plate Recognition

The recognition phase is the last step in the development of the automatic license plate reader
system. Thus, it closes all the processes passing by the acquisition of the image, followed by
the location of the plate until the segmentation. The recognition must be made from the images
characters obtained at the end of the segmentation phase as Fig.5. The learning model that will
be used for this recognition must be able to read an image and to render the corresponding
character.
We obtained a set of data composed of 36 classes and for each class we have 30–40 images of
28X28 pixel dimension PNG formats: numbers from 0 to 9 and the word in Latin.
This project proposed a CNN model to automatically extract the signal features by multiple
CNN layers. The deeper layers are expected to get more abstract of the data. The proposed
structure of the CNN model is shown in Fig.6 The proposed model has 4 Conv2D layers, a
Max Pooling layer, 3 Dense layers, a Dropout layer and a SoftMax classifier layer at last

Figure 5. Determine the frame of characters in the number plate

 22x22 ConV2D,

16x16 ConV2D,

8x8 ConV2D, 64

4x4 ConV2D, 128

MaxPooling, pool size = (4,4)

Flatten

FC, 512

Dropout, 0.4

FC, 128

FC, 36

Figure 6. CNN architecture

 IMPLEMENTATION & RESULT
The original model uses the SoftMax activation function to fit the multilabel classification
problem. We use the sparse categorical cross entropy loss function because there are more than
two labels with Adam Optimizer.
The initial results obtained are quite good when the numbers on the license plate are all
predicted. However, because of limitations in the segmentation step, the algorithm recognizes
digits in addition to unwanted characters. According to Figure 7.
Figure 8 and Figure 9 show the loss function results and the accuracy is acceptable for the
characterization model. This shows that the model performs well when used for prediction.
However, the existing errors due to the segmentation step are still not working as expected
when defining character frames, there are still many limitations.

Figure 7. Result

Figure 8. Loss function in train and validation Figure 9. Accuracy in train and validation
 CONCLUSION & FUTURE PLAN
Our approach has yielded positive results despite the shortcomings of small steps. The
results were satisfactory when tested in practice and the selected model worked well.
This process helps us better understand Deep Learning and how it applies to today's
technologies and techniques.