Real Time Object Detection And Recognition In Machine

Learning Using Jetson Nano

Dr Mohd Nazeer 1 Mohammed Qayyum2 Dr Abdul Ahad 3
1
Department of Computer Science and Engineering, Anurag University, Telangana.
2
Dept. of Computer Engg, King Khalid University, Abha, Saudi Arabia.
3
Department of Artificial Intelligence, Anurag University, Telangana. mohdnazeercse@anurag.edu.in
mdqayyum.se@gmail.com, drabdulahadai@anurag.edu.in

Abstract: Object detection is the technique of determining the presence of an object and
estimating its location in the image canvas. Object recognition classifies the detected object
from the list of previously seen (trained on) objects. In an image with multiple objects, it is
a challenging task to determine the location of all the individual objects (detection) and
then recognize them, due to several reasons: 1) There can be a possible overlap between
multiple objects causing collusions for one or all. 2)Objects in the image can have varying
orientations. 3)The objects could only be partially present in the image. 4)Images from low
fps video stream can be blurry and distort the features of the object. Jetson Nano is a GPU-
enabled edge computing platform for Machine Learning and deep learning applications.
The GPU-powered platform is capable of training models and deploying online learning
models but is most suited for deploying pre-trained AI models for real-time high-
performance inference. In this research paper we provide a mechanism to create real-time
multiple object detection and recognition application using python on the Jetson Nano
developer kit using the camera and deep learning models and libraries. We are able to
detect various objects using proposed mechanism in the research lab and storing the data of
the object to be utilized for various application for attendance and surveillance.

Keywords: Jetson Nano, Object Detection, Classification, Recognition

other hand, imagine the difficulty of such

I INTRODUCTION
Object detection is technologically robot in detecting the machine on kitchen
challenging and practically useful slab that is cluttered by other utensils,
problems in the field of computer vision. gadgets, tools, etc. The searching or
[1] object detection is a way of finding the recognition process in such scenario is
object in a given image or in a video clip very difficult. So far, no effective solution
correctly and identifying whether the has been found for this problem. [2] A lot
object belongs to the appropriate category of research is being done in the area of
or not when objects are placed in object recognition and detection during
arbitrary poses in cluttered and occluded the last two decades. The research on
environment. As an example, it might be object detection is multi-disciplinary and
easy to train a domestic help robot to often involves the fields of image
recognize the presence of coffee machine processing, machine learning, linear
with nothing else in the image. On the algebra, topology, statistics/probability,
optimization, etc. As different objects
Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 118

Electronic copy available at: https://ssrn.com/abstract=4286087

may appear in any positions of the image recognize objects and classify them.
and have different aspect ratios or sizes, it Feature detection is a low-level image
is a natural choice to scan the whole processing operation. A graph-based
image with a multi-scale sliding window. algorithm in generates proposals of
Although this exhaustive strategy can find vehicles with better quality than other
out all possible positions of the objects, traditional region proposal approaches
its shortcomings are also obvious. Due to DPM is the latest successful classic
a large number of candidate windows, it is object detector with significantly
computationally expensive and produces improved detection accuracy. However
too many redundant windows. However, the computation complexity of DPM is
if only a fixed number of sliding window still very high and its detection
templates are applied, unsatisfactory accuracy is low for driving object
regions may be produced. detection. While classic object detection
gets stuck in a bottleneck, there is a
II. Related Work large break through on visual object
detection with deep learning models,
Enhanced object detection: Object
especially CNN models. Powered by
detection is a popular research over the
GPUcomputers and huge object detection
past few decades. Classic object detectors
samples, CNN models can automatically
use hand-crafted features, such as
learn complex and efficient features
histogram of oriented gradients (HOG)
from sample images.
[3], integral channel features (ICF) and
aggregated channel features (ACF).
Object Detection API Provided by
From the aspect of feature enhancement,
Tenser Flow: The Tensor Flow
Feature Extraction is one of the most
Object Detection API Single-Shot
popular research areas in the field of
Detector models converted to Tensor
image analysis as it is a prime
Flow Lite that describes the signature
requirement in order to represent an
of it any object detection technique is
object. Feature extraction is a process of
firstly trained in order to detect the
dimensionality reduction by which an
boundaries of an object that belonging
initial set of raw data is reduced to more
to multiple classes. For example, if a
manageable groups for processing. A
model is trained with some pictures that
characteristic of these large data sets is
contain many other pictures but all of
a large number of variables that require a
them are related, also provides the
lot of computing resources to process.
label that specifies which specifies the
This feature vector is used to

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 119

Electronic copy available at: https://ssrn.com/abstract=4286087

 class of an object (e.g. an book, a car, or Network (ConvNet/CNN) is a Deep
a pole), and data specifying where Learning algorithm which can take in an
each object appears in the image. When input image, and details which are
an image or live video is continuously important are shared to various objects in
recording in a specified model, it will the picture and they can categories the
recognize the objects and provides a different objects. The pre- processing
list of images related to it while required in a CNN is much lower as
detecting, the location of a bounding compared to other classification
box that contains each object, and a algorithms. While in primitive methods
accuracy of an detected object is given filters are hand-engineered, with enough
in terms of confidence. training, CNN have the ability to learn
these filters/characteristics. The sensor of
Deep Convolutional Neural Network: agriculture field deployed in [4] is used
In deep learning theory, CNN is one of for detection of pesticides. we can obtain
the most important model because there the data for detecting the object from the
are many applications that uses CNN sensor which has been deployed by using
model for image recognition and [5]. [6] cross layer mechanism can be
classification purpose [7]. It is introduced utilized to reduce the computational
from artificial neural networks. CNN uses power and complexity of the complex
the back- propagation algorithm to update data sets.
the different parameters. In CNN model
previously used layer or an output layer is III. Proposed Work
In these research we have used an edge
used as the input layer for coming next
device for measurement of real-time
layers. The CNN algorithm contains many
performance of object detection for that
network layers, that takes colored image
we have selected the NVIDIA’s JETSON
as the input, and that colored image is
NANO. [8] In NVIDIA Jetson Nano is
converted into grey color because the
the smallest offering for specialized
average value of each pixel is contributed
SOCs for AI applications, it offers a CPU
to convert into grey color. Grey color
and GPU integrated into the SOC and has
intensifies the image for pre-processing
acomputational power of 472 FLOPs, at a
image. CNN model performs many
power draw of 5 Watts. These
strategies some of them are pooling,
specifications fall within what one might
convolution, dropout and convolution all
reasonably consider for a mass-available
these are used to improve the fault
edge device. In These era there is a
tolerance. A Convolutional Neural
rapidly increasing demand for fast and

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 120

Electronic copy available at: https://ssrn.com/abstract=4286087

accurate object detection algorithm that Jetson Nano to be clear, the problem that
can run on edge devices in real-time. To object detection intended to solve it
solve these problem, we have tuning involves identifying location the object
deep learning architectures to obtain an is, and what actually the object is.
optimal balance between performance However, solving this issue is not so easy
and accuracy where it is a most popular [9]. where a computer vision processes
research field over the past few years. So images in multiple dimensions.
we tried to solve this issue by deploying Furthermore, the size of the object, its
object detection models onto an edge orientation in the space, its attitude, and
device like jet son Nano for measurement its location in the image can all vary
of real-time object recognition greatly.
performance. We selected NVIDIA’s

Figure 1: Object Detection steps

• Capturing the image from pictures and the quality of the image so that we
also from a live video stream and can analyses it in a better way.
converting those pictures into grey • Processing are the steps taken to
image is very important because it is format images before they are used by
an intuitive way to convert a color model training and inference.
image 3D array to a grayscale 2D Data Flow Diagram: The basic steps that
array is, it takes the average of red, are performed in First Data flow
blue, green pixel values to get a diagrams is to capture an input image and
grayscale. [10] Each pixel implies the the performing some kind of algorithms
intensity details of the light or different models such as in these paper
• Pre-processing: It is used to improve we have used the Jetson Nano and after

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 121

Electronic copy available at: https://ssrn.com/abstract=4286087

then recognition of that particular object with the database that we havecollected.

Recognition: The processed image is

retrieved from the stored data I.e. from
database and then both the images are
compared If both the image comparison
rate is maximum then it passes it into
Figure 2: Data flow diagram
nextstage I.e. testing.

Preprocessing: Here captured image is

passes through the preprocessing stage so
that the image turns into grey color and
the internal grids/pixels divides and
search into multiple different directions
in different dimensions.

Figure 4: Detecting the object

As shown above in figure 4 detecting of

the object such as fruits and remote by
using Jetson Nano in the research lab.

Testing: The final process it tests the

compared image if that image is found
then it gives it as a Matched along with
these it also givesthe accuracy rate of that
Figure 3: preprocessing particular objects. If in case the tested
Processing: As stated above these object image is not found in database, then it
using the back propagation method so gives as Not Matched. It identifies the
for the output of previous stage is used as remote and a person from a live video
in input an another stage. The previous stream. Not only a single image is
preprocessing details are taken input for detected here we have made these
these stage and some algorithms are project such that multiple objects can be
performed here e.g. R- KNN It checks identified/Detected at once only I.e.
the distance between different objects in Sequence of Images are recognized.
a single image and then comparing it

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 122

Electronic copy available at: https://ssrn.com/abstract=4286087

 purpose in their detections. The
ImageNet and the Detection Net
techniques that we have used in our
project it plays a major role of having
image database where are these image
data is used to compare an object final
stages of process. All the requirements
Figure 5: Detecting the object from live streaming
are achieved for detection such as pixel-
As shown above in figure 5 It detects the
level detection and for the process. The
mobiles and a person in the image.
Jetson Nano developer kit is a small
Therefore, the proposed algorithm work
accelerated kit platform it optimizes the
successfully in detecting and recognizing
problem and to avoid the computational
the object from live video streaming. we
overload Jetson Nano uses a low FTS of
can also consider taking live streaming
10 and it can use for detecting the object
video from various form of mobility
from images and live streaming using
sensors as mentioned in this research
the proposed mechanism.
article [11].
VI. FUTURE WORK
To use the data provided by this project to
V. CONCLUSION
Object detection and recognition are one be utilized for the surveillance and
of the most important contents of AI attendance of student and faculty instead
techniques where it has several of taking it manually. It can also be used
applications in automated electric cars for detecting the number of resources in a
because the camera sensor has the particular location.
recognize the object and it as to pass the
whole details that are captured to the
REFERENCE
[1] zhong-qiu zhao, member, ieee, peng
cloud and from the commands have to zheng, shou-tao xu, and xindong wu, fellow
be passed then only electric cars will be object detection with deep learning: a review,
safer without keeping in dangerous ieee ieee transactions on neural networks and
situation as accidents. It is one of the learning systems 2019

basic application where object detection [2] jun deng1,a, xiaojing xuan2,b, weifeng
wang3 , zhao li4 , hanwen yao5 , zhiqiang
is used. It can be also used in traffic
wang6 a review of research on object
systems, hospitals and in robotics. This
detection based on deep learning journal of
technique is mainly used in real-world
physics: doi:10.1088/1742-
because a lot projects are depending on
6596/1684/1/012028
object detection for higher resolution [3] ajeet ram pathak, manjusha pandey,

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 123

Electronic copy available at: https://ssrn.com/abstract=4286087

siddharth rautaray, application of deep and advanced technology (ijeat) issn: 2249 –
learning for object detection, procedia 8958, volume-8 issue-2, december
computer science,volume 132, 2018,pages [7] https://towardsdatascience.com/yolov5-
1706-1717, object-detection-on-nvidia-jetson-nano-
[4] Mohammed nazeer 1*, garimella rama 148cfa21a024
murthy 2, aishwarya jain3 “energy efficient [8] https://github.com/dusty-nv/jetson-
clustering in wsn using weighted centriod)” inference
international conference on soft computing [9] https://digitalenvironment.org/jetson-
and signal processing aug:21-22 springer, nano-object-detection-and-image-
goggle scholar, scopus. classification/
[5] Garimella rama murthy 1 mohammed [10]https://www.forecr.io/blogs/ai-
nazeer 2, padmalaya nayak 3, “energy algorithms/how-to-run-nvidia%c2%ae-
efficient design of mobile wireless sensor jetson%e2%84%a2-inference-example-on-
networks: constrained clustering”. forecr-products-installation-detectnet
international journal of innovative technology [11] Mohammed Nazeer, G Rama murthy,”
and exploring engineering (ijitee), scopus, Protocols in mobile cognitive sensor
may2019. networks” International Journal of Applied
[6] Mohammed nazeer, garimella rama Engineering Research ISSN 0973-4562 Vol.
murthy cognitive cross-layer, energy efficient 13, Number 12, pp: 10268-10275, 2018.
mac protocol in mobile wireless sensor
networks international journal of engineering

Vol 11 Issue 10, Oct 2022 ISSN 2456 – 5083 Page 124

Electronic copy available at: https://ssrn.com/abstract=4286087