Special Issue of Multidisciplinary Sciences and Advanced Technology

Received: 15 August, 2022, Accepted: 08 October, 2022, Online: 31 October, 2022

DOI: https://dx.doi.org/10.55708/js0110008

The Current Trends of Deep Learning in Autonomous

Vehicles: A Review
Jing Ren*,1 , Raymond Ning Huang 1 , Jing Ren 2 , Hossam A. Gabbar 3
1Department of Mechanical Engineering, University of Toronto, Toronto, M5S 1A4, Canada
2Department of Electrical and Computer Engineering, Ontario Tech University, Oshawa, L1H 7K4, Canada
3Department of Energy and Nuclear Engineering, Ontario Tech University, Oshawa, L1H 7K4, Canada

*Corresponding author: Jing Ren, 2000 Simcoe Street N. Oshawa, ON, Canada, jing.ren@uoit.ca

ABSTRACT: Autonomous vehicles are the future of road traffic. In addition to improving safety and
efficiency from reduced errors compared to conventional vehicles, autonomous vehicles can also be
implemented in applications that may be inconvenient or dangerous to a human driver. To realize
this vision, seven essential technologies need to be evolved and refined including path planning,
computer vision, sensor fusion, data security, fault diagnosis, control, and lastly, communication and
networking. The contributions and the novelty of this paper are: 1) provide a comprehensive review
of the recent advances in using deep learning for autonomous vehicle research, 2) offer insights into
several important aspects of this emerging area, and 3) identify five directions for future research. To
the best of our knowledge, there is no previous work that provides similar reviews for autonomous
vehicle design.

KEYWORDS: Deep learning, Autonomous Vehicles, Control

1. Introduction safety and performance in control and path planning. In

effect, the use of more data can improve system safety and
Autonomous vehicles are the future of road traffic. As performance. A commons strategy deployed to increase
intelligent agents equipped with sensing technology the amount of data AVs has access to is facilitating vehicle-
including GPS, Inertial Navigation Systems (INS), Lidar, to-vehicle (V2V) communication, allowing vehicles to
and Cameras combined with advanced control systems, have access to data from more sensors and more accurate
autonomous vehicles have important applications in the state information of surrounding vehicles. Beyond cellular
future. In addition to improving safety and efficiency from V2V communication, it is also important to consider the
reduced errors compared to conventional vehicles, issue of bandwidth efficiency and the application of
autonomous vehicles can also be implemented in unmanned autonomous vehicles (UAVs) and other
applications that may prove to be difficult or unsafe for a technologies to facilitate communication. However, as
human driver. communication increases and deep learning is
However, before they can be safely commercially increasingly used more in Avs, it is essential to ensure user
introduced, several essential technologies necessary for data is secure and the vehicles are resistant to any
the design and operation of autonomous vehicles must be malicious attacks. Because autonomous vehicles require
developed and refined. The function of an autonomous accurate sensory and system information, it is especially
vehicle can be simplified to be a vehicle that can plan and relevant to detect and diagnose faults. In this paper, we
follow a safe and efficient path from given starting points will consider seven key technologies needed for
to endpoints with some control constraints. To achieve this autonomous vehicles: path planning, computer vision,
goal, it is important to have strong understanding of sensor fusion, data security, fault diagnosis, control, and
human visual system and plan the substitution with an lastly, communication and networking.
effective visual sensor system with robust data analytics. While conventional methods have been tried for these
Beyond simple visual data, other sensory data such as technologies, deep learning is arguably the most
location and inertial data can be integrated to improve promising method. This is due to deep learning’s ability to

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 56

 J. Ren et al., The Current Trends of Deep Learning

accurately approximate complex nonlinear relationship steering of autonomous angles. The authors mapped input
using multi-layer transformations. The difference is camera data directly to a steering angle to implicitly solve
especially highlighted in imaging tasks such as object path planning tasks. More commonly, CNN is applied to
recognition and image classification. The key types of extract features to be applied in a path planning subtask.
deep neural networks are Convolutional Neural Network In one study, an unmanned aerial vehicle (UAV) was
(CNN), Deep Autoencoder (DAE), Deep Belief Network tasked with generating a path through 26 gates in an
(DBN) and Deep Reinforcement Learning (DRL). DBNs indoor 3-D environment. In this task, CNN was employed
and DAEs can conduct unsupervised pre-training on the to detect the center of a gate to enable the guidance of the
weights, which can ease the difficulty of the subsequent vehicle without collision in real time [4]. CNN can also be
supervised training of the deep networks. However, a applied to tasks of higher difficulty beyond simple 2-D
fundamental problem in DBNs and DAEs is that there are and 3-D single vehicle path planning tasks. In [5], the
too many weights to train when the inputs are raw signals authors applied CNN in a path planning task of guiding
or their time-frequency representations. multiple UAVs in a 3-D environment using 2-D CNNs.
In contrast, CNN can avoid these issues by using LSTM is another type of network used to process image
strategies like local receptive field and weight sharing to data. As a recurrent neural network, it is often used for
reduce computational complexity during training. sequential image data. In [6], the authors use a model
However, CNN has a relevant disadvantage which is its where a LSTM network is applied to extract hidden
tendency to fall into a local minimum in training. The last features to aid in accurately planning sequential moves of
key network, Deep Reinforcement Learning (DRL), can AVs. In further works, LSTM has also been successful
achieve exceptional results using a Q-learning or policy applied for path planning in an environment with
gradient algorithm to gain the best rewards for the chosen vulnerable road users which includes smaller vehicles
actions. such as bicycles and motorcycles as well as pedestrians. In
another study, the authors use a model-free planning
Our paper is an extension of “Applying Deep Learning
approach with a deep stacked LSTM network to assess
to Autonomous Vehicles: A Survey” [1]. We will review
pedestrians’ intentions and plan a vehicle’s motion
the headway in deep learning algorithms to review these
accordingly [7].
seven essential fields. Some relevant reviews have been
published in recent years [2], where some new functions The last primary deep learning tool is deep
are discussed. The main contributions of our paper are: reinforcement learning which does not rely on labelled
data sets and can achieve extraordinary result. In the
1. New data security, and communication and
recent past, DRL has been commonly used in path
networking sections.
planning applications for various types unmanned
2. Reviewed most recent papers for path planning, vehicles in diverse environments. The first application
computer vision, sensor fusion, fault diagnosis, and involves navigation for road vehicles in an environment
control. with mixed autonomous and traditional vehicles. In this
3. Identified five directions for future research application, DRL is used to generate a shareable driving
policy which doesn’t need to consider facts such as system
2. Deep Learning for Path Planning dynamics compared with traditional control methods [8].
DRL has frequently been applied to path planning
In an ordinary environment, path planning aims to
problems in environments with cluttered obstacles and
guide a vehicle on a collision-free paths for where both
rough terrain. With an application for robots employed for
static and dynamic obstacles need to be avoided. This
urban search and rescue missions, the authors in [9]
method can either have a model-based or model-free
proposed a path planning method using DRL with the
design and include local or global planning and
input of depth images, elevation maps, and orientation to
optimization for some criteria. Models usually aim to find
generate navigation actions. In [10], the authors apply
the shortest time or shortest path method. In the following
DRL to address the issues of path planning in large
section, we will review recent literature on path planning
complex environments where simultaneous localization
for autonomous vehicles and the three primary deep
and mapping (SLAM) and other conventional methods
learning tools applied., i.e. DRL, CNN, and Long Short-
are less effective and have reduced accuracy due to
Term Memory (LSTM).
computational constraints. Thus, DRL is used to directly
CNN is commonly used for classification tasks due to map sensor input data to output control system directives.
its ability to extract features from images. While image While conventional control strategies for aerial vehicles
classification problems are different from path planning are somewhat mature, there is still reliance on human
tasks, CNN can be used when generating control signals intervention. Deep learning methods are promising
from sensor data from sources such as cameras and lidars. approaches for the control of UAVs. The first of these
In [3], a high-level control framework was created for the
www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 57
 J. Ren et al., The Current Trends of Deep Learning

methods involves using a deterministic policy gradient categories, research works on environmental perception
method in path planning for multiple aerial vehicles. This generally focus on cameras due to its lower cost and mass
model can achieve real-time performance in a dynamic availability. Cameras are further divided into two
environment due to its parameter requirements of only the common types, the first of which is a monocular camera
locations of threat areas, targets, and other UAVs [11]. with the ability to extract precise information from images
With applications in search and rescue missions as well as in the form of pixel intensities. to the arrangement of pixel
aerial inspections for oil and gas fields, the authors in [12] densities can be used identify to identify properties such
propose a path planning strategy of UAVs based on a as texture and shape. However, monocular cameras have
Nokia Snake game strategy. This control policy can be inadequate accuracy in estimating the size and position of
applied to plan more complex paths than comparable an object because of the limited depth data available in a
previous DRL methods. In [13], path planning for drones single image. Thus, a stereo camera system is often used
using a DQN method which combines deep learning with to improve depth estimation. Beyond monocular and
time-difference learning. The authors used a drone stereo cameras, there exists other more specialized
equipped with ultrasonic sensors in a 3D simulation cameras, such as Time-of-flight cameras, which can
environment. In [14], a similar path planning problem accurately estimate depth from the delay between
using the DQN method was used to conduct local path transmitting and receiving infrared range pulses [19]. An
planning of unmanned surface vehicles equipped with integral part of object detection is distance estimation.
360-degree lidar in a 3D simulation environment. While conventional methods have been applicable to only
a singular type of camera, the authors in [20] have
In recent years, mobile edge computing, where
designed a depth perception model that can be
computations are done locally at an edge node of a
generalized to a variety of camera with varying camera
network, has been developed. In [15], a UAV-mounted
geometries.
mobile edge computing system is developed to
dynamically take and compute tasks from mobile terminal Weather and light conditions can negatively influence
users. With terminal users starting at random locations the accuracy of camera sensors, especially at night or in
with random travel paths, the authors apply DDQN to snowy and rainy weather, where calculations such as
optimize path plans with time- varying targets. In [16], the depth perception become more complex. In [21], the
authors innovate by applying a multi-agent system to the authors designed a method to address detecting vehicles
mobile edge computing network path planning task. at night with a monocular camera. This was accomplished
Using energy consumption, distance, and computational using support-vector machines to detect headlight and
intensity of each task as parameters, the proposed taillights. While this is an improvement, cameras are still
algorithm is a form of a multi-agent DDPG utilizing a not reliable when used as the only sensor, and lidar still
framework of centralized training and decentralized provides the most accurate measurement during night-
execution. An alternative approach to this problem is time. Object detection and identification is necessary for
shown in [17], where a multi-agent DQL-based algorithm the safe operation of autonomous vehicles, and it is
is used. In this study, each UAV is trained with an essential to balance the costs of lidar and the reliability of
independent DQN and autonomously executes its actions, a camera.
only receiving state information from each UAV. In [18],
To avoid accidents, it is crucial for autonomous ground
the authors use a mobile edge computing network to
vehicles to detect domain of the path it takes and any
tackle the problem of traffic. This study claims that one
objects along it. Thus, the first step in detecting vehicles,
major issue in traffic and higher fuel consumption in an
pedestrians, and other obstacles on the road. In [22], the
environment with autonomous vehicles can be solved by
study conducted proposed a road detection model using a
having cars travel in a platoon model by reducing changes
CNN network with residual learning and pyramid
in speeds and increasing aerodynamic flow. A path can be
pooling techniques using monocular vision data. In [23],
planned using Q-Learning to optimize speed and fuel
a method incorporating CNN is proposed to detect
consumption.
various types of speed bumps and solve the problem of
3. Deep Learning for Computer Vision the dynamic appearance of speed bumps. Potholes present
another problem and the authors in [24] tackles this
Machine learning has always been crucial for problem by automatically developing a strategy to detect
environmental perception and computer vision, and their potholes using information from stereo cameras.
applications such as autonomous vehicles. Deep learning
methods have improved on their understanding of sensor In the operation of vehicles, accidents most commonly
data and to some extent, are able to accomplish perception, occur at road intersections. Therefore, it is critical to obtain
localization, and mapping. Sensors are divided into two information from sensors or connected technologies of
categories: passive sensors such as cameras and active nearby road agent positions at intersections. In [25], the
sensors such as lidar, radar, and sonar. Between these two authors conducted a study about the effectiveness of

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 58

 J. Ren et al., The Current Trends of Deep Learning

various deep neural networks on autonomous vehicles With eco-friendliness and fuel costs taking greater
learning road vehicle information from aerial importance, research has expanded more on optimizing
photographs. This can have applications in a smart city driving patterns to reduce fuel consumption. Applying
where connected technologies enable the sharing of aerial CNN and computer vision, the authors [32] propose an
photography data between road agents. Later in this object detection method that increases the fuel efficiency
review, we will discuss using wireless connections of hybrid vehicles. In a test, they achieved 96.5% fuel
between vehicles to share various data that may have economy of the global optimum with dynamic
applications for road intersection safety. programming, increasing fuel efficiency by up to 8.8%
over an existing method. In [33], the authors propose a
Beyond detecting obstacles and nearby road agents at
different approach to reduce fuel consumption. They
intersections, it is also crucial to be able to detect and
proposed a DQN-based car-following strategy and a
interpret traffic control signals. Traffic Light Recognition
learning-based energy management strategy to achieve a
(TLR) techniques are comprised of two steps: detecting
low fuel consumption while maintaining a safe real-time
traffic signals and estimating the state of the signal. Some
distance. In [34], they tackled fuel efficiency for fuel cell
key challenges are addressing false positives and
vehicles using a spatiotemporal-vision-based deep neural
computational complexity while under dynamic lighting
network. This method improved the accuracy of predicted
conditions. One proposed TLR method uses a multi-
speed, especially when the traffic was dense.
channel high dynamic range (HDR) camera to capture
images with multiple exposure values where deep neural 4. Deep Learning for Sensor Fusion
networks are used to estimate traffic light signals. In this
study, the network detects traffic lights from bright frame Autonomous vehicles are typically equipped with
data and classifies the signal using dark frame data [26]. In multiple sensors such as Global Positioning System (GPS),
another study, a TLR method with using a video dataset Inertia Measurement Unit (IMU), cameras, radar,
input with six color spaces was proposed. The authors ultrasound, as well as light detection and ranging (Lidar).
applied three different networks based on region-based While each sensor provides key data, they each have their
deep learning network models with the best result limitations. However, by combining the strengths of each
attained with pairing of RGB color space with a R-CNN sensor, together, they can provide autonomous vehicles
model [27]. with superior information to render decisions for control,
path planning, and fault management.
In object detection and identification, key targets for
autonomous vehicle systems to reliably detect are Traditionally, sensor data was fused using the Kalman
passengers and pedestrians. In one approach, the authors filter algorithm. However, deep learning has become an
of [28] combined RGB-D stereo vision and thermal increasingly more popular method of combining sensor
cameras. In this study, the authors compared Histogram data due to its effectiveness and relative simplicity. One
of Oriented Gradient (HOG) and Convolutional Channel key application in sensor fusion is overcoming the
Features (CCF) methods with the results indicating CCF shortfall of cameras. While cameras are used to capture
superior to HOG for pedestrian detection. In [29], a important data such as object size and shape, it lacks the
vision-based system using monocular camera data was ability to accurately measure key values such as distance
developed to predict passenger movement and to detect and velocity, that sensors like radars and lidars can
other objects on the road. This study used a party affinity compensate for. In [35], the authors explore the application
fields model to estimate the pose of pedestrians in of a CNN network in the fusion of camera raw pixels and
combination with AI to aid in estimating results for risk lidar depth values to generate a feature vector. This study
assessment. In emergency situations, it can be important employed a novel temporal-history based attention
for autonomous vehicles to detect nearby vehicle mechanism which proved to be resilient to errors in sensor
passengers to make more informed decisions. This issue is signals. To address the lack of clarity in camera data in
addressed in [30], which proposes a CNN-based method severe weather conditions and at night, the authors in [36]
for detecting nearby cars and passengers within those cars conducted a study fusing camera and radar data with a
using monocular cameras. model based on RetinaNet. In [37], the authors propose a
vehicle detection model based on the fusion of lidar and
With respect to traffic and road rules, CV-based deep camera data. In this model, possible vehicle locations are
learning techniques can also be applied to detecting obtained from lidar point cloud data and a CNN network
vehicle road violations. In [31], YoloV3, a real-time object- is used to refine and detect vehicle locations. Beyond
detection algorithm for detection and tracking integrates a detecting vehicles, lidar camera fusion was applied in [38]
license plate recognition system with LPRNet and for high accuracy road detection even in difficult road
MTCNN. Using this, they can track traffic violations and conditions such as in extreme weather. In sensor fusion
impolite pedestrians. and deep learning for segmentation tasks, the quality of
training data plays a vital role in these studies' success. In
www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 59
 J. Ren et al., The Current Trends of Deep Learning

[39], a new collaborative method of collective multi-sensor these models also inadvertently include auxiliary that
training data and automatically generating accurate labels malicious actors can use to infer information about the
is proposed. individuals such as location and trajectory. In [49], the
authors use a GAN to generate privacy-preserving data
In [40], the authors propose a deep convolutional
that still retain its usefulness in training.
network for vehicle detection with three modalities: color
image, lidar reflectance map, and lidar depth map. This In recent times, federated learning has been
model is able to produce a more accurate prediction using introduced, where a model is downloaded and trained
joint learning because joint learning generates more data locally with private data before being uploaded, and
for safely operating vehicles compared to learning model aggregation occurs. It became popular due to
environmental data and driving policy independently. In offloading some computational power to individual
[41], the authors propose a dual-modal DNN to create an devices, and privacy concerns as users don’t have to share
improved detection model in severe environmental their private data [50] [51]. In federated learning, two
conditions such as rain, snow, and night-time where problems being tackled are detecting bad actors that
features can be blurry. This network fuses color and maliciously upload faulty data to distort the accuracy of
infrared images and achieves improved performance for the model and preserving privacy before model
low-observable targets. In [42], the authors proposed an aggregation. In [52], the authors use a blockchain method
Integrated Multimodality Fusion Deep Neural Network, for UAVs, which replaces the central curator to combine
which processed each modality independently before the learned parameters in the model. In addition, they
being processed together in further networks, which also use a local differential privacy algorithm to mask
creates modularity and increased flexibility, thus personal data. In [53], the authors propose a different
providing a greater ability for generalization. method to tackle this problem. They introduce a privacy-
In [43], a cooperative perception system was proposed preserving model aggregation scheme named FedLoc by
to expand the scope of vehicle perception and eliminate using homomorphic encryption and a bounded Laplace
blind spots by integrating data from multiple vehicle mechanism. In [54], the authors introduced CLONE, a
sources using both graphic and semantic alignment. In collaborative edge learning framework using federated
[44], the authors introduce a cooperative visual-free sensor learning techniques. This can be applied to a multitask
fusion technique combining vehicle detector, remote tracking problem or in EV battery fault detection. In [55],
microwave sensors, and toll collection data to predict fine- the authors tackle the problem of sharing data for model
grain flow traffic. In [45], the authors presented a model training for data collected by independent
integrating various smartphone sensor data to detect real companies/vehicles by using federated learning. A
time vehicle maneuvers. This system uses GPS, gyroscope, blockchain structure can also be used to enhance privacy.
accelerometer, and magnetometer to detect turns and In [56], a hybrid blockchain architecture is proposed to be
other movements to be communicated to enhance safety. used in federated learning for vehicular applications. In
In [46], a smartphone sensor-based method is proposed to [57], the authors also use blockchain to facilitate
detect human activity recognition. In [47], the authors communication in a mobile edge computing application,
propose Gated Recurrent Fusion Units (GRFU) which however, with the innovation of applying a hybrid model
have gating mechanisms similar to those in LSTM to create intrusion detection system to the data. Their proposed
a new joint learning mechanism, which proved to have an framework has shown reduced false alarm rate and a high
improved error rate. In [48], a novel end-to-end driving accuracy of 99%.
DNN is proposed. This proposed network that In addition to data-privacy, it is also essential to
incorporates scene understanding, which understands detect and prevent malicious attacks. These attacks can
spatial, functional, and semantic relationships and uses corrupt the model during training or operation by
lidar and camera. providing malicious input. One example of an attack
5. Deep Learning for Data Security using adversarial GPS trajectories against crowdsourced
navigation systems is Cybil attacks. In [58], a Bayesian
With the rapid development of deep learning, deep learning method was used to identify Sybil attacks.
increasing amounts of user data are required to train and In [59], the authors discuss the development of poisoning
models. In some systems, the privacy of user data can be and evasion attacks and review recent methods used to
a concern. In addition, it is also essential for systems to address them. The methods used against poisoning
accurately detect attacks with intentions to corrupt the attacks include ensemble learning methods to increase
model. Current data security methods will be discussed resistance against variance and comparing classifiers for
for centralized and federated learning models. Model each training data set. Adversarial training is the main
training requires a large number of data samples. In method used against evasion attacks which works by
addition to the information required to train the models, introducing both legalized and adversarial samples to

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 60

 J. Ren et al., The Current Trends of Deep Learning

train the model on detecting adversarial samples. In [60], load system. A deep autoencoder-based clustering system
the authors proposed a multi-strength adversarial and a CNN-based classification method is used to process
training technique which combines adversarial training high-dimensional signal data to detect and classify faults.
examples with different adversarial strengths. In [66] a combined CNN and LSTM model is used to
detect the pre-ignition of engine control signals using in-
6. Deep Learning for Data Security
vehicle data. In [67], the training data are generated from
As the field of autonomous vehicles develops, these the UAV system model. One dimensional signal is then
vehicles require more sensors and actuators and become extended to time-frequency domains using wavelet
increasingly reliant on them. However, the proliferation transform. Then, deep learning is performed on the image
of sensors and actuators in vehicles which heavily rely on data to find different sensor or actuator faults.
their accuracy also increases the occurrence of a vehicle Electric vehicles have microgrids that encompasses
fault. To ensure the safe operation of autonomous energy storage systems, electric motor, motor drive and
vehicles, fault detection methods need to be able to be protective components. With complete reliance on micro-
more reliable. Conventional fault detection methods can grids, detecting faults in the micro-grid of an electric
be classified into three categories: model based, signal vehicle is crucial to the safety of the vehicle. In [68], a
based and knowledge based. In recent years, DNN-based CNN-based method was studied to detects false battery
fault detection has become popularized as it can achieve data in battery energy storage systems, with application
faster and more accurate results. In addition, deep to those in electric vehicles. In [69], the authors used a
learning can accurately map complex patterns and signals CNN-based model to solve the fault classification
to accurately assess the health condition of the problem for micro-grids. This method uses voltage and
components, leading to its prospects of becoming a other measures from inverters, converter, capacitors to
promising research field. In recent years, CNN, DAE and create a fault detection method to reinforce traditional
DBN have all been applied in fault diagnosis tasks. methods. It is especially important to reliably detect
Planetary gearboxes are commonly used in faults in unmanned autonomous vehicles due to the
mechanical systems such as transmission systems present higher cost of failure. In [70], the authors presented a
in ICE and some electric vehicles. Using a deep residual strategy for diagnosing faults in actuators of multi-rotor
network with vibration signals as input, the authors in [61] UAVs based on a hybrid LSTM-CNN model. One
created a model integrating the network with domain common constraint in UAVs is the challenge of running
knowledge to identify faults and the condition of complicated fault detection methods in real-time, which
planetary gearboxes. An alternate approach is taken to have size, weight, and power consumption constraints.
detect faults in a planetary gearbox in [62] where a model To tackle this problem, [71] proposes an LSTM-based
is formed with transfer learning combined with a deep fault detection model acceleration engine. In [72], the
autoencoder with wavelet activation functions. The authors use LSTM to estimate the estimated wheel angle
resulting model is effective under variable conditions and an improved sequential probability ratio test to detect
such as changing speed and location. a fault in vehicle wheel angle signals.

In-vehicle gateways are modules that connect to and In the future, connected and automated vehicles
receives data from various sensors in a vehicle. Several communicating in real-time are expected to improve road
studies have been conducted to utilize this information safety. In [73], the problem of anomalous sensor readings
for fault diagnosis. In [63], the authors combined a LSTM is tackled with a CNN-based sensor anomaly detection
network with an in-vehicle gateway to diagnose faults and identification method. In [74], the authors address
based on fault data by using comparisons with previous detecting malicious actors in connected and automated
sensor data. In [64], an IoT Gateway combined with deep vehicles during cruise control. Using a multi-agent DRL
learning is used to diagnose the faults of the sensors. This method, they can cooperatively and accurately detect
self-diagnosis information can be used for self-repairing. attackers.
The inputs of the deep learning network are sensor 7. Deep Learning for Control Algorithms
signals, and the outputs are the condition of parts, of
which the driver will be informed through diagnostic Autonomous vehicle control models consist of two
results. One innovation of the work is that data collected parts: perception planning and control paradigm.
by a gateway are from different protocols such as CAN, Traditionally, control systems methods relied on
FlexRay, and MOST. mathematical models including optimal control, robust
control, PID control, and adaptive control. While
Deep learning can be used to detect faults in many
conventional models are more easily interpreted and
components of the vehicle. In [65], electrical signals are
have a theoretical foundation, they performance worse
analyzed to detect the fault in the spacecraft's electronic
for more complex data or larger data sets. In comparison,

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 61

 J. Ren et al., The Current Trends of Deep Learning

deep learning control approaches are model- free, data- underwater vehicles relying on conventional controllers
driven which indicates its applicability to both discrete have imposed paths and pre-planned tasks. In [80], the
and continuous systems. Due to these differences, deep authors present a model based on deep interactive
learning can’t be directly applied in conventional models. reinforcement learning to facilitate path following. This
Instead, contemporary deep learning solutions for model uses a dual reward method by which the network
autonomous driving employs end-to-end controllers to can learn from both human and environmental feedback
improve or provide state estimation. simultaneously. In [81], the authors investigated a low-
level DRL-based control strategy for underwater vehicles.
Recently, deep learning has been the chosen method
A deep reinforcement learning network is introduced
for the state estimation of different controllers [75] and
with sensory signals serving as the sole input of the
enhancing state estimation quality [76]. In control systems,
network without prior knowledge of vehicle dynamics. In
the dynamic model that identifies the uncertainties and
[82], a deep learning tracking control algorithm is applied
hidden states form the foundation. However, the
to improve the accuracy and adaptability of driving
conventional system identification method does not
trajectory tracking. In [83], the deep learning network is
easily identify model parameters. Helicopters require
used to analyze the environment to predict lateral and
complex control systems due to having complex
longitudinal control. In this network, two separate
interactions with external forces and internal controls. In
models are used for vehicle speed and steering, where the
[77], the authors propose a deep convolutional neural
inputs are road images, and the outputs are the speed and
network-based dynamic identifier for a controller. This
steering. In [84], the authors propose to use a CNN
scheme can accurately identify the helicopter’s dynamic
network as an end-to-end controller for driving, while
behavior, maintain stability even in untrained maneuvers.
two other CNN-based deep networks generate both the
In [75], the authors present a flight control method for
feature map and error map to help the controller better
autonomous helicopters. A deep learning network is used
understand the scene. An attention model is used to
as the identifier for an adaptive control scheme. The
identify the regions that affect the output most. In [85], the
complete model includes a first principle-based dynamic
authors have explored the use of reinforcement learning
model and a CNN based model for modelling hidden
for high-level decision-making in the context of a robotic
states and uncertainties where all parameters and weights
game. In this hierarchy structure, the high-level DL is
are trained with real flight data. In [76], a deep learning
combined with low-level controllers to deliver a better
network with the drop out technique is used to improve
control performance. The design of the controller with
the performance of the attitude state estimation by the
multiple levels can accommodate the challenges in the
Kalman filter. This network is trained to model the
game, such as action delay. In [86], DRL was applied in
measurement noises, which in turn is used to filter out the
intelligent control with a self-organizing control system
noise and enhance the quality of the Kalman filter. Deep
based on DDPG. Using simulations, the reference signal
learning is used to compensate for delays and
self-organizing control system was able to stabilize an
measurement noises. The information extracted by a
inverted pendulum using a rotor. Autonomous vehicle
modular deep recurrent neural network is combined with
control systems often have trouble with hard to predict
sensory readings before being fed into the Kalman filter
actions such as cut-in maneuvers. In [87], a control
for state prediction and update. This deep learning
strategy is developed using a two-part training strategy
network can detect the hidden states, which are normally
of experience screening followed by policy learning to
difficult to be measured by sensors. In [78], a CNN and
increase performance in uncertain scenarios.
LSTM-based observer is presented. First, LSTM processes
videos and adds a temporal dimension to the cost map. Deep learning can also be applied to larger-scale
Then a particle filter is used for state estimation. Lastly, control algorithms such as wide-range traffic control and
the cost map generated by the deep learning network is power grids. Inefficient traffic control results in more
combined with readings from IMU and wheel speed stop-and-go traffic, increasing wait times and fuel
encoders to predicate and update the states for model consumption. In [88], RL is applied to adaptive traffic
predictive control. signal control. This is achieved by using multi-agent RL
that distributes global control to each local agent with a
Deep learning can also replace conventional discrete
new decentralized multi-agent RL algorithm that has
controllers such as PID controllers and instead use a deep
improved observability and reduced the learning
learning model to generate output control actions which
difficulty of each agent. In [89], the authors present a more
can be either discrete or continuous [79]. Another
centralized method of traffic control. Using information
application combines the learning with the conventional
about vehicles near a particular intersection, including
controller to form a hierarchical or better control system.
speed and location as input, they train a model that
Oceans and other large bodies of water have complex
controls the duration of traffic signal timings to reduce
environments and as a result, existing autonomous
vehicle wait times and trip lengths.
www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 62
 J. Ren et al., The Current Trends of Deep Learning

In the electrical grid, conventional control systems are especially in depopulated areas. However, satellites have
not well optimized and do not adapt to changes well. limited computing and communication resources. To
Although with high difficulty and complexity, AI has tackle this issue, the authors in [99] used deep learning
been applied in power grid control, which will be an with the Lagrange multiplier method to improve joint
important research front for future autonomous vehicles. task offloading and resource allocation. Maritime
A novel two-timescale voltage control system is communications are often bottlenecked by the immense
presented in [90]. Using a feed-forward DQN with data volumes required. In [100], the authors propose a
physics driven optimization, a two-timescale approach transmission scheduling strategy based using a deep-Q
was able to minimize voltage deviations and optimize network. This strategy optimizes the network routing.
power flow. [91] developed a DRL based autonomous
9. Discussion
voltage control for power grid operations. They proposed
to use both DQN and DDPG to create autonomous Autonomous vehicles will significantly impact the
voltage control strategies to better adapt to unknown future of the automobile industry. Fully autonomous
system changes. vehicles can improve safety and travel comfort as smooth
8. Deep Learning for Communication and and consistent driving will reduce congestion. They have
various benefits and advantages as follows:
Networking
• More independent mobility. Better access for people
Autonomous vehicles can play an important role in
who cannot drive, including the elderly and young
communication and networking. In communications,
people.
there is often an inefficient use of allocated bandwidth. In
• Facilitating car sharing and ride sharing. An increase
[92], a deep learning-based channel and carrier frequency
in car-sharing opportunities will reduce the need to
offset equalization technique is proposed to improve
own a car and associated costs.
bandwidth efficiency. In emergencies, it is common for
• More efficient vehicle traffic. Reduces congestion
base stations and power sources to be destroyed,
and roadway costs due to more consistent behavior
restricting access to communication networks when
on the road.
needed. As UAVs evolved, they have been purposed to
• Fewer cars on the road. Reduce drivers’ stress and
assist in emergency communication networks as a base
increase productivity. While traveling, motorists can
station. A fundamental problem being solved is the
rest, play, and work.
optimization of resources, while UAVs are both limited
• Greater safety. Several opinions say that autonomous
by their coverage area and energy consumption. In [93],
vehicles will eliminate 95% of all human error. Thus,
the authors propose a novel DRL method to optimize
autonomous vehicles reduce crash risks and high-risk
energy consumption. In [94], the authors approach this
driving since they are not impacted by human
problem with a DRL based on Q-Learning and CNN to
emotions or bias while driving.
optimize macro base power allocation and UAV service
selection. Recently, vehicular ad hoc networks have been Because deep learning is able to learn or discover very
used in autonomous vehicles for vehicles to improve complex high-dimensional nonlinear patterns or
safety and comfort [95]. However, vehicles often have a relationships from a large amount of training samples,
restrictive communication range. To address this issue, deep learning has been successfully applied in seven
communication between vehicles and other types of research areas in autonomous vehicles. However, we still
devices is used. In [96], a deep learning-based algorithm need to further investigate new techniques to overcome
is proposed for transmission mode selection and resource some limitations associated with most deep learning
allocation for cellular vehicle-to-everything algorithms, including easily getting trapped in a local
communication. In [97], the authors propose using UAVs minimum, slow convergence during training especially
as relays in these networks. Using DISCOUNT, a DRL for deep reinforcement learning, requiring a large set of
framework, an organized and intelligent group of UAVs training samples or overfitting for small training datasets.
are optimized to increase connectivity and minimize
energy consumption. A common issue in cellular- • Future directions. We still need to overcome serious
connected UAVs is interference between each relay. In major challenges before fully autonomous vehicles
[98], the authors propose a deep learning algorithm based are ready for public use. In the near future,
on echo state network architecture to create an autonomous vehicles may be limited to some specific
interference-aware path planning strategy. scenarios, such as narrower situations and clearer
weather.
Beyond UAVs, which have range and power • Employ deep learning to develop new sensor fusion
constraints, satellites have merit as a solution to improve techniques for autonomous cars with different
vehicle-to-vehicle communication on the ground, certain road conditions. Current techniques mainly

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 63

 J. Ren et al., The Current Trends of Deep Learning

focus on narrower good road conditions. They cannot • Early fault diagnosis and prognosis of autonomous
handle more complicated road conditions such as vehicles. Since the occurrence of faults increases due
changing road conditions, night conditions, unlit to the significant increase of sensors and components
roads at night, unmarked roads, even unpaved roads, in autonomous vehicles, early fault diagnosis and
unexpected conditions such as animals suddenly prognosis are more important and more challenging
crossing roads, or combinations of the above to ensure the vehicle safety. We need to investigate
situations. These complicated conditions require new real-time early fault diagnosis methods for more
novel sensor fusion algorithms and probably require complicated scenarios considering not only the
to develop new perception devices. Deep learning is components and sensors associated with the vehicle
more suitable for these very complex scenarios than itself but also the faults or disruption of V2V
conventional methods. We need to investigate novel communications and the reliability of some global
deep learning-based fusion techniques for more information from networking infrastructure such as
complicated road conditions. real-time road conditions. Early fault diagnosis under
• Developing new sensor devices and novel relatively normal driving conditions is especially
algorithms for challenging bad weather conditions. important to discover potential issues at an earlier
Current sensing techniques work relatively well with stage, provide early warning or alert, and take actions
clearer weather. However, like our eyes, vehicle such as timely checkup and maintenance to prevent
sensors do not work as well in bad weather getting stuck in the middle of remote roads or even
conditions such as rain, fog, snow, and ice, which not catastrophic accidents. Since many faults at early
only reduce the visibility but also cause dangerous stage often involve small or subtle changes, it is more
road conditions. Many autonomous cars employ difficult to accurately detect these small anomalies,
Lidar technology using lasers. However, snow and especially under normal driving operations due to
ice absorb laser light rather than reflecting it, making the lack of special expensive instruments in car
these vehicles blind in inclement weather conditions dealers. However, this is a very important research
and making it difficult for Lidar to accurately identify topic for vehicle safety and timely maintenance.
obstacles. Therefore, these bad conditions make • Developing novel low-cost techniques to make
autonomous cars harder to navigate and cause autonomous vehicles more approachable and
potential safety issues for other drivers and affordable. Autonomous cars are currently very
pedestrians alike. They also make conventional costly, which makes investing in them difficult for
processing algorithms more challenging to obtain most people. Even current techniques can achieve the
accurate perception from low quality sensor data in required performance criteria under good road and
bad weather conditions. We need to develop novel weather conditions, we still need to investigate novel
intelligent sensors to obtain accurate perception and effective and efficient algorithms for more
corresponding processing techniques for these bad complicated scenarios where they require more
weather conditions. Deep learning techniques have advanced sensors and more computation power.
the potential to deal with these more challenging However, it is a time-critical mission for autonomous
scenarios. vehicles with rapid response time. We believe deep
• Investigate novel deep reinforcement learning learning will play an important role in developing
techniques to achieve multiple objectives in the novel intelligent technologies for environment
design of autonomous vehicles. The design of fully perception, planning and navigation on challenging
autonomous vehicles often involves multiple, even roads while keeping low cost.
conflicting, objectives or criteria. For example, car
10. Conclusion
connectivity using vehicle-to-vehicle communication
(V2V) communications with surrounding vehicles In this article, we reviewed recent developments in
makes many tasks, such as merging easier, but the area of deep learning applications in autonomous
securing the communication system could be vehicles. These seven active research areas are control,
extremely difficult. Thus, it also increases computer vision, sensor fusion, path planning, fault
cybersecurity risks since there are more ways to get diagnosis, communication and networking, and data
into them and disrupt what they're doing as vehicles security. Several types of deep neural networks were
get more connected. Therefore, we need to investigate reviewed and compared, in which deep learning were
new techniques and strategies in order balance the successfully applied to various design and operation
benefits of using V2V communications and aspects of autonomous vehicles. Deep learning has taken
cybersecurity risks. We believe DRL is one suitable a significant role in the development of technologies for
technique to make the optimal decision for these autonomous vehicles. It will continue to play an
complicated situations.

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 64

 J. Ren et al., The Current Trends of Deep Learning

important role in the future development and refinement We acknowledge the financial support of the Natural
of these technologies. Sciences and Engineering Research Council of Canada
(NSERC).
Acknowledgment
Table 1: Path Planning Solutions

Research paper Application(s) Deep learning Network Comments

method
[3] Unmanned Vehicles ReLU CNN Camera map steering angle
[4] Unmanned Aerial Vehicles - CNN Real-Time

[5] Unmanned Vehicles Imitation Learning CNN 3D

[6] Unmanned Vehicles LSTM LSTM Feature Extraction
[7] Unmanned Vehicles LSTM LSTM Pedestrian Detection
[8] Unmanned Vehicles DRL DDQN Mixed environment with
manual vehicles
[9] Unmanned Vehicles DRL DDPG Unknown Environment
[10] Unmanned Vehicles DRL DQN 3D, Autonomous, Real-Time
[11] Unmanned Vehicles DRL MADDPG Multi-Agent, Dynamic
[12] Unmanned Aerial Vehicles DRL A3C Model
[13] Unmanned Aerial Vehicles DRL DQN Ultrasonic Sensor, 3D ,
Obstacle Avoidance
[14] Unmanned Surface Vehicles DRL DQN High Degrees of Freedom

[15] Unmanned Aerial Vehicles DRL DQN Mobile-Edge Computing

[16] Multi-UAV (mobile communications DRL DDPG Multi-Agent,
system) 3D, Real-Time, Mobile-Edge
Computing
[17] Multi-UAV (wireless DRL DQN Mobile Edge Computing
communications system)
[18] Multi-Vehicle optimization DRL DQN Mobile-Edge computing

Table 2: Fault Diagnosis Solution Comparisons

Research Application(s) Deep learning method Network Comments

paper
[61] Fault Detection Gearbox CNN DRN Vibratory Signals
[62] Fault Detection Gearbox Autoencoder Vibratory Signals
[63] Part-Diagnosis UAV CNN CNN, LSTM In Vehicle,
[64] Fault Detection UAV - NN Vehicle Data, Sensors
[65] Spacecraft Electronic Systems CNN CNN High Dimensional Electric Data
[66] Fault Detection Parts and Pre- CNN CNN, LSTM Vehicle Data
Ignition
[67] Fault Detection and LQR DNN CIFTA Graphs
Identification UAV
[68] False Battery Data Detection CNN CNN Battery Health Sensor, Charging
Sensor
[69] Fault Detection UEV Microgrid CNN DNN, CNN Converter, Inverter Data
[70] Fault Detection UAV CNN CNN, LSTM Actuator, Flight Data
[71] Fault Detection UAV - LSTM Real-Time
[72] Fault Detection Signal - LSTM Wheel Angle Data
[73] Fault Detection UAV Sensor CNN CNN Connected Vehicles, Real-Time
[74] Attack Detection UAV MDP, Gradient Descent DRL Connected Vehicles, Multi-Agent

References [3] V. Rausch et al, “Learning a Deep Neural Net Policy for End-to-End
Control of Autonomous Vehicles” American Control Conference, Seattle,
May 2017.
[1] J. Ren, H. Gaber and S. S. Al Jabar, "Applying Deep Learning to
[4] S. Jung et al, “Perception, Guidance, and Navigation for Indoor
Autonomous Vehicles: A Survey," 2021 4th International Conference on
Autonomous Drone Racing Using Deep Learning” IEEE Robotics and
Artificial Intelligence and Big Data (ICAIBD), 2021, pp. 247-252, doi:
Automation Letters, 2018.
10.1109/ICAIBD51990.2021.9458968.
[5] K. Wu et al, “TDPP-Net: Achieving Three-Dimensional Path Planning via
[2] C. Hodges, S. An, H. Rahmani, & M. Bennamoun. Deep Learning for
a Deep Neural Network Architecture” Neurocomputing, vol. 357, pp. 151-
Driverless Vehicles. In V. Balas, S. Roy, D. Sharma, & P. Samui (Eds.),
162, 2019.
Handbook of Deep Learning Applications pp. 83-99. ( Smart Innovation,
Systems and Technologies; Vol. 136). Springer, 2019

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 65

 J. Ren et al., The Current Trends of Deep Learning

[6] Z. Bai et al, “Deep Learning Based Motion Planning for Autonomous [29] D. K. Dewangan and S. P. Sahu, "Deep Learning-Based Speed Bump
Vehicle Using Spatiotemporal LSTM Network” Chinese Automation Detection Model for Intelligent Vehicle System Using Raspberry Pi," in
Congress, 2018. IEEE Sensors Journal, vol. 21, no. 3, pp. 3570-3578, 1 Feb.1, 2021, doi:
[7] K. Saleh, M. Hossny and S. Nahavandi, “Intent Prediction of Pedestrians 10.1109/JSEN.2020.3027097.
via Motion Trajectories using Stacked Recurrent Neural Networks” IEEE [30] A. Dhiman and R. Klette, "Pothole Detection Using Computer Vision and
Transaction on Intelligent Vehicles,vol. 3, no. 4, 2018. Learning," in IEEE Transactions on Intelligent Transportation Systems,
[8] K. Makantasis et al, “Deep Reinforcement -Learning-Based Driving vol. 21, no. 8, pp. 3536-3550, Aug. 2020, doi:
Policy for Autonomous Road Vehicles” IET Intelligent Transport Systems, 10.1109/TITS.2019.2931297.
2020. [31] R. Xu, Y. Chen, X. Chen and S. Chen, "Deep learning based vehicle
[9] K. Zhang et al, “Robot Navigation of Environments with Unknown Rough violation detection system," 2021 6th International Conference on
Terrain Using Deep Reinforcement Learning” Proc. IEE Int. Symp. Sat. Intelligent Computing and Signal Processing (ICSP), 2021, pp. 796-799,
Secur. Rescue Robot., 2018. doi: 10.1109/ICSP51882.2021.9408935.
[10] C. Wang et al, “Autonomous Navigation of UAVs in Large-Scale [32] Y. Wang, H. Tan, Y. Wu and J. Peng, "Hybrid Electric Vehicle Energy
Complex Environment: A Deep Reinforcement Learning Approach” Management With Computer Vision and Deep Reinforcement Learning,"
IEEE Transactions on Vehicle Technology, 2019. in IEEE Transactions on Industrial Informatics, vol. 17, no. 6, pp. 3857-
[11] H. Qie et al, “Joint Optimization of Multi-UAV Target Assignment and 3868, June 2021, doi: 10.1109/TII.2020.3015748.
Path Planning Based on Multi-Agent Reinforcement Learning,” IEEE [33] X. Tang, J. Chen, K. Yang, M. Toyoda, T. Liu and X. Hu, "Visual
Access, vol. 7, 2019. Detection and Deep Reinforcement Learning-Based Car Following and
[12] C. Wu et al, “UAV Autonomous Target Search Based on Deep Energy Management for Hybrid Electric Vehicles," in IEEE Transactions
Reinforcement Learning in Complex Disaster Scene,” IEEE Access, vol. on Transportation Electrification, vol. 8, no. 2, pp. 2501-2515, June 2022,
7, 2019. doi: 10.1109/TTE.2022.3141780.
[13] G. -T. Tu and J. -G. Juang, "Path Planning and Obstacle Avoidance Based [34] Y. Zhang et al., "Improved Short-Term Speed Prediction Using
on Reinforcement Learning for UAV Application," 2021 International Spatiotemporal-Vision-Based Deep Neural Network for Intelligent Fuel
Conference on System Science and Engineering (ICSSE), 2021, pp. 352- Cell Vehicles," in IEEE Transactions on Industrial Informatics, vol. 17,
355, doi: 10.1109/ICSSE52999.2021.9537945. no. 9, pp. 6004-6013, Sept. 2021, doi: 10.1109/TII.2020.3033980.
[14] H. Zhai, W. Wang, W. Zhang and Q. Li, "Path Planning Algorithms for [35] H. Unlu et al, “Sliding-Window Temporal Attention Based Deep Learning
USVs via Deep Reinforcement Learning," 2021 China Automation System for Robust Sensor Modality Fusion for UGV Navigation,” IEEE
Congress (CAC), 2021, pp. 4281-4286, doi: Robotics and Automation Letters, vol. 4, no. 4, 2019.
10.1109/CAC53003.2021.9728038. [36] F. Nobis et al, “A Deep Learning-Based Radar and Camera Sensor Fusion
[15] Q. Liu, L. Shi, L. Sun, J. Li, M. Ding and F. Shu, "Path Planning for UAV- Architectur for Object Detection,” Sensor Data Fusion: Trends, Solutions,
Mounted Mobile Edge Computing With Deep Reinforcement Learning," Applications,2019.
in IEEE Transactions on Vehicular Technology, vol. 69, no. 5, pp. 5723- [37] X. Du et al, “Car Detection for Autonomous Vehicle: LIDAR and Vision
5728, May 2020, doi: 10.1109/TVT.2020.2982508. Fusion Approach through Deep Learning Framework,” International
[16] L. Wang, K. Wang, C. Pan, W. Xu, N. Aslam and L. Hanzo, "Multi-Agent Conference on Intelligent Robots and Systems, 2017.
Deep Reinforcement Learning-Based Trajectory Planning for Multi-UAV [38] C. Luca et al, “Lidar-Camera Fusion for Road Detection Using Fully
Assisted Mobile Edge Computing," in IEEE Transactions on Cognitive Convolutional Neural Networks,” Robotics and Autonomous Systems, vol.
Communications and Networking, vol. 7, no. 1, pp. 73-84, March 2021, 111, 2019.
doi: 10.1109/TCCN.2020.3027695. [39] H. Liu and D. M. Blough, "MultiVTrain: Collaborative Multi-View
[17] J. Tang, J. Song, J. Ou, J. Luo, X. Zhang and K. -K. Wong, "Minimum Active Learning for Segmentation in Connected Vehicles," 2021 IEEE
Throughput Maximization for Multi-UAV Enabled WPCN: A Deep 18th International Conference on Mobile Ad Hoc and Smart Systems
Reinforcement Learning Method," in IEEE Access, vol. 8, pp. 9124-9132, (MASS), 2021, pp. 428-436, doi: 10.1109/MASS52906.2021.00060.
2020, doi: 10.1109/ACCESS.2020.2964042. [40] A. Alireza et al, “Multimodal Vehicle Detection: Fusing 3D-LIDAR and
[18] C. Chen, J. Jiang, N. Lv and S. Li, "An Intelligent Path Planning Scheme Color Camera Data,” Pattern Recognition Letters, vol. 115, 2018.
of Autonomous Vehicles Platoon Using Deep Reinforcement Learning on [41] K. Geng et al, “Low-Observable Targets Detection for Autonomous
Network Edge," in IEEE Access, vol. 8, pp. 99059-99069, 2020, doi: Vehicles Based on Dual-Modal Sensor Fusion with Deep Learning
10.1109/ACCESS.2020.2998015. Approach,” Proceedings of the Institutiion of Mechanical Engineers, Part
[19] E. Arnold, O.Y. Al-Jarrah, M. Dianati, S. Fallah, D. Oxtoby, and A. D: Journal of Automobile Engineering, 2019.
Mouzakitis. “A Survey on 3D Object Detection Methods for Autonomous [42] J. Nie, J. Yan, H. Yin, L. Ren and Q. Meng, "A Multimodality Fusion
Driving Applications,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 10, Deep Neural Network and Safety Test Strategy for Intelligent Vehicles,"
Oct. 2019. in IEEE Transactions on Intelligent Vehicles, vol. 6, no. 2, pp. 310-322,
[20] V. Ravi Kumar et al., "SVDistNet: Self-Supervised Near-Field Distance June 2021, doi: 10.1109/TIV.2020.3027319.
Estimation on Surround View Fisheye Cameras," in IEEE Transactions [43] Z. Xiao et al, “Multimedia Fusion at Semantic Level in Vehicle
on Intelligent Transportation Systems, vol. 23, no. 8, pp. 10252-10261, Cooperative Perception,” IEEE International Conference on Multimedia
Aug. 2022, doi: 10.1109/TITS.2021.3088950. & Expo Workshops, 2018.
[21] N.Kosaka andG.Ohashi, "Vision-based night-time vehicle detection using [44] P. Wang, W. Hao and Y. Jin, "Fine-Grained Traffic Flow Prediction of
CenSurE and SVM," IEEE Trans. Intell. Transp. Syst., vol. 16, no. 5, pp. Various Vehicle Types via Fusion of Multisource Data and Deep Learning
2599–2608, 2015. Approaches," in IEEE Transactions on Intelligent Transportation
[22] D. A. Yudina, A. Skrynnikb, A. Krishtopika, I. Belkina, and A. I. Panova, Systems, vol. 22, no. 11, pp. 6921-6930, Nov. 2021, doi:
“Object Detection with Deep Neural Networks for Reinforcement 10.1109/TITS.2020.2997412.
Learning in the Task of Autonomous Vehicles Path Planning at the [45] P. Li, M. Abdel-Aty, Q. Cai and Z. Islam, "A Deep Learning Approach to
Intersection,” Optical Memory and Neural Networks (Information Detect Real-Time Vehicle Maneuvers Based on Smartphone Sensors," in
Optics), vol. 28, no. 4, p 283-295, 2019. IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 4,
[23] J. Wang, and L. Zhou “Traffic Light Recognition With High Dynamic pp. 3148-3157, April 2022, doi: 10.1109/TITS.2020.3032055.
Range Imaging and Deep Learning,” IEEE Trans. Intell. Transp. [46] Z. Chen, C. Jiang, S. Xiang, J. Ding, M. Wu and X. Li, "Smartphone
Syst, vol. 20, no. 4, Apr. 2019. Sensor-Based Human Activity Recognition Using Feature Fusion and
[24] H. Kim, J. H. Park, and H. Y. Jung, “An Efficient Color Space for Deep- Maximum Full a Posteriori," in IEEE Transactions on Instrumentation and
Learning Based Traffic Light Recognition,” Journal of Advanced Measurement, vol. 69, no. 7, pp. 3992-4001, July 2020, doi:
Transportation, Dec. 2018. 10.1109/TIM.2019.2945467.
[25] Z. Chen, and X. Huang, “Pedestrian Detection for Autonomous Vehicle [47] A. Narayanan, A. Siravuru and B. Dariush, “Gated Recurrent Fusion to
Using Multi-Spectral Cameras,” IEEE Transactions on Intelligent Learn Driving Behavior from Temporal Multimodal Data,” IEEE
Vehicles, vol. 4, No. 2, Jun. 2019. Robotics and Automation Letters, vol. 5, no. 2, 2020.
[26] A. Amanatiadis, E. Karakasis, L. Bampis, S. Ploumpis, and A. Gasteratos, [48] Z. Huang, C. Lv, Y. Xing and J. Wu, "Multi-Modal Sensor Fusion-Based
“ViPED: On-road vehicle passenger detection for autonomous vehicles,” Deep Neural Network for End-to-End Autonomous Driving With Scene
Robotics and Autonomous Systems, Dec. 2018. Understanding," in IEEE Sensors Journal, vol. 21, no. 10, pp. 11781-
[27] Y. Li et al., "A Deep Learning-Based Hybrid Framework for Object 11790, 15 May15, 2021, doi: 10.1109/JSEN.2020.3003121.
Detection and Recognition in Autonomous Driving," in IEEE Access, vol. [49] Z. Xiong, Z. Cai, Q. Han, A. Alrawais and W. Li, "ADGAN: Protect Your
8, pp. 194228-194239, 2020, doi: 10.1109/ACCESS.2020.3033289. Location Privacy in Camera Data of Auto-Driving Vehicles," in IEEE
[28] X. Liu, and Z. Deng, “Segmentation of Drivable Road Using Deep Fully Transactions on Industrial Informatics, vol. 17, no. 9, pp. 6200-6210, Sept.
Convolutional Residual Network with Pyramid Pooling,” Cognitive 2021, doi: 10.1109/TII.2020.3032352.
Computation, Nov. 2017. [50] M. Yang, Y. He and J. Qiao, "Federated Learning-Based Privacy-
Preserving and Security: Survey," 2021 Computing, Communications and

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 66

 J. Ren et al., The Current Trends of Deep Learning

IoT Applications (ComComAp), 2021, pp. 312-317, doi: vol. 21, no. 15, pp. 17290-17299, 1 Aug.1, 2021, doi:
10.1109/ComComAp53641.2021.9653016. 10.1109/JSEN.2021.3079118.
[51] M. A. Ferrag, O. Friha, L. Maglaras, H. Janicke and L. Shu, "Federated [73] F. van Wyk, Y. Wang, A. Khojandi and N. Masoud, "Real-Time Sensor
Deep Learning for Cyber Security in the Internet of Things: Concepts, Anomaly Detection and Identification in Automated Vehicles," in IEEE
Applications, and Experimental Analysis," in IEEE Access, vol. 9, pp. Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp.
138509-138542, 2021, doi: 10.1109/ACCESS.2021.3118642. 1264-1276, March 2020, doi: 10.1109/TITS.2019.2906038.
[52] Y. Wang, Z. Su, N. Zhang and A. Benslimane, "Learning in the Air: [74] G. Raja, K. Kottursamy, K. Dev, R. Narayanan, A. Raja and K. B. V.
Secure Federated Learning for UAV-Assisted Crowdsensing," in IEEE Karthik, "Blockchain-Integrated Multiagent Deep Reinforcement
Transactions on Network Science and Engineering, vol. 8, no. 2, pp. 1055- Learning for Securing Cooperative Adaptive Cruise Control," in IEEE
1069, 1 April-June 2021, doi: 10.1109/TNSE.2020.3014385. Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp.
[53] Q. Kong et al., "Privacy-Preserving Aggregation for Federated Learning- 9630-9639, July 2022, doi: 10.1109/TITS.2022.3168486.
Based Navigation in Vehicular Fog," in IEEE Transactions on Industrial [75] B. G. Maciel-Pearson et al, “Multi-Task Regression-Based Learning for
Informatics, vol. 17, no. 12, pp. 8453-8463, Dec. 2021, doi: Autonomous Unmanned Aerial Vehicle Flight Control within
10.1109/TII.2021.3075683. Unstructured Outdoor Environment,” IEEE Robotics and Automation
[54] S. Lu, Y. Yao and W. Shi, "CLONE: Collaborative Learning on the Letters, vol. 4, no. 4, 2019.
Edges," in IEEE Internet of Things Journal, vol. 8, no. 13, pp. 10222- [76] M. K. Al-Sharman et al, “Deep-Learning-Based Neural Network Training
10236, 1 July1, 2021, doi: 10.1109/JIOT.2020.3030278. for State Estimation Enhancement: Application to Attitude Estimation,”
[55] W. Y. B. Lim et al., "Towards Federated Learning in UAV-Enabled IEEE Transactions on Instrumentation and Measurement, vol. 69, issue 1,
Internet of Vehicles: A Multi-Dimensional Contract-Matching 2020.
Approach," in IEEE Transactions on Intelligent Transportation Systems, [77] Y. Kang et al, “Deep Convolutional Identifier for Dynamic Modeling and
vol. 22, no. 8, pp. 5140-5154, Aug. 2021, doi: Adaptive Control of Unmanned Helicopter,” IEEE Transactions on
10.1109/TITS.2021.3056341 Neural Networks and Learning Systems, vol. 30, no. 2, 2019.
[56] Y. Lu, X. Huang, K. Zhang, S. Maharjan and Y. Zhang, "Blockchain [78] Y. Li et al, “Compensating Delays and Noises in Motion Control of
Empowered Asynchronous Federated Learning for Secure Data Sharing Autonomous Electric Vehicles by Using Deep Learning and Unscented
in Internet of Vehicles," in IEEE Transactions on Vehicular Technology, Kalman Predictor,” IEEE Transactions on Systems, Man and Cybernetics,
vol. 69, no. 4, pp. 4298-4311, April 2020, doi: 2018.
10.1109/TVT.2020.2973651. [79] P. Drews et al, “Vision-Based High-Speed Driving with a Deep Dynamic
[57] P. Kumar, R. Kumar, G. P. Gupta and R. Tripathi, "BDEdge: Blockchain Observer,” IEEE Robotics and Automation Letters, vol. 4, no. 2, 2019.
and Deep-Learning for Secure Edge-Envisioned Green CAVs," in IEEE [80] Q. Zhang et al, “Deep Interactive Reinforcement Learning for Path
Transactions on Green Communications and Networking, vol. 6, no. 3, Following of Autonomous Underwater Vehicle,” IEEE Access, 2020.
pp. 1330-1339, Sept. 2022, doi: 10.1109/TGCN.2022.3165692. [81] I. Carlucho et al, “Adaptive Low-Level Control of Autonomous
[58] J. J. Q. Yu, "Sybil Attack Identification for Crowdsourced Navigation: A Underwater Vehicles Using Deep Reinforcement Learning,”
Self-Supervised Deep Learning Approach," in IEEE Transactions on Robotics and Autonomous Systems, 2018.
Intelligent Transportation Systems, vol. 22, no. 7, pp. 4622-4634, July [82] Y. Shang and W. Qiao, "Intelligent driving trajectory tracking control
2021, doi: 10.1109/TITS.2020.3036085. algorithm based on deep learning," 2021 IEEE 4th International
[59] W. Jiang, H. Li, S. Liu, X. Luo and R. Lu, "Poisoning and Evasion Attacks Conference on Automation, Electronics and Electrical Engineering
Against Deep Learning Algorithms in Autonomous Vehicles," in IEEE (AUTEEE), 2021, pp. 618-621, doi:
Transactions on Vehicular Technology, vol. 69, no. 4, pp. 4439-4449, 10.1109/AUTEEE52864.2021.9668724.
April 2020, doi: 10.1109/TVT.2020.2977378. [83] S. Sharma, G. Tewolde and J. Kwon, “Lateral and Longitudinal Motion
[60] C. Song et al., "MAT: A Multi-strength Adversarial Training Method to Control of Autonomous Vehicles Using Deep Learning,” IEEE
Mitigate Adversarial Attacks," 2018 IEEE Computer Society Annual International Conference on Electro Information Technology, 2019.
Symposium on VLSI (ISVLSI), 2018, pp. 476-481, doi: [84] S. Yang et al, “Scene Understanding in Deep Learning-based End-to-End
10.1109/ISVLSI.2018.00092. Controllers for Autonomous Vehicles,” IEEE Transaction on Systems,
[61] M. Zhao et al, “Deep Residual Networks with Dynamically Weighted Man, and Cybernetics, vol. 49, no. 1, 2019.
Wavelet Coefficients for Fault Diagnosis of Planetary Gearboxes,” IEEE [85] J. Zhu et al, “Hierarchical Decision and Control for Continuous
Transactions on Industrial Electronics, vol. 65, no. 5, 2018. Multitarget Problem: Policy Evaluation with Action Delay,” IEEE
[62] Z. He et al, “Improved Deep Transfer Auto-encoder for Fault Diagnosis Transactions on Neural Networks and Learning Systems, vol. 30, no. 2,
of Gearbox under Variable Working Conditions with Small Training 2019.
Samples,” IEEE Access, vol. 7, 2019. [86] Q. Chen, W. Zhao, L. Li, C. Wang and F. Chen, "ES-DQN: A Learning
[63] K. Kim et al, “A Deep Learning Part-Diagnosis Platform (DLPP) Based Method for Vehicle Intelligent Speed Control Strategy Under Uncertain
on an in-Vehicle on-Board Gateway for an Autonomous Vehicle,” KSII Cut-In Scenario," in IEEE Transactions on Vehicular Technology, vol. 71,
Transactions on Internet and Information Systems, vol. 13, no. 8, 2019. no. 3, pp. 2472-2484, March 2022, doi: 10.1109/TVT.2022.3143840.
[64] Y. Jeong et al, “An Integrated Self-Diagnosis System for an Autonomous [87] H. Iwasaki and A. Okuyama, "Development of a Reference Signal Self-
Vehicle Based on an IoT Gateway and Deep Learning,” Applied Science, Organizing Control System Based on Deep Reinforcement Learning,"
2018. 2021 IEEE International Conference on Mechatronics (ICM), 2021, pp. 1-
[65] Y. Liu et al, “MRD-Nets: Multi-Scale Residual Networks w. Dilated 5, doi: 10.1109/ICM46511.2021.9385676.
Convolutions for Classification and Clustering Analysis of Spacecraft [88] T. Chu, J. Wang, L. Codecà and Z. Li, "Multi-Agent Deep Reinforcement
Electrical Signal,” IEEE Access, vol. 7, 2019. Learning for Large-Scale Traffic Signal Control," in IEEE Transactions
[66] P. Wolf et al, “Pre-ignition Detection Using Deep Neural Networks: A on Intelligent Transportation Systems, vol. 21, no. 3, pp. 1086-1095,
Step Towards Dat-Driven Automotive Diagnostics,” International March 2020, doi: 10.1109/TITS.2019.2901791.
Conferernce on Intelligent Transportation Systems, 2018. [89] A. Boukerche, D. Zhong and P. Sun, "FECO: An Efficient Deep
[67] M. Olyael et al, “Fault Detection and Identification on UAV System with Reinforcement Learning-Based Fuel-Economic Traffic Signal Control
CITFA Algorithm Based on Deep Learning,” Iranian Conference on Scheme," in IEEE Transactions on Sustainable Computing, vol. 7, no. 1,
Electrical Engineering, 2018. pp. 144-156, 1 Jan.-March 2022, doi: 10.1109/TSUSC.2021.3138926.
[68] H. -J. Lee, K. -T. Kim, J. -H. Park, G. Bere, J. J. Ochoa and T. Kim, [90] Q. Yang, G. Wang, A. Sadeghi, G. B. Giannakis and J. Sun, "Two-
"Convolutional Neural Network-Based False Battery Data Detection and Timescale Voltage Control in Distribution Grids Using Deep
Classification for Battery Energy Storage Systems," in IEEE Transactions Reinforcement Learning," in IEEE Transactions on Smart Grid, vol. 11,
on Energy Conversion, vol. 36, no. 4, pp. 3108-3117, Dec. 2021, doi: no. 3, pp. 2313-2323, May 2020, doi: 10.1109/TSG.2019.2951769.
10.1109/TEC.2021.3061493. [91] J. Duan et al., "Deep-Reinforcement-Learning-Based Autonomous
[69] H. Moinul et al, “Deep Learning Based Micro-Grid Fault Detection and Voltage Control for Power Grid Operations," in IEEE Transactions on
Classification in Future Smart Vehicle,” IEEE Transportation and Power Systems, vol. 35, no. 1, pp. 814-817, Jan. 2020, doi:
Electrification Conference and Expo, 2018. 10.1109/TPWRS.2019.2941134.
[70] J. Fu et al, “A Hybrid CNN-LSTM Model Based Actuator Fault Diagnosis [92] S. Kumari, K. K. Srinivas and P. Kumar, "Channel and Carrier Frequency
for Six-Rotor UAVs,” Chinese Control and Decision Conference, 2019. Offset Equalization for OFDM Based UAV Communications Using Deep
[71] B. Wang, X. Peng, M. Jiang and D. Liu, "Real-Time Fault Detection for Learning," in IEEE Communications Letters, vol. 25, no. 3, pp. 850-853,
UAV Based on Model Acceleration Engine," in IEEE Transactions on March 2021, doi: 10.1109/LCOMM.2020.3036493.
Instrumentation and Measurement, vol. 69, no. 12, pp. 9505-9516, Dec. [93] C. H. Liu, Z. Chen, J. Tang, J. Xu and C. Piao, "Energy-Efficient UAV
2020, doi: 10.1109/TIM.2020.3001659. Control for Effective and Fair Communication Coverage: A Deep
[72] S. Zou, W. Zhao, C. Wang and F. Chen, "Fault Detection Strategy of Reinforcement Learning Approach," in IEEE Journal on Selected Areas
Vehicle Wheel Angle Signal via Long Short-Term Memory Network and in Communications, vol. 36, no. 9, pp. 2059-2070, Sept. 2018, doi:
Improved Sequential Probability Ratio Test," in IEEE Sensors Journal, 10.1109/JSAC.2018.2864373.

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 67

 J. Ren et al., The Current Trends of Deep Learning

[94] C. Wang, D. Deng, L. Xu and W. Wang, "Resource Scheduling Based on

Deep Reinforcement Learning in UAV Assisted Emergency
Communication Networks," in IEEE Transactions on Communications,
vol. 70, no. 6, pp. 3834-3848, June 2022, doi:
10.1109/TCOMM.2022.3170458.
[95] M. Fadda, M. Murroni and V. Popescu, "Interference Issues for VANET
Communications in the TVWS in Urban Environments," in IEEE
Transactions on Vehicular Technology, vol. 65, no. 7, pp. 4952-4958, July
2016, doi: 10.1109/TVT.2015.2453633.
[96] X. Zhang, M. Peng, S. Yan and Y. Sun, "Deep-Reinforcement-Learning-
Based Mode Selection and Resource Allocation for Cellular V2X
Communications," in IEEE Internet of Things Journal, vol. 7, no. 7, pp.
6380-6391, July 2020, doi: 10.1109/JIOT.2019.2962715.
[97] O. S. Oubbati, M. Atiquzzaman, A. Baz, H. Alhakami and J. Ben-Othman,
"Dispatch of UAVs for Urban Vehicular Networks: A Deep
Reinforcement Learning Approach," in IEEE Transactions on Vehicular
Technology, vol. 70, no. 12, pp. 13174-13189, Dec. 2021, doi:
10.1109/TVT.2021.3119070.
[98] U. Challita, W. Saad and C. Bettstetter, "Interference Management for
Cellular-Connected UAVs: A Deep Reinforcement Learning Approach,"
in IEEE Transactions on Wireless Communications, vol. 18, no. 4, pp.
2125-2140, April 2019, doi: 10.1109/TWC.2019.2900035.
[99] G. Cui, Y. Long, L. Xu and W. Wang, "Joint Offloading and Resource
Allocation for Satellite Assisted Vehicle-to-Vehicle Communication," in
IEEE Systems Journal, vol. 15, no. 3, pp. 3958-3969, Sept. 2021, doi:
10.1109/JSYST.2020.3017710.
[100] T. Yang, J. Li, H. Feng, N. Cheng and W. Guan, "A Novel Transmission
Scheduling Based on Deep Reinforcement Learning in Software-Defined
Maritime Communication Networks," in IEEE Transactions on Cognitive
Communications and Networking, vol. 5, no. 4, pp. 1155-1166, Dec.
2019, doi: 10.1109/TCCN.2019.2939813.

Copyright: This work is licensed under a Creative Commons Attribution-

ShareAlike 4.0 International License. For more information, see
https://creativecommons.org/licenses/by-sa/4.0/

RAYMOND NING HUANG is currently pursing a bachelor's degree in

Mechanical Engineering at the University of Toronto. His research
interests include deep learning and robotics.

JING REN has done her bachelor's degree from Shandong University in
1993. She has done her master’s degree from Western University in 2003.
She has completed her PhD degree in Robotics from Western University
in 2005.
Dr. Ren is an associate professor in the Faculty of Engineering and
Applies Science at Ontario Tech University. Her research interests
include deep learning, image processing and robotics. She is a recipient
of University Faculty Award in 2006.

HOSSAM A. GABBAR obtained his B.Sc. degree in 1988 with first class
of honor from the Faculty of Engineering, Alexandria University
(Egypt). In 2001, he obtained his Ph.D. degree from Okayama University
(Japan).
Dr. Gabbar is a full Professor in the Faculty of Energy Systems and
Nuclear Science, and cross appointed in the Faculty of Engineering and
Applied Science, at Ontario Tech University (UOIT), where he has
established the Energy Safety and Control Lab (ESCL), Smart Energy
Systems Lab, and Advanced Plasma Engineering Lab. He is the recipient
of the Senior Research Excellence Aware for 2016, UOIT. He is
recognized among the top 2% of worldwide scientists with high citation
in the area of energy. He is a Distinguished Lecturer of IEEE NPSS. He
is leading national and international research in the areas of smart
energy grids, energy safety and control systems, and waste to energy
using advanced plasma technologies. From 2001 till 2004, he joined
Tokyo Institute of Technology (Japan), as a research associate. From 2004
till 2008, he joined Okayama University (Japan) as an Associate
Professor, in the Division of Industrial Innovation Sciences. From 2007
till 2008, he was a Visiting Professor at the University of Toronto. He
also worked as process control, safety, and automation specialist in
energy and oil & gas industries. Dr. Gabbar has more than 230
publications, including patents, books / chapters, journal and conference
papers.

www.jenrs.com Journal of Engineering Research and Sciences, 1(10): 56-68, 2022 68