Visvesvaraya Technological University, Belagavi

Rajeev institute of Technology

Department of Computer Science and Engineering

Project Presentation on

“Self Driving Car Using Artificial Intelligence ”

Submitted by
Bhumika L
4RA16CS016
Chaitra p 4RA16CS017
Darshan M N 4RA16CS018
Shashi Kumar M S 4RA16CS422

Under the Guidance of

Mrs. Shashikala M K BE, MTech
Department of Computer science and engineering
RIT, Hassan
 ABSTRACT
Our project is a Self-Driving Car, which is designed to give human driver relaxed
driving. In the field of automobile various aspects have been considered which makes a
vehicle automated.
In this project we have focused on application of automated car, in which the car will
detect obstacle, traffic signal and stop signs.
It also drives itself on the track and monitoring and surveillance the environment with
the help of Ultrasonic sensors,Lidar sensors.
Arduino, Raspberry pi which is used to collect all the input data from sensors and
cameras.
The other application is automated driving during the heavy traffic jam, hence relaxing
driver from continuously pushing break, accelerator or clutch. Since taking intelligent
decision in the traffic is also an issue for the automated vehicle so this aspect has been
also under consideration.
 TABLE OF CONTENTS

01 INTRODUCTION 06 USE CASE DIAGRAMS

02 LITERATURE 07 IMPLEMENTATION
SURVEY
03 EXISTING SYSTEM 08 TESTING

04 PROPOSED SYSTEM 09 RESULT

05 SYSTEM 10 CONCLUSION
REQUIREMENTS
INTRODUCT 01
ION
 INTRODUCTION
► An autonomous car is a robotic vehicle that is designed to travel between destinations without any
human intervention.

► This impacts of autonomous (also called self-driving, driverless or robotic) vehicles, and their
implications for transportation planning. It investigates how quickly such vehicles are likely to
develop and be deployed based on experience with previous vehicle technologies.

► Their likely benefits and costs, how they will affect travel activity and their impacts on planning
decisions such as optimal road, parking and public transit supply.

► An Self-driving cars are autonomous decision-making systems that process streams of observations
coming from different on-board sources, such as cameras, radars, LiDAR's, ultrasonic sensors, GPS
units and/or inertial sensor’s.

► Convolutional Neural Networks (CNN) are dedicated to vision-based approaches and they are quite
feasible for Graphics Processing Unit (GPU) acceleration in the real-time application.
 INTRODUCTION
► Autonomous vehicles make use of different types of sensors together to decide on control of vehicle
based on condition changes such as steering or applying thrust in case of emergency..

► However, detecting objects in a video stream frame-by-frame is difficult when delay of milliseconds
can lead to collision.

► Object detection is basically divided into two steps. First one being, image classification and second
one being image localization.

► Image classification determines the type of objects present in an image while localization of image
provides the almost exact location of these objects in the images (in bounding boxes).
LITERATUR 02
E SURVEY
 LITERATURE SURVEY

 “Looking at vehicles on the road: A survey of vision-based vehicle detection,

tracking, and behaviour analysis”.
vision-based vehicle detection, tracking, and on-road
behaviour analysis. We place vision-based vehicle detection in the context of on-road
environmental perception, briefly detailing complimentary modalities that are
commonly used for vehicle detection, namely, radar and lidar.

 "Road Safety and Accident Prevention in India” :

Autonomous vehicles developed from the basic robotic cars to much
efficient and practical vision guided vehicles. The development of Mercedes- Benz
vision guided autonomous van by Ernst Dickmanns and his team gave a paradigm
shift to the approach followed in autonomous cars.
 LITERATURE SURVEY

 “Lane Detection and Following Approach in Self-Driving Miniature Vehicles”.

Lane markings are always present during intersections.
Intersections do not appear directly after a curve; a straight road segment eases the
transition between the two. A method that requires as much data as available to
function as intended, it remove the invalid data by fitting a desired shape to the
detected features, i.e. a curve, a straight line, a circle, and then later applies
smoothing a smooth curve.

 "Obstacle Detection and Classification for Tracking in High-Speed Autonomous

Driving” : The detection and classification of on-road objects using a
variant of CNN and its implementation in GPU. During the real-time detection
phase, the on-road object detections are filtered such that the entire system is made to
detect only the classes which correspond to on-road objects.
EXISTING 03
SYSTEM
 EXISTING SYSTEM

 Autonomous cars involves training each car separately one by one until they have enough
experience to drive safely without any accident.

 Various Deep Learning algorithms are used with high precision to make the car learn to drive
on its own.
 Ex: Waymo, Tesla

Disadvantages of Existing system :

 Safety and security concerns.

 High-technology vehicles and equipment are expensive.

 Sensors failures often happened during drastic weather conditions. This may not work
during a blizzard or a heavy snowfall.

 Autonomous vehicles could be the next major target of the hackers as this vehicle
continuously tracks and monitors details of the owner.
PROPOSED 04
SYSTEM
 PROPOSED SYSTEM

 To detect the path whether it is smooth or curve edge and also take pictures in rainy
environment.

 To detect the sudden obstacles which comes in the path.

 To know the security of that car weather it is moving in correct direction, towards
defined destination and also do the basic things of a human driving features with the car
at low cost.

 Gives same performance as when the weather changes interferes with sensors.
 SYSTEM
REQUIREME
05
NTS
 SYSTEM REQUIREMENTS

Hardware Requirements
► Monitor with HDMI cable

► Robot chassis with 4WD motors

► Raspberry Pi 3 B+ WIFI module

► RaspiCam 2 Camera

► Arduino Uno

► L298N Bridge

► 10000 mAh Power bank with USB cables.

► 16GB SD Card

► Connecting wires and Cables

 SYSTEM REQUIREMENTS

Software Requirements
► Raspberry OS

► Windows OS

► Arduino IDE

► Python 3

► OpenCV

► NumPy

► Keras

► GNU Editor
 USECASE
DIAGRAMS
06
USECASE DIAGRAMS

Flow Chart Block Diagram

USECASE DIAGRAMS

ER Diagram Car Navigation System

IMPLEMENTAT
ION
07
 IMPLEMENTATION
Lane Detection
We use the canny and HAAR – Cascade for detecting the
lane spot depend on the edge detection in the road. Depend on the lane
edge detected the RC car is move that using OpenCV and Commanded
by the Convectional Neural network.

For about the Curve detection we define the 3 left and right for curve and
U turn depend on the perspective view of the OpenCV the difference of
previous is turn or give left and right command by Convectional Neural
network.
 IMPLEMENTATION

Distance Measurement
Raspberry Pi can only support one pi camera
module. Using two USB web cameras will bring extra weight to the RC
car and also seems unpractical. Therefore, monocular vision method is
chosen. This project adapted a geometry model of detecting distance to
an object using monocular vision method proposed by Chu, Ji, Guo, Li
and Wang (2004).

v is the camera coordinates on y-axis and can be returned

from the object detection process. All other parameters are camera’s
intrinsic parameters that can be retrieved from camera matrix.

OpenCV provides functions for camera calibration. Camera

matrix for the 5MP pi camera is returned after calibration. Ideally, a-x
and a-y have the same value. Variance of these two values will result in
non-square pixels in the image. The matrix below indicates that the fixed
focal length lens on pi camera provides a reasonably good result in
handling distortion aspect. Here is an interesting article discussing the
 IMPLEMENTATION

Obstacle Detection
This project adapted the shape-based approach used for
object detection. Since each object requires its own classifier and follows
the same process in training and detection, this project focused on the
obstacle detection based on the Ultrasonic sensor and distance between
the obstacle.

OpenCV provides a trainer as well as detector. Positive and Negative

samples (contain target object) were acquired using a cell phone, and
were cropped that only desired object is visible and using Keras we
simply identify the objectives.
 IMPLEMENTATION

Traffic Sign and Light Detection

Negative samples (without target object), on
the other hand, were collected randomly. In particular, traffic light
positive samples contains equal number of red traffic lights and green
traffic light.

The same negative sample dataset was used for both stop sign and traffic
light training. Below shows some positive and negative samples used in
this project.
Firstly, trained cascade classifier is used to detect traffic
light. The bounding box is considered as a region of
interest (ROI). Secondly, Gaussian blur is applied inside
the ROI to reduce noises. Thirdly, find the brightest point
in the ROI. Finally, red or green states are determined
simply based on the position of the brightest spot in the
ROI.
TESTING
08
 TESTING

Purpose of Testing
The purpose of testing is to discover errors. Testing is the process of trying to discover
every conceivable fault or weakness in a work product. It provides a way to check the functionality
of components, assemblies and/or a finished product it is the process of exercising software with the
intent of ensuring that the software system meets its requirements and user expectations and does
not fail in an unacceptable manner. There are various types of test. Each test type addresses a
specific testing requirement.

Type 1 : Unit Testing

Unit testing involves the design of test cases that validate that the internal program
logic is functioning properly, and that program inputs produce valid outputs. All decision branches
and internal code flow should be validated.
 TESTING

Type 2 : Integration testing

Integration tests are designed to test integrated software components to determine if
they actually run as one program. Testing is event driven and is more concerned with the basic
outcome of screens or fields. Integration testing is specifically aimed at exposing the problems that
arise from the combination of components.

Type 3 : System Testing

System testing ensures that the entire integrated software system meets requirements. It
tests a configuration to ensure known and predictable results. an example of system testing is the
configuration oriented system integration test. System testing is based on process descriptions and
flows, emphasizing pre-driven process links and integration points.

.
 TESTING

Type 4 : White Box Testing

White box testing is a testing in which in which the software tester has knowledge of
the inner workings, structure and language of the software, or at least its purpose. It is purpose. It is
used to test areas that cannot be reached from a black box level.

Type 5 : Black box Testing

Black box testing is testing the software without any knowledge of the inner workings,
structure or language of the module being tested. Black box tests, as most other kinds of tests, must
be written from a definitive source document, such as specification or requirements document, such
as specification or requirements document. It is a testing in which the software under test is treated,
as a black box.

.
RESULTS
09
 RESULTS

Prediction on the testing samples returns an accuracy of 85% compared to the accuracy of 96% that
the training samples returns. In actual driving situation, predictions are generated about 10 times a
second (streaming rate roughly 10 frames/s).

HAAR features by nature are rotation sensitive. In this project, however, rotation is not a concern as
both the stop sign and the traffic light are fixed objects, which is also a general case in real world
environment

For distance measurement aspect, the ultrasonic sensor is only used to determine the distance to an
obstacle in front of the SD car and provides accurate results when taking proper sensing angle and
surface condition into considerations.

On the other hand, Pi camera provides “good enough” measurement results. In fact, as long as we
know the corresponding number to the actual distance, we know when to stop the RC car.
Experimental results of detecting distance using pi camera are as shown:
 RESULTS

In this project, the accuracy of distance measurement using monocular vision approach could be
influenced by the following factors:
1 Errors in actual values measurement,
2 Object bounding box variations in detecting process,
3 Errors in camera calibration process,

Nonlinear relationship between distance and camera coordinate :

The further the distance, the more rapid change of camera coordinate, thus
the greater the error.

Overall, the RC car could successfully navigate on the track with the ability to avoid front collision,
and respond to stop sign and traffic light accordingly.
CONCLUSIO 10
N
 CONCLUSION

We have empirically demonstrated that CNNs are able to learn the entire task of
lane and road following without manual decomposition into road or lane marking
detection, semantic abstraction, path planning, and control. A small amount of
training data from less than a hundred hours of driving was sufficient to train the
car to operate in diverse conditions, on highways, local and residential roads in
sunny, cloudy, and rainy conditions. The CNN is able to learn meaningful road
features from a very sparse training signal (steering alone).

The system learns for example to detect the outline of a road without the need of
explicit labels during training

More work is needed to improve the robustness of the network, to find methods to
verify the robustness, and to improve visualization of the network-internal
processing steps.
THAN
KS!