Group 14 Project Report
Group 14 Project Report
Bachelor of Science
(Computer Science)
by
2021-22
CERTIFICATE
This is to certify that the project entitled Weapon Detection and Classification
using CNN is a bonafide work of Shaikh Hala Mehwish Mohammad Shoeb
submitted to the University of Mumbai in partial fulfillment of the requirement for
the award of the degree of Bachelor of Science – Computer Science.
Supervisor Examiner
Examiner
..............................
Supervisor
..............................
Date:
Place:
Declaration
I declare that this written submission represents my ideas in my own words and
where others’ ideas or words have been included, I have adequately cited and
referenced the original sources. I also declare that I have adhered to all
principles of academic honesty and integrity and have not misrepresented or
fabricated or falsified any idea/data/fact/source in my submission. I understand
that any violation of the above will be cause for disciplinary action by the
Institute and can also evoke penal action from the sources which have thus not
been properly cited or from whom proper permission has not been taken when
needed.
…………………….
Shaikh Hala Mehwish
Date:
Acknowledgments
I have taken efforts in this project. However, it would not have been possible
without the kind support and help of many individuals. I would like to extend my
sincere thanks to all of them.
I am highly indebted to Prof. Diana Fernandez for their guidance and constant
supervision as well as for providing necessary information regarding the project &
also for their support in completing the project.
We would like to express my gratitude towards my parents & Staff of B.K Birla
College of Arts, Science and Commerce – CS dept for their kind co-operation and
encouragement which help me in completion of this project.
.
My thanks and appreciations also go people who have willingly helped me out
with their abilities.
vi
Abstract:
Limiting gun violence has been a priority in recent years. In this project, we will be
implementing the use of Deep learning algorithms to detect any firearms / weapons
to improve response time and reduce potential harm. The proposed system in this
project is a weapon detection system using TensorFlow object detection API. The
project will involve the use of various resources for implementation, such as :
Python , TensorFlow object detection API , OpenCV , Google Colab , NumPy etc.
Keywords:
TensorFlow, Weapon Detection, OpenCV, CNN, Image Classification, Machine
Learning
vii
Contents
1 Introduction 1
1.1 Statement of Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1
1.1.1 Project Architecture . . . . . . . . . . . . . . . . . . . . . . . . . .. 1
1.1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1
1.2 Objective and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1
2 Literature Review 2
2.1 Paper Title . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2
2.1.1 Weaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2
2.1.2 How to Overcome . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2
3 Technical Details 3
3.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3
3.2 Project Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3
3.2.1 Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . .. 3
3.2.2 Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . .. 3
References 6
viii
Keywords And Glossary
Chapter 1
Introduction
Statement of Project
Project Architecture
I will attempt to train a model with a large data set of images of various types of guns
and use it to detect weapons in real time for security purposes
Motivation
Everyone has the right to feel safe and secure in public places with the amount of
policing, technology, and surveillance in place
Yet, despite all these measures the threat of an attack remains very real. Any attack or
assault in a public place with firearms isn’t dealt with until the attacker has already
committed the crime they set out to do. This is due to the massive response time to
threats even with CCTV cameras and how difficult it is to spot a semi-concealed
weapon; making the response to such threats often reactive rather than proactive.
1
In this project, I propose the use of Deep learning algorithms to detect any such
firearms/weapons before the crime has even occurred to improve response time and
reduce potential harm.
It can provide people a sense of security in public settings when these detectors are
employed.
Allows security personnel to control and monitor larger crowds
Reduce response time to all threats aiding in reduction of potential harm
Deter potential criminals and make them think twice about walking across a security
camera
2
Chapter 2
Literature Review
Review:
In this Paper Faster R-CNN is used to detect guns in images. A set of images of guns are taken
in different situations with different backgrounds and are used to train Faster R-CNN. Then the
model is evaluated using another set of images. Faster R-CNN has been widely used for Object
Detection with satisfying results. Different CNN architectures such as ResNet50, Inception-
ResNetV2, VGG16 and MobileNetV2, have been used separately as feature extractors in faster
R-CNN in order to select the most efficient one. MobileNetV2 has been chosen due to its
suitability for mobile applications. To evaluate the efficiency of Faster R-CNN in detecting
Weapons, YOLOv2 with ResNet as a feature extractor is also used for detecting guns. Dataset
has been collected from IMFDB (Internet Movies Firearm Database). The dataset contains
7,909 images with different sizes where 4,656 are labeled and 3,263 are unlabeled. To label the
images I used the MATLAB ImageLabeler tool. They have tried different models such as CNN
architectures, ResNet50, Inception ResNetV2 and VGG16 and Finally MobileNetV2 are used
as feature extractor in faster R-CNN. In addition YOLOv2, an Object Detection approach based
on Convolutional Neural Network known for its high speed is used also to compare its results
with Faster R-CNN results. Two cloud service platforms such as Microsoft Azure and AWS
have been used during the training of the model.
3
Paper Title: Weapon Detection using Artificial Intelligence and Deep
Learning for Security Applications
Review :
This research paper focuses on automatic weapon detection utilising SSD and Faster RCNN
techniques based on convolutional neural networks (CNNs). Two datasets were used in the
implementation. One included pre-labelled images, while the other contained images that were
manually labelled. Fatkun Batch Image Downloader was used to acquire the datasets in bulk. For
weapon detection, SSD and Faster RCNN algorithms are simulated on a pre-labelled and self-
created image dataset. Both algorithms are efficient and produce decent results, but their use in
real time requires a compromise between speed and accuracy. In terms of performance, the SSD
algorithm is faster, at 0.736 s/frame. Meanwhile, faster RCNN provides a speed of 1.606 s/frame,
which is slow when compared to SSD. Faster RCNN has a higher accuracy of 84.6 percent. SSD,
on the other hand, has a lower accuracy of 73.8 percent. Because of its higher speed, SSD
provided real-time detection, but faster RCNN provided superior accuracy. Some resources used
for implementations are: OpenCV 3.4, Python, COCO Dataset (dataset containing common
objects with respective labels), Anaconda and TensorFlow
Review :
This study is primarily intended to recognize items in photographs that are fed into the system as
input. It can also recognize color photos with any number of attributes. Another handy solution is
that whenever a weapon is entered as input, it displays a pop-up notification indicating that it is a
weapon. As the items are taught more and more, machine learning techniques let us recognize the
thing with the highest accuracy. The Convolution Neural Network is the smartest method for
detecting objects using a fully linked network. Each layer finds objects with distinct properties
and extracts the required object from the image, making it very easy for the computer to
recognize.
4
Chapter 3
Technical Details
Methodology
Methodology:
Python:
Python is a high-level, general-purpose programming language that is interpreted. The use of
considerable indentation in its design philosophy promotes code readability. Its language
elements and object-oriented approach are aimed at assisting programmers in writing clear,
logical code for both small and large-scale projects.
OpenCV:
OpenCV is a large open-source computer vision, machine learning, and image processing library.
It may be used to recognise objects and faces in photos and movies. Python can process the
OpenCV array structure for analysis when it is combined with other modules such as NumPy. We
employ vector space and execute mathematical operations on these characteristics to identify
visual patterns and their different features.
Google Colab:
The Colab environment is a cloud-based Jupyter notebook environment. Most significantly, there
is no setup required, and the notebooks you create can be modified by your team members
concurrently. Many major machine learning libraries are supported by Colab, and can be loaded
into our notebook with ease.
5
Following is the block diagram for the working process of our project.
6
Project Requirements
Software Requirements
Hardware Requirements
7
Chapter 5
Code
from google.colab import drive
drive.mount('/content/drive')
cd /content/drive/MyDrive/WeaponDetection
!unzip /content/drive/MyDrive/WeaponDetection/images.zip -d '/
content/drive/MyDrive/WeaponDetection/Tensorflow/workspace'
import os
CUSTOM_MODEL_NAME = 'my_ssd_mobnet'
PRETRAINED_MODEL_NAME = 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu
-8'
PRETRAINED_MODEL_URL = 'http://download.tensorflow.org/models/
object_detection/tf2/20200711/
ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz'
TF_RECORD_SCRIPT_NAME = 'generate_tfrecord.py'
LABEL_MAP_NAME = 'label_map.pbtxt'
paths = {
'WORKSPACE_PATH': os.path.join('Tensorflow', 'workspace'),
8
'SCRIPTS_PATH': os.path.join('Tensorflow','scripts'),
'APIMODEL_PATH': os.path.join('Tensorflow','models'),
'ANNOTATION_PATH': os.path.join('Tensorflow', 'workspace','annot
ations'),
'IMAGE_PATH': os.path.join('Tensorflow', 'workspace','images'),
'MODEL_PATH': os.path.join('Tensorflow', 'workspace','models'),
'PRETRAINED_MODEL_PATH': os.path.join('Tensorflow', 'workspace',
'pre-trained-models'),
'CHECKPOINT_PATH': os.path.join('Tensorflow', 'workspace','model
s',CUSTOM_MODEL_NAME),
'OUTPUT_PATH': os.path.join('Tensorflow', 'workspace','models',C
USTOM_MODEL_NAME, 'export'),
'TFJS_PATH':os.path.join('Tensorflow', 'workspace','models',CUST
OM_MODEL_NAME, 'tfjsexport'),
'TFLITE_PATH':os.path.join('Tensorflow', 'workspace','models',CU
STOM_MODEL_NAME, 'tfliteexport'),
'PROTOC_PATH':os.path.join('Tensorflow','protoc')
}
files = {
'PIPELINE_CONFIG':os.path.join('Tensorflow', 'workspace','models
', CUSTOM_MODEL_NAME, 'pipeline.config'),
'TF_RECORD_SCRIPT': os.path.join(paths['SCRIPTS_PATH'], TF_RECOR
D_SCRIPT_NAME),
'LABELMAP': os.path.join(paths['ANNOTATION_PATH'], LABEL_MAP_NAM
E)
}
for path in paths.values():
if not os.path.exists(path):
if os.name == 'posix':
!mkdir -p {path}
if os.name == 'nt':
!mkdir {path}
if os.name=='nt':
!pip install wget
import wget
if not os.path.exists(os.path.join(paths['APIMODEL_PATH'], 'research
', 'object_detection')):
9
!git clone https://github.com/tensorflow/models {paths
['APIMODEL_PATH']}
if os.name=='posix':
!apt-get install protobuf-compiler
!cd Tensorflow/models/research && protoc object_detection/
protos/*.proto --python_out=. && cp object_detection/packages/tf2/
setup.py . && python -m pip install .
if os.name=='nt':
url="https://github.com/protocolbuffers/protobuf/releases/
download/v3.15.6/protoc-3.15.6-win64.zip"
wget.download(url)
!move protoc-3.15.6-win64.zip {paths['PROTOC_PATH']}
!cd {paths['PROTOC_PATH']} && tar -xf protoc-3.15.6-win64.zip
os.environ['PATH'] += os.pathsep + os.path.abspath(os.path.join(
paths['PROTOC_PATH'], 'bin'))
!cd Tensorflow/models/research && protoc object_detection/
protos/*.proto --python_out=. && copy object_detection\\packages\\
tf2\\setup.py setup.py && python setup.py build && python setup.py i
nstall
!cd Tensorflow/models/research/slim && pip install -e
VERIFICATION_SCRIPT = os.path.join(paths['APIMODEL_PATH'], 'research
', 'object_detection', 'builders', 'model_builder_tf2_test.py')
# Verify Installation
!python {VERIFICATION_SCRIPT}
!pip install tensorflow --upgrade
!pip uninstall protobuf matplotlib -y
!pip install protobuf matplotlib==3.2
import object_detection
!pip list
if os.name =='posix':
!wget {PRETRAINED_MODEL_URL}
!mv {PRETRAINED_MODEL_NAME+'.tar.gz'} {paths
['PRETRAINED_MODEL_PATH']}
!cd {paths['PRETRAINED_MODEL_PATH']} && tar -zxvf {PRETRAINED_MO
DEL_NAME+'.tar.gz'}
if os.name == 'nt':
wget.download(PRETRAINED_MODEL_URL)
10
!move {PRETRAINED_MODEL_NAME+'.tar.gz'} {paths
['PRETRAINED_MODEL_PATH']}
!cd {paths['PRETRAINED_MODEL_PATH']} && tar -zxvf {PRETRAINED_MO
DEL_NAME+'.tar.gz'}
labels = [{'name':'Pistol', 'id':1},{'name':'Rifle','id':2},
{'name':'ShotGun','id':3}]
with open(files['LABELMAP'], 'w') as f:
for label in labels:
f.write('item { \n')
f.write('\tname:\'{}\'\n'.format(label['name']))
f.write('\tid:{}\n'.format(label['id']))
f.write('}\n')
if not os.path.exists(files['TF_RECORD_SCRIPT']):
!git clone https://github.com/nicknochnack/GenerateTFRecord {pat
hs['SCRIPTS_PATH']}
!python {files['TF_RECORD_SCRIPT']} -x {os.path.join(paths
['IMAGE_PATH'], 'train')} -l {files['LABELMAP']} -o {os.path.join(pa
ths['ANNOTATION_PATH'], 'train.record')}
!python {files['TF_RECORD_SCRIPT']} -x {os.path.join(paths
['IMAGE_PATH'], 'test')} -l {files['LABELMAP']} -o {os.path.join(pat
hs['ANNOTATION_PATH'], 'test.record')}
if os.name =='posix':
!cp {os.path.join(paths['PRETRAINED_MODEL_PATH'], PRETRAINED_MOD
EL_NAME, 'pipeline.config')} {os.path.join(paths['CHECKPOINT_PATH'])
}
if os.name == 'nt':
!copy {os.path.join(paths['PRETRAINED_MODEL_PATH'], PRETRAINED_M
ODEL_NAME, 'pipeline.config')} {os.path.join(paths['CHECKPOINT_PATH'
])}
import tensorflow as tf
from object_detection.utils import config_util
from object_detection.protos import pipeline_pb2
from google.protobuf import text_format
config = config_util.get_configs_from_pipeline_file(files['PIPELINE_
CONFIG'])
config
11
pipeline_config = pipeline_pb2.TrainEvalPipelineConfig()
with tf.io.gfile.GFile(files['PIPELINE_CONFIG'], "r") as f:
proto_str = f.read()
text_format.Merge(proto_str, pipeline_config)
pipeline_config.model.ssd.num_classes = 4
pipeline_config.train_config.batch_size = 32
pipeline_config.train_config.fine_tune_checkpoint = os.path.join(pat
hs['PRETRAINED_MODEL_PATH'], PRETRAINED_MODEL_NAME, 'checkpoint', 'c
kpt-0')
pipeline_config.train_config.fine_tune_checkpoint_type = "detection"
pipeline_config.train_input_reader.label_map_path= files['LABELMAP']
pipeline_config.train_input_reader.tf_record_input_reader.input_path
[:] = [os.path.join(paths['ANNOTATION_PATH'], 'train.record')]
pipeline_config.eval_input_reader[0].label_map_path = files['LABELMA
P']
pipeline_config.eval_input_reader[0].tf_record_input_reader.input_pa
th[:] = [os.path.join(paths['ANNOTATION_PATH'], 'test.record')]
config_text = text_format.MessageToString(pipeline_config)
with tf.io.gfile.GFile(files['PIPELINE_CONFIG'], "wb") as f:
f.write(config_text)
TRAINING_SCRIPT = os.path.join(paths['APIMODEL_PATH'], 'research', '
object_detection', 'model_main_tf2.py')
command = "python {} --model_dir={} --pipeline_config_path={} --
num_train_steps=15000".format(TRAINING_SCRIPT, paths['CHECKPOINT_PAT
H'],files['PIPELINE_CONFIG'])
print(command)
!{command}
12
command = "python {} --model_dir={} --pipeline_config_path={} --
checkpoint_dir={}".format(TRAINING_SCRIPT, paths['CHECKPOINT_PATH'],
files['PIPELINE_CONFIG'], paths['CHECKPOINT_PATH'])
print(command)
!{command}
!tensorboard --logdir=/content/drive/MyDrive/TensorFlowProject/
Tensorflow/workspace/models/my_ssd_mobnet/eval #optional graphical r
epresention to represnt struct
import os
import tensorflow as tf
from object_detection.utils import label_map_util
from object_detection.utils import visualization_utils as viz_utils
from object_detection.builders import model_builder
from object_detection.utils import config_util
# Load pipeline config and build a detection model
configs = config_util.get_configs_from_pipeline_file(files['PIPELINE
_CONFIG'])
detection_model = model_builder.build(model_config=configs['model'],
is_training=False)
# Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore(os.path.join(paths['CHECKPOINT_PATH'], 'ckpt-
10')).expect_partial()
@tf.function
def detect_fn(image):
image, shapes = detection_model.preprocess(image)
prediction_dict = detection_model.predict(image, shapes)
detections = detection_model.postprocess(prediction_dict, shapes
)
return detections
13
import cv2
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
category_index = label_map_util.create_category_index_from_labelmap(
files['LABELMAP'])
IMAGE_PATH = os.path.join(paths['IMAGE_PATH'], 'test', 'Rifle105.jpg
')
img = cv2.imread(IMAGE_PATH)
image_np = np.array(img)
input_tensor = tf.convert_to_tensor(np.expand_dims(image_np, 0), dty
pe=tf.float32)
detections = detect_fn(input_tensor)
num_detections = int(detections.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections
# detection_classes should be ints.
detections['detection_classes'] = detections['detection_classes'].as
type(np.int64)
label_id_offset = 1
image_np_with_detections = image_np.copy()
viz_utils.visualize_boxes_and_labels_on_image_array(
image_np_with_detections,
detections['detection_boxes'],
detections['detection_classes']+label_id_offset,
detections['detection_scores'],
category_index,
use_normalized_coordinates=True,
max_boxes_to_draw=5,
min_score_thresh=.8,
agnostic_mode=False)
plt.imshow(cv2.cvtColor(image_np_with_detections, cv2.COLOR_BGR2RGB)
)
plt.figure(figsize=(200,200))
14
plt.show()
Output
15
Chapter 5
Conclusion
In this paper the details of how data has been collected from various sources and how data is
filtered and labelled. The Tensorflowobject identification model was implemented and trained
for weapon detection over our collected dataset in this study. We offer a solution that gives a
machine or robot a visionary sense to detect an unsafe weapon and can also inform whoever is
responsible for security , when a weapon or firearm is visible near the edge. The current
surveillance capabilities must be updated immediately with improved resources to enable
monitoring the effectiveness of human operators. Current outdated surveillance systems would
be completely replaced by smart surveillance systems.With the expanding availability of low-
cost storage, video infrastructure, and better video processing capabilities, current infrastructure
will become obsolete. We tried a few other models and found that Tensorflow object detection
api was the most accurate object detector out of all of them.
Future Scope
In future detection of hand grenades and larger "weapons" such as tanks, launchers, and other
larger weapons could be added to the project. A larger dataset could be used to train the model,
resulting in higher accuracy. It would be useful to have a more detailed classification of the
weapon detected, such as the weapon name and version
16
References
[2] Akshaya, R. Kumar H.S. and D. Bhat, A novel method to recognize object in Images using
Convolution Neural Networks 2019 International Conference on Intelligent Computing and
Control Systems (ICCS), 2019, pp. 425-430
[3] H. Jain, A. Vikram, Mohana, A. Kashyap and A. Jain, Weapon Detection using Artificial
Intelligence and Deep Learning for Security Applications 2020 International Conference on
Electronics and Sustainable Communication Systems (ICESC), 2020, pp. 193-198
[4] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection
with Region Proposal Networks,” inNeural Information Processing Systems 2015, 2015, pp. 241–
294.
[5] R. Olmos, S. Tabik, and F. Herrera, “Automatic Handgun Detection Alarm in Videos Using
Deep Learning,” CoRR, vol. abs/1702.0,Feb. 2017.
[6] G. K. Verma and A. Dhillon, “A Handheld Gun Detection using Faster R-CNN Deep
Learning,” Proceedings of the 7thInternational Conference on Computer and Communication
Technology - ICCCT-2017, 2017.
[7] M. M. Fernandez-Carrobles, O. Deniz, and F. Maroto, “Gun and Knife Detection Based on
Faster R-CNN for Video Surveillance,”Pattern Recognition and Image Analysis Lecture Notes in
ComputerScience, pp. 441–452, 2019.
[8] R. F. Azevedo Kanehisa and A. Almeida Neto, “Firearm Detection using Convolutional
Neural Networks,” in 11th InternationalConference on Agents and Artificial Intelligence, 2019,
pp. 707–714.
[9] Sandeep Kumar, Aman Balyan, Manvi Chawla, “Object detection and recognition in images,”
IJDRI1704166, vol. 5, pp. 1029-1034, 2017.
[10]Khushboo Khurana, ReetuAwast hi, “Techniques for object recognition in images and multi
object detection,” IJARCET, vol. 2, pp. 1383 -1388, 2013.
17
18