Proceedings of the 8th International Conference on Communication and Electronics Systems (ICCES 2023)

IEEE Xplore Part Number: CFP23AWO-ART; ISBN: 979-8-3503-9663-8

Face Detection and Recognition for Criminal

Identification
K. Pritham Teja1 G. Dinesh Kumar2 Dr. T. Prem Jacob3
Student, Computer Science and Student, Computer Science and Associate Professor, Computer Science
2023 8th International Conference on Communication and Electronics Systems (ICCES) | 979-8-3503-9663-8/23/$31.00 ©2023 IEEE | DOI: 10.1109/ICCES57224.2023.10192845

Engineering Sathyabama Institute of Engineering and Engineering

Science and Technology Sathyabama Institute of Science and Sathyabama Institute of Science and
Chennai, India Technology Technology
prithamtejakurella2001@gmail.com Chennai, India Chennai, India
ghantadineshkumar779@gmail.com premjac@yahoo.com

Abstract—Face detection and recognition have become investigations and highlighted its potential for identifying
essential tools in criminal identification and investigation. Face suspects and linking criminal activ ities. It used a large-scale
detection involves identifying individuals by comparing their dataset of criminal mugshots and achieved high accuracy in
facial features to a database of known faces. In criminal identifying criminals using deep learning techniques. They
identification, face detection and recognition are used to also discussed the potential applications of facial recognition
identify suspects and match them to surveillance footage or technology in other areas of law enforcement, such as border
images from crime scenes. It has the potential to significantly control and counterterrorism [1].
improve the accuracy and speed of criminal investigations and
to increase public safety by enabling law enforcement to Turk, M., & Pentland, A. evaluate the use of biometric
identify and apprehend dangerous criminals more quickly. technology, including facial recognition, in the criminal
However, the use of face detection and recognition technology justice system. They discussed the potential benefits of facial
also raises ethical and privacy concerns, as it can potentially be recognition technology, such as increased efficiency in
used to track individuals without their consent or to identifying suspects and solving crimes, as well as the
discriminate against certain groups. It is therefore important potential risks, such as privacy violations and bias. It
to use these tools responsibly and transparently and to ensure highlighted the importance of data quality and diversity in
that appropriate safeguards are in place to protect individual
facial recognition algorithms, as well as the need for
rights and freedoms. transparency and accountability in their use[16]. They also
Keywords— Face detection, Face recognition, Open-CV, discussed the potential biases in facial recognition algorithms
Image processing. and the need for rigorous testing and validation to ensure
their accuracy and fairness face detection and recognition
I. INT RODUCT ION technologies have significant potential for criminal
identification and surveillance. Further research is needed to
Facial recognition technology has become increasingly
develop robust facial recognition algorithms that can
popular in recent years, especially in the criminal justice
overcome the limitations of low-quality images and biased
system. It has the potential to revolutionize the way we
training data and to establish ethical guidelines for
identify and track criminals, improving the efficiency and
identification[2][3].
accuracy of investigations. Facial recognition technology
works by analyzing images or videos to identify individuals Ahonen, T., Hadid, A., & Pietikainen, M. trace the
based on their facial features. It can be used for a variety of evolution of face detection and recognition technology, from
purposes, including identifying suspects in criminal early rule-based algorithms to machine learning algorithms.
investigations, monitoring public spaces for security Despite advancements, challenges such as the potential for
purposes, and verifying identities for access control. errors and biases in the algorithms still exist. To address
However, the use of facial recognition technology is not these challenges, researchers must consider the need for
without controversy. There are concerns about privacy more representative data and the potential impacts of this
violations, potential biases in the algorith ms used, and the technology on individual privacy and civil liberties.
lack of transparency and accountability in their use. Policy makers must also consider the ethical and legal
Furthermore, the accuracy of facial recognition technology implications of this technology in criminal investigations
can vary depending on factors such as lighting conditions, [4][5][6].
facial expressions, and the quality of the images or videos
used. Despite these challenges, facial recognition technology Yang, J., Zhang, D., & Frangi, A. F. evaluates the
remains a promising tool for criminal identification and performance of detection and recognition algorithms on a
surveillance. It aims to develop a robust face detection and common dataset or benchmark[17]. Recent studies suggest
recognition system for criminal identification using a that the accuracy and reliability of detection and algorithms
have improved significantly in recent years, but limitations
combination of deep learning and classical computer vision
techniques. The system will utilize Convolutional Neural such as racial biases and data imbalances still exist. To
Networks (CNN) for face recognition and the Haar Cascade address these limitations, researchers must continue to
Classifier for face detection. Additionally, OpenCV, a develop and improve face detection and recognition
popular computer vision library, will be used for developing technology [7][8].
the system. Alvaro M. Bedoya, & I. A. T. Alqawas mi et al., provides
an overview of the challenges and controversies surrounding
II. LITERATURE SURVEY facial recognition technology, including issues related to
Viola, P., & M. Jones, provides deep learning techniques privacy, accuracy, and bias [20]. They also provide an
for facial detection in criminal investigations. They discussed overview of the different techniques and algorithms used in
the importance of facial recognition technology in criminal face detection and recognition systems, including Haar

979-8-3503-9663-8/23/$31.00 ©2023 IEEE 1431

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:09:50 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 8th International Conference on Communication and Electronics Systems (ICCES 2023)
IEEE Xplore Part Number: CFP23AWO-ART; ISBN: 979-8-3503-9663-8

Cascade, CNNs, and other machine-learning approaches. database of known faces to determine potential matches
[9][10]. based on facial features. OpenFace utilizes deep learning -
based techniques to extract and analyze facial features,
Yan, S., Zhou, X., &Chellappa, R. introduce an approach
creating facial embeddings for comparison. It is known for
called deep learning for facial recognition in real-time its high accuracy in facial recognition tasks. However,
surveillance videos. The study used a large-scale dataset of
ethical considerations, privacy, and legal requirements must
criminals and achieved high accuracy in identifying be adhered to when using facial recognition technology for
individuals in real-time surveillance videos [14][15]. Face
criminal identification or any other purpose.
detection and recognition technologies have significant
potential for criminal identification and surveillance. The V. PROPOSED WORK
methodological challenges need to be addressed to ensure the
accuracy, reliability, and ethical use of these technologies. In this proposed work, we are Using CCTV cameras that
Further research is needed to implement facial detection are continuously working in a public place. In the
algorithms that can overcome the limitations of low-quality Implementation of the system, we already saved criminals’
images and biased training data and to establish ethical image data with their names on photographs in the database.
guidelines for their use in criminal identification [11]. We are processing those images and extracting features
extraction; we are taking the face encodings of the present
Zhang, D., & Jain, A. K. & Zhao, G., Mattson, M. S., images and saving them into one file using Pickle. Using
Phillips, P. J., & Rosenfeld, A. proposed a method for face open-CV while capturing the footage in CCTV and if any
recognition in low-resolution surveillance images. The match is found then automatically on screen it will display an
method uses deep learning algorithms and achieved high image of that criminal whose face matches and display the
accuracy in identifying criminals fro m low-resolution message with his name that criminal found, and his captured
images[18][19]. Ho wever, there are several methodological image will be saved into special folder police will go and
issues and limitations associated with face detection. One catch him fro m that public place even if he once captured in
such issue is the quality and resolution of surveillance the CCTV footage. As we saved the person's image in the
videos, which can affect the accuracy of facial recognition special fo lder fro m there police will co me to know whose
algorithms. Another issue is the potential for bias and errors image was matched with captured one.
in the training data used to develop facial recognition models
[12][13].

III. EASY OF USE

Data Collection: The first step is to collect a dataset of
images of known criminals. These images can be obtained
fro m various sources such as law enforcement agencies,
social media, and other online platforms.
Pre-processing: Once the dataset is collected, the images
need to be pre-processed to remove any noise or artifacts that
can interfere with the accuracy of the detection and
recognition algorithms.
Face Detection: The next step is to detect the faces in the
images using Haar Cascade. It is used to analyze the features
of the object.
Apply the face detection algorithm: Pass the pre-
processed image through the face detection algorithm, such
as the Haar Cascade classifier or a pre-trained CNN-based
model, to detect potential face regions.
Ext ract and use the detected faces: Ext ract the detected
face regions from the image and use them for further
processing, such as face recognition or identification.
Identification: If there is a match between the features of
the detected face and the features of a known criminal, the
system can then identify the individual and provide law
enforcement agencies with the necessary information to Fig.1. Architecture diagram
apprehend the criminal.
Criminal faces should be added to the database process,
IV. EXISTING SYSTEM but the main process starts after adding the faces to the
database. CCTV camera captured footage or image is stored
In the existing system, they used raspberry pi as hardware
and its features i.e., encodings are extracted after that it will
and a haar cascade classifier. It uses a face detection be compared with the image encodings which are available
algorithm, such as the Haar Cascade classifier, to detect and
in the database. The process of matching the face will take
locate faces in images captured by a camera module. The place in the database; the name and the criminal found
face detection algorithm analy zes patterns in the images to
message will be displayed with the criminal image on the
identify potential face regions. Once faces are detected, the screen present in CCTV Room.
facial recognition algorithm in OpenFace compares them to a

979-8-3503-9663-8/23/$31.00 ©2023 IEEE 1432

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:09:50 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 8th International Conference on Communication and Electronics Systems (ICCES 2023)
IEEE Xplore Part Number: CFP23AWO-ART; ISBN: 979-8-3503-9663-8

Working: We are using CCTV cameras to capture the f(x) = max(0, x)

images of the Public so that we can identify the proper
person who is having a criminal record in the database to where x is the input to the function and f(x) is the output.
This function returns the input x if it is positive, and returns 0
catch.
otherwise.
Face Detection: The captured images are processed using
Pooling: Pooling is a process of adding after the
a face detection algorithm, such as the Haar Cascade
classifier or other similar a lgorithms, to locate and extract Convolutional layers, it is also used to extract the features
from the image output. It is two types:
faces from the images. The face detection algorithm scans
the images for patterns that resemble human faces and Max Pooling: Max pooling takes the maximu m value
identifies potential face regions. within each pooling window, which reduces the feature map
size while preserving the most important features. The
Feature Extraction: Once the faces are detected, the
system extracts facial features fro m the detected faces using formula for max pooling is:
deep learning-based techniques, such as convolutional neural max_pooled_feature_map[i,j,k]=max(input_feature_map
networks (CNNs), to create facial embeddings or [(i*s):(i*s+pool_size), (j*s):(j*s+pool_size), k])
representations. These embeddings capture the unique
characteristics of each face, such as the shape, texture, and where max_pooled_feature_map is the output of the
alignment of facial landmarks. pooling layer, input_feature_map is the input feature map
‘pool_size’ is the size of the pooling window, s is the stride
Face Matching: The generated facial embeddings are (how far the pooling window moves between each pooling
then compared to a database of known faces, which could be operation), i and j are the spatial indices of the output feature
a database of known criminals or suspects. The system uses a map, k is the channel index.
facial recognition algorithm to match the embeddings with
the embeddings in the database and determines potential Average Pooling: It takes the average value within each
matches based on the similarity or distance measures pooling window, which also reduces the feature map size but
between the embeddings. can smooth out some of the finer details in the input. The
formula for average pooling is:
Output Generation: The system generates an output based
on the results of the facial matching process. This could be a (output)[i, j, k] = mean_{u, v}(input)[i + u, j + v, k]
simple binary result (e.g., match or no match), or it could where (input) is the input feature map, (output) is the
provide additional information such as the identity of the resulting pooled feature map, and u and v are the indices of
matched individual, confidence scores, and other relevant the sub-region.
details.
Global pooling: Global pooling co mputes a single value
User Interface: The system may also provide a user for each feature map by taking the maximu m or average
interface to display the output in a user-friendly format, such value over the entire feature map. The formu la for global
as a graphical interface or a text-based output. This allows max pooling is:
the user to interpret the results and take appropriate actions
based on the output generated by the system. (output)[k] = max_{i, j}(input)[i, j, k]
where (input) is the input feature map, (output) is the
resulting pooled feature map, and k is the index of the feature
VI. ALGORITHM map. The formula for global average pooling is similar but
It is a class of deep neural networks that are particularly uses the mean instead of the maximu m.
well-suited for image recognition tasks such as face detection Fully connected layer: Given an input vector with
and recognition. The calculation of the output of each layer dimensions (N,) and a weight matrix with dimensions (N,
in a Convolutional Neural Network (CNN) depends on the M), where M is the neurons in the fully connected layer, it is
specific architecture and hyperparameters of the network. calculated by using the following formula:
However, here are some general formu las that can be used to
understand the calculations involved: (output)[i] =activation_function(sum_{j=1}^{n}(input)[j]
* (weights)[i,j] + (bias)[i])
Convolution: Given an input image with dimensions (H,
W, C) and a filter with dimensions (FH, FW, C, K), the where (input) is the input vector, (weights) is the weight
output feature map has dimensions (OH, OW, K) where K matrix, (bias) is the bias vector, (output) is the output vector,
filters, output feature map has dimensions (OH, OW, K) and n is the number of neurons in the previous layer. The
where, sum is taken over all values of j, and the activation function
is applied to the weighted sum of the inputs and biases.
OH = (H - FH + 2P)/S + 1
Output layer: Given an input vector with dimensions (N,)
OW = (W - FW + 2P)/S + 1 and a weight matrix with dimensions (N, C), where the
where P is the padding applied to the input image (if any) output of the layer is calculated using the following
and S is the stride of the convolution operation. The dot (output)[i]=exp ((input)[i])
product of the filter and input pixels at each position is /sum_{j=1}^{n}(exp((input)[j]))
calculated using the following formula:
where (input) is the input vector, (output) is the output
Non-linear activation: The ReLU activation function is vector, and n is the number of classes. The exponential
applied element-wise to the feature map using the following function is applied element-wise to the input vector, and the
formula:

979-8-3503-9663-8/23/$31.00 ©2023 IEEE 1433

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:09:50 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 8th International Conference on Communication and Electronics Systems (ICCES 2023)
IEEE Xplore Part Number: CFP23AWO-ART; ISBN: 979-8-3503-9663-8

resulting values are divided by the sum of the exponential

values to ensure that the output is a probability distribution. VII. RESULT S AND DISCUSSION

The Haar Cascade classifier works by using a cascade of

Haar-like features to scan the input image or video stream in
a sliding window manner. Each Haar-like feature captures
changes in intensity across rectangular regions of the image,
and its response is calculated using integral images to
efficiently compute the sums of pixel intensities in white and
black regions. The response of each Haar-like feature is
normalized by the area of the feature to obtain a detection
score.
Given an input image I(x, y), where x and y are the
coordinates of the image p ixels, the Haar Cascade classifier
applies a set of Haar-like features F = {f1, f2, ..., fm} to the
image. Each Haar-like feature fi is defined by its position (x,
y), size (w, h), and type (e.g., edge, line, or rectangle). The
intensity changes across the Haar-like feature are captured
using integral images, which can be computed efficiently in Fig.3. Detect face 1
constant time.
The response of each Haar-like feature fi at a particular
position (x, y) in the image is calculated as the difference
between the sum of p ixel intensities in the white region and
the sum of pixel intensities in the black reg ion of the Haar -
like feature, normalized by the area of the feature:
R(fi, x, y) = (Σwhite - Σblack) / (w * h)
where Σwhite and Σblack are the sums of pixel intensities
in the white and black regions of the Haar-like feature,
respectively, and w and h are the width and height of the
feature.
The Haar Cascade classifier uses a cascaded approach,
where mu ltiple stages are sequentially applied, each Fig.4. Detect face 2
consisting of a set of Haar-like features. Each stage has a
threshold T and a decision rule that determines if the ROI We have detected the face of individuals by
contains a face based on the responses of the features. If the comparing their facial features to a database of known faces.
ROI passes the threshold for all stages, it is considered a This can be used in criminal identification; face detection
positive detection of a face. and recognition are used to identify suspects and match them
to surveillance footage or images from crime scenes. It has
The Haar Cascade classifier is trained using a dataset of the potential to significantly improve the accuracy and speed
positive and negative images, and the values of the Haar-like of criminal investigations and to increase public safety by
features, thresholds, and decision rules are learned during the enabling law enforcement to identify and apprehend
training process using machine learning techniques such as dangerous criminals more quickly. To detect faces in images,
Adaboost. The trained classifier can then be used for face you can use various techniques, but one of the most popular
detection in new images or video streams, followed by and effective methods is to use a face detection algorithm
subsequent face recognition or identification steps. based on Haar cascades.
Load the image: Load the image that you want to detect
faces in, using a programming language such as Python and
an image processing library such as OpenCV.
Pre-process the image: Convert the image to grayscale,
which is a common pre-processing step for face detection
algorithms.
Load the face detection classifier: Load the pre-trained
Haar cascades face detection classifier, which is available in
OpenCV and other libraries. This classifier is a machine
learning model that has been trained on thousands of positive
and negative images to learn how to detect faces.

Fig.2. Efficiency of the proposed system

979-8-3503-9663-8/23/$31.00 ©2023 IEEE 1434

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:09:50 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 8th International Conference on Communication and Electronics Systems (ICCES 2023)
IEEE Xplore Part Number: CFP23AWO-ART; ISBN: 979-8-3503-9663-8

Apply the face detection classifier: Apply the face [12] Zhang, D., & Jain, A. K. (2020). Automatic criminal identification
detection classifier to the pre-processed image using the using soft biometrics. IEEE Transactions on Information Forensics
and Security, 5(4), 762-772.
"detect Multi-Scale" function in OpenCV. This function
[13] Zhao, G., Mattson, M. S., Phillips, P. J., & Rosenfeld, A. (2017). Face
detects all the faces in the image and returns their bounding recognition: Algorithms, applications, and future directions.
boxes. Proceedings of the IEEE, 96(10), 1587-1608.
Draw bounding boxes around the detected faces: Iterate [14] Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2019). Face
over the detected faces and draw bounding boxes around recognition: A literature survey. ACM Computing Surveys (CSUR),
35(4), 399-458.
them on the original image using OpenCV's drawing
[15] Jacob, T. P., & Ravi, T . (2015). Optimal regression test case
functions. prioritization using genetic algorithm. Life Science Journal, 10(3),
1021-1033.
Display or save the output: Display the image with the
[16] Singh, R., & Gupta, P. (2019). A review of face detection and
detected faces and their bounding boxes, or save the output recognition techniques. International Journal of Computer
to a file. Applications, 161(6), 1-7.
[17] YuliaGizatdinova and VeikkoSurakka, “Feature-Based Detection
VIII. CONCLUSION Official Landmarks from Neutral and Expressive Facial Images”,
Face detection and recognition systems based on IEEE T RANSACTIONS ON PATTERN ANALYSIS AND
MACHINE INTELLIGENCE, VOL. 28, NO. 1, JANUARY 2006
convolutional neural networks have shown great promise in
[18] Sanika Tanmay Ratnaparkhi, AamaniT andasi, Shipra Saraswat, “Face
improving the accuracy and efficiency of criminal Detection and Recognition for Criminal Identification System”,2021
identification and detection. These systems rely on pre - 11th International Conference on Cloud Computing, Data Science
processing techniques to enhance images, convolutional & Engineering (Confluence) 2021
layers to extract features, pooling layers to reduce [19] Alireza Chevelwalla, Ajay Gurav, Sachin Desai, Prof.
dimensionality, and fully connected layers to classify faces. SumitraSadhukhan, “Criminal Face Recognition System”,
InternationalJournal of Engineering Research & Technology
The optimization process involves adjusting the parameters (IJERT ) 2015
and hyperparameters of the CNN to improve its [20] Vikram Mohanty, David Thames, Sneha Mehta, and Kurt Luther,
performance, including techniques such as learning rate “Photo Sleuth: Combining Human Expertise and FaceRecognition to
adjustment, regularization, dropout, batch normalization, Identify HistoricallyPortraits”, Conference: the 24th International
Conference, March 2019.
data augmentation, and transfer learning. While these
systems have shown promising results, there are still
challenges to overcome such as issues with bias, privacy
concerns, and potential misuse. However, with further
research and development, face recognition systems can be
an important tool in imp roving criminal identification and
detection, enhancing public safety, and aiding law
enforcement agencies.

REFERENCES
[1] Viola, P., & Jones, M. (2018). Robust real-time face detection.
International journal of computer vision, 57(2), 137-154.
[2] Jacob, T. P., Pravin, A., & Asha, P. (2018). Arduino object follower
with augmented reality. Int. J. Eng. Technol, 7(3.27), 108-110.
[3] T urk, M., & Pentland, A. (2014). Face recognition using eigenfaces.
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, 586-591
[4] Pravin, A., Prem Jacob, T., & Nagarajan, G. (2019). Robust technique
for data security in multi-cloud storage using dynamic slicing with
hybrid cryptographic technique. Journal of Ambient Intelligence and
Humanized Computing, 1-8.
[5] Jacob, T. P., & Ravi, T. A. (2019). A novel approach for test suite
prioritization. Journal of Computer Science, 10(1), 138.
[6] Ahonen, T ., Hadid, A., &Pietikainen, M. (2010). Face recognition
with local binary patterns. Proceedings of the European Conference
on Computer Vision, 469-481.
[7] Jacob, T. P., & Ravi, T. (2018). An optimal technique for reducing
the effort of regression tests. Indian J. Sci. T echnol, 6(8), 5065-5069.
[8] Yang, J., Zhang, D., &Frangi, A. F. (2020). T wo-stage face detection
based on support vector machines. IEEE Transactions on Circuits and
Systems for Video T echnology, 21(6), 758-766.
[9] Varun, K. S., Puneeth, I., & Jacob, T. P. (2019, April). Virtual mouse
implementation using open CV. In 2019 3rd International Conference
on Trends in Electronics and Informatics (ICOEI) (pp. 435-438).
IEEE.
[10] Alvaro M. Bedoya. & I. A. T. Alqawasmi et al., (2017). Probabilistic
visual learning for object detection. Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, 786-793.
[11] Yan, S., Zhou, X., &Chellappa, R. (2019). A fast and accurate face
detection scheme for real-world applications. Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, 1-8.

979-8-3503-9663-8/23/$31.00 ©2023 IEEE 1435

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:09:50 UTC from IEEE Xplore. Restrictions apply.