FACIAL RECOGNITION

by

Roll No: 1712013

Siddharth Dumbre
Roll No: 1712014
Anirudh Anilkumar

Roll No: 1712019

Arya Bafna

Roll No: 1712026

Saptarshi Kundu

Guide:
Prof. BHARGAVI KASLIKAR

Department of Electronics Engineering

K. J. Somaiya College of Engineering, Mumbai-77
(Autonomous College Affiliated to University of Mumbai)
ACKNOWLEDGEMENT
We feel profound pleasure in bringing out this project report for which we have to go from pillar
to post to make it a reality. On the submission of our project report on “ Facial Recognition”, we
would like to extend our gratitude and sincere thanks to our respected professor Prof. Bhargavi
Kaslikar for her constant motivation and support during the course of our work throughout this
semester. We truly appreciate and value her esteemed guidance and encouragement from the
beginning to the end of this project. We are indebted to her for having helped us shape the
problem and providing insights towards the solution.Last but not the least we express our sincere
thanks to the institute K.J Somaiya College of Engineering for providing such a platform for
implementing the ideas in our minds.
CONTENTS

SR No. TITLE PAGE No.

1 ABSTRACT 4

2 INTRODUCTION 5

3 IMPORTANCE OF PROJECT 5

4 LITERATURE REVIEW 6

5 METHODOLOGY AND IMPLEMENTATION 7

6 FLOWCHART 11

7 CODE 12

8 RESULTS 17

9 APPLICATION AREAS 18

10 CHALLENGES 18

11 ADVANTAGES 18

12 DISADVANTAGES 18

13 FUTURE SCOPE & CONCLUSION 19

14 REFERENCES 20
ABSTRACT

Human face detection and recognition play important roles in many applications such as video
surveillance and face image database management. In our project,we have studied & worked on
both face recognition and detection techniques and developed algorithms for them. In facial
recognition the algorithm used is Haar Cascade is a machine learning object detection algorithm
used to identify objects in an image or video and based on the concept of features proposed by
Paul Viola and Michael Jones in their paper in which we recognize an unknown test image by
comparing it with the known training images stored in the database as well as give information
regarding the person recognized. This technique produces results with high accuracy and
speed.In face detection, we have developed an algorithm that can detect human faces from an
image. We have taken real life examples and simulated the algorithms in Python successfully.
INTRODUCTION

The face is crucial for human identity. It is the feature which best distinguishes a person. Face
recognition is probably the biometric method that is used to identify people mainly from their
faces.Face recognition is an interesting and challenging problem, and impacts important
applications in many areas such as identification for law enforcement, authentication for banking
and security system access, and personal identification among others.Face recognition is one of
the most challenging aspects in the field of image analysis.Face recognition has been a topic of
active research since the 1980’s, proposing solutions to several practical problems.Automatic
face recognition is all about extracting those meaningful features from an image,putting them
into a useful representation and performing some classifications on them.Face recognition is an
important part of the capability of human perception system and is a routine task for humans,
while building a similar computational model of face recognition. The computational model not
only contribute to theoretical insights but also to many practical applications like automated
crowd surveillance, access control, design of human computer interface (HCI), content based
image database management,criminal identification and so on.The whole process can be divided
in three major steps where the first step is to find a good database of faces with multiple images
for each individual.The next step is to detect faces in the database images and use them to train
the face recognizer and the last step is to test the face recognizer to recognize faces it was trained
for.

IMPORTANCE OF FACIAL RECOGNITION

Face discovery has picked up a great deal of consideration because of its continuous
applications. A great deal of research has been done and as yet it continues for enhanced and
quick execution of the face discovery calculation.Face discovery isn't that simple as it involves a
bunch of varieties of picture appearance, for example, present variety (front, non-front),
impediment, picture introduction, enlightenment changes and outward appearance. There are a
few enterprises profiting from this innovation. Law authorization offices are utilizing face
acknowledgment to keep networks more secure. Retailers are avoiding wrongdoing and
savagery. Air terminals are enhancing voyagers' comfort and security. What's more, cell phone
organizations are utilizing face acknowledgment to furnish customers with new layers of
biometric security. OpenCV contains numerous pre-prepared classifiers for confront, eyes, grin
and so on. The XML documents of pre-prepared classifiers are put away in opencv.For confront
identification particularly, there are pre-prepared classifiers: Haar Cascade Classifier
LITERATURE REVIEW

We reviewed many reputed and published research papers related to facial recognition based on
which we drew significant conclusions for effective understanding of the above topic .Face
recognition began as early as 1977 with the first automated system being introduced By Kanade
using a feature vector of human faces . In 1983, Sirovich and Kirby introduced the principal
component analysis(PCA) for feature extraction . Using PCA, Turk and Pentland Eigenface was
developed in 1991 and is considered a major milestone in technology . Local binary pattern
analysis for texture recognition was introduced in 1994 and is improved upon for facial
recognition later by incorporating Histograms(LBPH) . In 1996 Fisherface was developed using
Linear discriminant analysis (LDA) for dimensional reduction and can identify faces in different
illumination conditions, which was an issue in the Eigenface method . Viola and Jones
introduced a face detection technique using HAAR cascades and ADABoost . In 2007, A face
recognition technique was developed by Naruniec and Skarbek using Gabor Jets that are similar
to mammalian eyes . In this project, HAAR cascades are used for face detection and LBPH are
used for face recognition.
METHODOLOGY & IMPLEMENTATION

OPENCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and
machine learning software library. OpenCV was built to provide a common infrastructure for
computer vision applications and to accelerate the use of machine perception in the commercial
products. Being a BSD-licensed product, OpenCV makes it easy for businesses to utilize and
modify the code. The library has more than 2500 optimized algorithms, which includes a
comprehensive set of both classic and state-of-the-art computer vision and machine learning
algorithms.These algorithms can be used to detect and recognize faces,identify objects, classify
human actions in videos, track camera movements, track moving objects, extract 3D models
of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a
high resolution image of an entire scene, find similar images from an image database,
remove red eyes from images taken using flash, follow eye movements, recognize scenery and
establish markers to overlay it with augmented reality, etc. It has C++, Python, Java and
MATLAB interfaces and supports Windows,Linux, Android and Mac OS. OpenCV leans mostly
towards real-time vision applications and takes advantage of MMX and SSE instructions when
available. As our system is face recognition based attendance monitoring system, Open CV is
very useful for it. Also it requires low RAM usage (approx 60–70 MB), so we are using it in our
system.

HAAR CASCADE

Paul Viola and Michael Jones presented an approach for object detection which minimizes
computation time while achieving high detection accuracy. Paul Viola and Michael Jones
proposed a fast and robust method for face detection which is 15 times quicker than any
technique at the time of release with 95% accuracy at around 17 fps.

A Haar wavelet is a mathematical fiction that produces square-shaped waves with a beginning
and an end and used to create box shaped patterns to recognise signals with sudden
transformations. An example is shown in figure 1. By combining several wavelets, a cascade can
be created that can identify edges, lines and circles with different colour intensities. These sets
are used in Viola Jones face detection technique in 2001 and since then more patterns are
introduced for object detection as shown in figure 1.
To analyse an image using Haar cascades, a scale is selected smaller than the target image. It is
then placed on the image, and the average of the values of pixels in each section is taken. If the
difference between two values pass a given threshold, it is considered a match. Face detection on
a human face is performed by matching a combination of different Haar-like-features. For
example, forehead, eyebrows and eyes contrast as well as the nose with eyes as shown below in
figure A single classifier is not accurate enough. Several classifiers are combined to provide an
accurate face detection system as shown in the block diagram below in figure 3.in this project, a
similar method is used effectively to by identifying faces and eyes in combination resulting in
better face detection. Similarly, in viola Jones method , several classifiers were combined to
create stronger classifiers. ADA boost is a machine learning algorithm that tests out several week
classifiers on a selected location and chooses the most suitable . It can also reverse the direction
of the classifier and get better results if necessary . Furthermore, Weight-update-steps can be
updated only on misses to get better performance. The cascade is scaled by 1.25 and reiterated in
order to find different sized faces. Running the cascade on an image using conventional loops
takes a large amount of computing power and time. Viola Jones used a summed area table (an
integral image) to compute the matches fast. First developed in 1984 , it became popular after
2001 when Viola Jones implemented Haar-cascades for face detection. Using an integral image
enables matching features with a single pass over the image.
LBPH RECOGNISER

Local binary patterns were proposed as classifiers in computer vision and in 1990 By Li Wang
the combination of LBP with histogram oriented gradients was introduced in 2009 that increased
its performance in certain datasets . For feature encoding, the image is divided into cells (4 x 4
pixels).Using a clockwise or counter-clockwise direction surrounding pixel values are compared
with the central as shown in figure 6. The value of intensity or luminosity of each neighbour is
compared with the centre pixel. Depending if the difference is higher or lower than 0, a 1 or a 0
is assigned to the location. The result provides an 8-bit value to the cell. The advantage of this
technique is even if the luminosity of the image is changed , the result remains same as before.
Histograms are used in larger cells to find the frequency of occurrences of values making the
process faster. By analysing the results in the cell, edges can be detected as the values change. By
computing the values of all cells and concatenating the histograms,feature vectors can be
obtained. Images can be classified by processing with an ID attached. Input images are classified
using the same process and compared with the dataset and distance is obtained. By setting up a
threshold, it can be identified if it is a known or unknown face. Eigenface and Fisherface
compute the dominant features of the whole training set while LBPH analyse them individually.

STEPS TO IMPLEMENT THE PROBLEM

A. Import the required modules

The Modules required to perform the facial recognition are cv2, os, image module and numpy.
cv2 is the OpenCV module and contains the functions for face detection and recognition.
OS will be used to maneuver with image and directory names.First, we use this module to extract
the image names in the database directory and then from these names individual number is
extracted, which is used as a label for the face in that image. Image module from PIL is used to
read the image in grayscale format. Numpy arrays are used to store the images.
B. Load the face detection Cascade
To Load the face detection Cascade the first step is to detect the face in each image. Once we get
the region of interest containing the face in the image, we use it for training the recognizer. For
the purpose of face detection, we will use the Haar Cascade provided by OpenCV. The haar
cascades that come with OpenCV are located in the directory of OpenCV installation.
haarcascade frontalface default.xml is used for detecting the face. Cascade is loaded using the
cv2 CascadeClassifier function which takes the path to the cascade xml file. if the xml file is in
the current working directory, then relative path is used.

C. Create the Face Recognizer Object

The next step involve creating the face recognizer object.The face recognizer object has
functions like FaceRecognizer.train to train the recognizer and FaceRecognizer.predict
to recognize a face. OpenCV currently provides Local Binary Patterns Histograms Face
Recognizer.We have used Local Binary Patterns Histograms Face Recognizer to perform face
recognition. With Local Binary Patterns it is possible to describe the texture and shape of a
digital image. This is done by dividing an image into several small regions from which the
features are extracted that can be used to get a measure for the similarity between the images.

D. Prepare the training set & perform the training

To create the function to prepare the training set, we will define a function that takes the absolute
path to the image database as input argument and returns a tuple of 2 lists, one containing the
detected faces and the other containing the corresponding label for that face. For example, if the
ith index in the list of faces represents the 4th individual in the database, then the corresponding
ith location in the list of labels hasvalue equal to 4.Now to perform the training using the Face
Recognizer. Train function. It requires 2 arguments, the features which in this case are the images
of faces and the corresponding labels assigned to these faces which in this case are the individual
number that we extracted from the image names.

E. Testing

For testing the Face Recognizer, we check if the recognition was correct by comparing the
predicted label predicted with the actual label actual. The label actual is extracted using the os
module and the string operations from the name of the image.
FLOWCHART
CODE

A) CREATE THE DATASET

import cv2

import numpy as np

detector = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

cam = cv2.VideoCapture(0)

id = input('enter user id=')

sampleNum = 0

while (True):

ret, img = cam.read();

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

faces = detector.detectMultiScale(gray, 1.5, 5)

for (x, y, w, h) in faces:

# incrementing sample number

sampleNum = sampleNum + 1

# saving the captured face in the dataset folder

cv2.imwrite("data" "/User." + str(id) + "." + str(sampleNum) + ".jpg", gray[y:y + h, x:x + w])

cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 2)

cv2.waitKey(100);

cv2.imshow('Face', img)

cv2.waitKey(1);

if sampleNum > 20:

break

cam.release()

cv2.destroyAllWindows()
B) TRAIN THE DATASET

import numpy as np

from PIL import Image

import os, cv2

# Method to train custom classifier to recognize face

def train_classifer(data_dir):

# Read all the images in custom data-set

path = [os.path.join(data_dir, f) for f in os.listdir(data_dir)]

faces = []

ids = []

# Store images in a numpy format and ids of the user on the same index in imageNp and id lists

for image in path:

img = Image.open(image).convert('L')

imageNp = np.array(img, 'uint8')

id = int(os.path.split(image)[1].split(".")[1])

faces.append(imageNp)

ids.append(id)

ids = np.array(ids)

# Train and save classifier

clf = cv2.face.LBPHFaceRecognizer_create()

clf.train(faces, ids)

clf.write('classifier.yml')

print("pics upload")

train_classifer("data")
C) RECOGNISE THE DATASET

import cv2

def generate_dataset(img, id, img_id):

cv2.imwrite("data/user."+str(id)+"."+str(img_id)+".jpg",img)

def draw_boundary(img, classifier, scaleFactor, minNeighbors, color, text, clf):

#convert image to gray scale

gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

#detect features in gray scale,return coordinates,width and height of features

features = classifier.detectMultiScale(gray_img, scaleFactor, minNeighbors)

coords = []

#draw rectangle around the feature and label it

for (x, y, w, h) in features:

cv2.rectangle(img, (x, y), (x + w, y + h), color, 2)

#predict id of the user

id, _=clf.predict(gray_img[y:y+h,x:x+w])

#check id of the user and lable rectangle accordingly

if id==1:

cv2.putText(img, "saptarshi", (x, y - 4), cv2.FONT_HERSHEY_SIMPLEX, 0.8, color, 1, cv2.LINE_AA)

coords = [x, y, w, h]

# Method to recognize the person

def recognize(img, clf, faceCascade):

color = {"blue": (255, 0, 0), "red": (0, 0, 255), "green": (0, 255, 0), "white": (255, 255, 255)}

coords = draw_boundary(img, faceCascade, 1.1, 10, color["white"], "Face", clf)

return img
def detect(img,faceCascade,eyesCascade,noseCascade,mouthCascade,img_id):

color = {"blue": (255, 0, 0), "red": (0, 0, 255), "green": (0, 255, 0), "white": (255, 255, 255)}

coords = draw_boundary(img, faceCascade, 1.1, 10, color["white"], "Face",clf)

return img

if len(coords)==4:

roi_img = img[coords[1]:coords[1]+coords[3], coords[0]:coords[0]+coords[2]]

user_id = 1

generate_dataset(roi_img, user_id, img_id)

coords = draw_boundary(img, eyesCascade, 1.1, 14, color['blue'], "Eyes")

coords = draw_boundary(img, noseCascade, 1.1, 5, color['blue'], "Nose")

coords = draw_boundary(img, mouthCascade, 1.1, 20, color['blue'], "Mouth")

return img

# Loading classifier

faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

eyesCascade = cv2.CascadeClassifier('haarcascade_eye.xml')

noseCascade = cv2.CascadeClassifier('Nariz.xml')

mouthCascade = cv2.CascadeClassifier('Mouth.xml')

# Loading custom classifier to recognize

clf = cv2.face.LBPHFaceRecognizer_create()

clf.read("classifier.yml")
# Capturing real time video stream. 0 for built-in web-cams, 0 or -1 for external web-cams

video_capture = cv2.VideoCapture(0)

img_id=0

while True:

# Reading image from video stream

_, img = video_capture.read()

# Call method we defined above

img=detect(img,faceCascade,eyesCascade,noseCascade,mouthCascade,img_id)

img=recognize(img,clf,faceCascade)

# Writing processed image in a new window

cv2.imshow("face detection", img)

img_id+=1

if cv2.waitKey(1) & 0xFF == ord('q'):

break

# releasing web-cam

video_capture.release()

# Destroying output window

cv2.destroyAllWindows()
RESULT

FACE SAMPLES IN THE DATASET

DETECTED FACE WITH NAME LABEL

APPLICATION AREAS

● Public security
● Attendance Monitoring
● Law enforcement and commerce
● Credit card verification
● Criminal identification
● Human-computer intelligent interaction
● Digital libraries and information security

CHALLENGES

● Presence or absence of structural components: Facial features such as beards, mustaches,

and glasses may or may not be present and there is a great deal of variability among these
components including shape,color and size.
● Occlusion: Faces may be partially occluded by other objects. For example, in an image
with a group of people, some faces may partially occlude other faces (face identification).
● Image orientation: Face images directly vary for different rotations about the camera’s
optical axis.
● Imaging conditions: When the image is formed, factors such as lightning and camera
characteristics affect the appearance of a face.
● Age: Images taken after one or two year’s gap may not match with the images in the
database.

ADVANTAGES

● High detection accuracy(approx 95-99%)

● Low false positive rate
● Low cost and less storage space required

DISADVANTAGES

● Computationally Complex and slow

● Longer training time
FUTURE SCOPE

Our current facial recognition system acquires face images from files so camera and scanner can
be implemented for greater flexibility.Further this system can be converted into low cost
attendance monitoring systems with help of limited hardware like arduino or raspberry pi
module,thus creating a online database with automatic updating .Another addition would be that
the human-computer interaction would be vastly improved in various domains and hence it
would benefit mankind in immense way.

CONCLUSION

Face recognition is one of the challenging aspects in the field of image analysis and computer
vision. The focus towards face recognition has been increased in the last few years due to its
enormous applications in different domains. The algorithm related to face recognition technique
was thoroughly studied taking a number of test images and varying the conditions and variables.
The proposed work can further be improved using other optimization algorithms and can also be
applied on other benchmarked databases . Using Haar-cascades for face detection worked
extremely well even in relatively tough conditions. Real time video speed was satisfactory as
well devoid of noticeable frame lag. Considering all factors, LBPH combined with Haar-
cascades can be implemented as a cost effective face recognition platform. An example is a
system to identify known troublemakers in a mall or a supermarket to provide the owner a
warning to keep him alert or for automatic attendance taking in a class.
REFERENCES

1. Omkar Biradar,Anurag Bhave ,“Face Recognition based Attendance Monitoring

system”,Volume:06, Issue:01,Jan 2019
2. Pratibha Sukhija,Sunny Behal,Pritpal Singh, “Face Recognition system using Genetic
Algorithm” ,Science Direct
3. Sujata G.Bhele,V.H.Mankar, “Review paper on Face Recognition Techniques,Advanced
Research ISSN:2278-1323 in Computer Engineering and Technology
(IJARCET)”,Volume: 1 ,pages 339-343,Issue:8
4. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features.
In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition.CVPR 2001, volume 1, pages I–511–I–518 vol.1, 2001