Proceedings of the World Congress on Engineering and Computer Science 2008

WCECS 2008, October 22 - 24, 2008, San Francisco, USA

Person Authentication Using Face Detection

2
1
V.Vaidehi, S.Vasuhi, R.Kayalvizhi, K.Mariammal, 3Raghuraman.M.B, 4Sundara Raman.V,
5
Meenakshi.L, 6Anupriyadharshini.V, 7Thangamani.T

A complete pattern recognition system consists of: i) a sensor

Abstract This paper presents a technique for Person
authentication using face detection. A skin color based technique is that gathers the observations to be classified or described, ii) a
used for detecting frontal human face from the input image. The feature extraction mechanism that computes numeric or
visual features namely Profile Fourier Coefficients (PFC) are then symbolic information from the observations, iii) a classification
extracted using template matching. Using Principal Component or description scheme that does the actual job of classifying or
Analysis (PCA), the extracted features are projected into describing observations, based on the extracted features. The
one-dimensional subspace. This approach preserves the Euclidean classification or description scheme usually uses one of the
structure of face subspace that contains the local information. The following approaches: statistical (or decision theoretic),
projected face image is recognized using K nearest neighbor (KNN)
syntactic (or structural)[4].
classifier. This method is better and efficient than other existing
methods due to its reduced error rate. It works for images taken Many face recognition techniques [2], [5], [11], [12] have
under various lighting conditions and is also independent of the been studied; one of the most recently used techniques is the
image type and resolution. The proposed Detection and appearance-based method. In general, a face image of size n x m
Recognition algorithm (DR) can detect multiple faces of different
pixels is represented as a vector in an n x 1 dimensional space.
sizes with a wide variety of facial variations. Further, the algorithm
can detect both dark skin-tone and bright skin-tone because of the
This leads one to consider methods of dimensionality reduction
nonlinear transform of the Cb-Cr color space. that allows one to represent the data in a lower dimensional space.
Take the following typical cases: a face recognition system based
on n x m gray scale images which, by row concatenation, can be
Index Terms Euclidean space, face detection, k nearest neighbor transformed into n dimensional real vectors. In practice, one
classifier, Principal Component Analysis, Profile Fourier could have images of m = n = 256, or 65536-dimensional vectors
Coefficients. used as the classification system, the number would be
exceedingly large memory for the entire training database.
I. INTRODUCTION Therefore dimensionality reduction is essential. In practical
Person authentication involves verification of a persons situations, when n is prohibitively large, one is often forced to
identity based on his/her physiological or behavioral use linear techniques [4]. Consequently, projective maps [1]
characteristics. Image processing is a form of signal processing have been the subject of considerable investigation [6].
for which the input is an image, such as photographs or frames of Eigenface [7] method is the most popular linear techniques for
video, the output of image processing [3] can be either an image face recognition. Eigenface applies Principal Component
or a set of characteristics or parameters related to the image. Analysis (PCA) to project the data points along the directions of
Most image-processing techniques involve treating the image as maximal variances. Eigenface method is unsupervised, ability to
a two-dimensional signal and applying standard learn and later recognize new faces. There is another popular
signal-processing techniques to it. Face detection procedure technique, Linear Discriminant Analysis (LDA) which is a
determines the locations and sizes of human faces in arbitrary supervised algorithm and this approach projects the face images
(digital) images. It detects facial features and ignores anything along the directions optimal for discrimination. But the
else, such as buildings, trees and bodies. Face detection can be eigenface is better because it provides for the ability to learn and
regarded as a more general case of face localization. In face later recognize new faces in an unsupervised manner.
localization, the task is to find the locations and sizes of a known
number of faces (usually one). In face detection, one does not KNN classifier is best suited for classifying persons based on
have this additional information. their images due to its lesser execution time and better accuracy
than other commonly used methods which include Hidden
Markov Model and kernel method. Although methods like SVM
Manuscript received Jul 12, 2008. and Adaboost Algorithm are proved to be more accurate than
V.Vaidehi and S.Vasuhi, R.Kayalvizhi are faculty and
K.Mariammal, Raghuraman.M.B, Sundara Raman.V, Meenakshi.L, KNN classifier, KNN classifier has a faster execution time and is
Anupriyadharshini.V, Thangamani.T are students of Department of dominant than SVM in sparse datasets [8].
Electronics Engineering, Madras Institute of Technology, Anna
University, Chennai, Tamil Nadu, India
(email id : vaidehivijay@gmail.com)

ISBN: 978-988-98671-0-2 WCECS 2008

Proceedings of the World Congress on Engineering and Computer Science 2008
WCECS 2008, October 22 - 24, 2008, San Francisco, USA

II. FACE DETECTION

A face detection algorithm for colour images in the presence
of varying lighting conditions as well as complex backgrounds is 50

proposed. In this work, a skin color based technique is used for

100
detecting frontal human faces in images where they appear. The
method consists of two image processing steps. First, skin
150
regions are separated from non-skin regions. Then locate the
frontal human face within the skin regions. In the first step, a 200
chroma chart is obtained that shows likelihoods of skin colors.
This chroma chart is used to generate a gray scale image from the 50 100 150 200 250 300
original color image [10]. This image has the property that the
gray value at a pixel shows the likelihood of that pixel Fig.1. The Boundary indicating the detected face
representing the skin. The gray scale image is segmented, to
separate skin regions from non skin regions [13]. The luminance
component is used, together with template matching to III. FACE RECOGNITION
determine if a given skin region represents a frontal human face The detected features are reduced in dimension using any
or not. dimensionality reduction technique and classified using a KNN
A. Algorithm classifier. The images of faces do not change radically when
projected into face space, while the projection of non-face
The input RGB image is converted into YCbCr color space.
images changes radically. The background can significantly
The Cb and Cr part are then extracted separately using the
affect the recognition performance, because Eigenface analysis
equations given below [9]:
does not distinguish the face from the rest of image. The
r = R/(R+G+B) (1) luminance and chrominance part is removed of the RGB image is
which is given as input and thus overcoming the above
b = B/(R+G+B) (2)
mentioned problem. The Eigenface[7] method is the most
From the Gaussian fitted skin color model of the input Fig. the popular linear technique for face recognition. Eigenface applies
likelihood of skin is computed using the equation . Principal Component Analysis (PCA) to project the data points
Likelihood=P(r, b) =exp [-0.5(x-m)T C-1(x-m)] (3) along the directions of maximal variances.
A. PCA Projection
Where x=(r, b) T
Principal Components Analysis is a method that reduces data
m =mean of x. dimensionality by performing a covariance analysis between
C=Covariance of x. subjects. As such, it is suitable for data sets in multiple
The grayscale image of the figure is then obtained, dimensions [1].
highlighting the skin regions. Using adaptive threshold, skin
regions are highlighted effectively. The face is then located from PCA projection involves the following steps.
the skin like segments by finding the number of holes in the skin 1. Arrange the data as a set of n column vectors x1, x2, xn, each
regions using the following equations of which has m rows. Place the column vectors into a single
matrix X of dimensions m x n.
E = Cc H (4) 2. Calculate the empirical mean along each dimension M=1m
and place the mean values into an empirical mean vector u of
E: is the Euler number
dimensions m x 1.
Cc: The number of connected components n
H: The number of holes in a region. U[m] = (1/n) X[ M, N] (6)
N=1
H=1E (5)
The template face is then resized according to the height and 3. Calculate the deviations (Dev) of the data matrix from the
width of the region computed and also oriented accordingly. mean vector u,
Thus by template matching a rectangle is drawn for the detected Dev = X u . h (7)
face boundary which is shown in Fig.1.
where h[N] = 1,for N = 1n .
The PFC is obtained by taking Fourier transform of the N: Feature vector
detected face region. This detected face image is used for n: Number of subjects
recognition.

ISBN: 978-988-98671-0-2 WCECS 2008

Proceedings of the World Congress on Engineering and Computer Science 2008
WCECS 2008, October 22 - 24, 2008, San Francisco, USA

4. Calculate the m x m covariance matrix C from the outer able to learn which regions are good for recognition and which
product of matrix B with itself, regions are not. Second, to recognize a set image we must
measure the distance to linear subspace for each person. While
C = E [ Dev Dev] = E [ Dev . Dev* ] = (1/n) Dev . Dev*(8)
this in an improvement over a correlation scheme that needs a
5. Compute the matrix V of eigenvectors of the covariance large number of images for each class, it is still too
matrix C, computationally expensive. Finally from storage point of view
V-1CV = D, (9) linear subspace algorithm must keep five images in memory for
Where D is the diagonal matrix of eigen values of C. every person.
B. KNN CLASSIFIER
Matrix D will take the form of an m x m diagonal matrix, where
The simplest classification scheme is a nearest neighbor
D[p,q] =m for p = q = m (10) classification in the image space. Under this scheme an image in
m is the mth eigen value of the covariance matrix C, and matrix the test set is recognized by assigning to it the label of the closest
V, also of dimension m x m, contains m column vectors, each of point in the learning set, where distance are measured in image
length m, which represent the m eigenvectors of the covariance space. If all images have been normalized to be zero mean and
matrix C. The eigenvalues and eigenvectors are ordered and have unit variance, then this procedure is equivalent to choosing
paired. The mth eigenvalue corresponds to the mth eigenvector. the image in learning set that best correlates with the test image.
Because of normalization process, the result is independent of
If the data is concentrated in a linear subspace, this provides a light source intensity and the effects of a video cameras
way to compress data without losing much information and automatic gain control. Feature selection is achieved using this
simplifying the representation. By picking the eigenvectors [1] learning algorithm by constraining each classifier to depend on
having the largest eigenvalues, there is a chance of losing some only a single feature [14].
information. A fixed number of eigenvectors and their respective
eigenvalues can be chosen and a representation, or abstraction of The Euclidean distance metric [4] is often chosen to determine
the data is obtained. In this way the face images xi are projected the closeness between the data points in KNN. A distance is
into the subspace by throwing away the components assigned between all pixels in a dataset. Distance is defined as
corresponding to zero eigenvalue. By PCA projection, the the Euclidean distance between two pixels. The Euclidean metric
extracted features are statistically uncorrelated and the rank of is the function d: Rn X Rn R that assigns to any two vectors in
the new data matrix is equal to the number of features Euclidean n-space X=(x1,, xn) and Y=( y1,, yn) the
(dimensions). number,

The Eigen face method is also based on linear projection of d(x,y) = (( x1 - y1)2 + .. + (xn - yn) 2) (11)
the image space to a low dimensional feature space. However the This gives the "standard" distance between any two vectors in Rn.
eigen face method which uses principal component From these distances, a distance matrix is constructed between
analysis(PCA) for dimensionality reduction yields projection all possible pairings of points (x, y).
directions that maximizes the total scatter across all classes i.e.
all images of all faces. In choosing the projection which C. KNN ALGORITHM
maximizes total scatter, PCA retains some unwanted variations 1) Each data pixel value within the data set has a class label in the
due to lighting and facial expression. The variations between set, Class = {c1,...,cn}.
images of the same face due to illumination and viewing 2) The data points', k-closest neighbors (k being the number of
direction are almost always larger than image variations due to neighbors) are then found by analyzing the distance matrix.
change in face identity. Thus PCA projections are optimal for 3) The k-closest data points are then analyzed to determine
reconstructing from a low dimensional basis; they may not be which class label is the most common among the set.
optimal from discrimination point of view. 4) The most common class label is then assigned to the data point
being analyzed.
PCA is used for dimensionality reduction in a data set by
retaining those characteristics of the data set that contribute most D. KNN PERFORMANCE VS CHOICE OF K
to its variance, by keeping lower-order principal components and 1) When noise is present in the locality of the query instance, the
ignoring higher-order ones. Such low-order components often noisy instance(s) win the majority vote, resulting in the incorrect
contain the "most important" aspects of the data, which are class being predicted. A larger k could solve this problem.
projected as Eigenfaces. 2) When the region defining the class, or fragment of the class, is
If there is no noise or self shadowing, the linear subspace so small that instances belonging to the class that surrounds the
would achieve error free classification under any lighting fragment win the majority vote. A smaller k could solve this
conditions. Nevertheless, there are several reasons to look else problem. The KNN shows superior performance for smaller
where, first due to self shadowing, facial expressions, some values of k compared to larger values of k.
regions of the face have variability that does not agree with linear Instances can be considered as points within an n-dimensional
subspace model. Given enough images of faces, we should be instance space where each of the n-dimensions corresponds to

ISBN: 978-988-98671-0-2 WCECS 2008

Proceedings of the World Congress on Engineering and Computer Science 2008
WCECS 2008, October 22 - 24, 2008, San Francisco, USA

one of the n-features that are used to describe an instance. The

absolute position of the instances within this space is not as Table II. Confidence value for 10 random persons
significant as the relative distance between instances. This
relative distance is determined by using a distance metric. Ideally,
the distance metric must minimize the distance between two Test Image Confidence value
similarly classified instances, while maximizing the distance
between instances of different classes. Subject 1 88.7182
Subject 2 84.8806
KNN predictions are based on the intuitive assumption that Subject 3 88.7468
objects close in distance are potentially similar, it makes good Subject 4 20.5647
sense to discriminate between the k nearest neighbors when Subject 5 89.3108
making predictions, i.e., let the closest points among the k Subject 6 88.6411
nearest neighbors have more say in affecting the outcome of the Subject 7 89.7412
query point. This procedure has several well known Subject 8 88.6166
disadvantages. First, if the image in the learning set and test set Subject 9 89.5824
are gathered under varying lighting conditions, then the Subject 10 24.7433
corresponding points in the image space will not be tightly
.. ..
clustered. So in order for this method to work reliably under
.
variations in lighting, a learning set which densely sampled the
continuum of possible lighting conditions, is required. Second,
Table III. Accuracy values of KNN and CB classifier
correlation is computationally expensive. For recognition, we
must correlate the image of the test face with each image in the
learning set to reduce computational time. Third, it requires large Type KNN CB
amounts of storage: i.e, the learning set must contain numerous Acc Test Train Both Test Train Both
images of each person. k=1 0.3428 0.3444 0.3428 0.2553 0.2945 0.2866

IV. IMPLEMENTATION From the Table III, it is observed that KNN classification
The work is implemented in Matlab. Yale Database is used for algorithm outperforms CB classification algorithm in
testing and training purposes. The database contains 165 classification accuracy [7].
grayscale images in GIF format of 15 individuals. There are 100
images per subject in the database.
Accuracy (Normalized)

0.7
The time taken for execution of each module in Pentium IV 0.6
processor is given in Table I. 0.5
0.4 KNN
Table I. Execution time for Face recognition 0.3 CB
0.2
0.1
Process Time Duration (in Seconds) 0
PFC extraction 0.1935 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
KNN classification 0.2325 Ratio Of Training Data

The confidence values for the recognition of a person is Fig 2. Comparison of KNN with Concept Vector based classifier
calculated using the Euclidean distance between the PCA
projected values of the test image and PCA projected values of
the train database. This value determines whether recognition of V. EXPERIMENTAL RESULTS
a face image using this method is dependable or not. When the The experimental results for face authentication system are
confidence value is low recognition is not dependable. The explained below. The input RGB image is converted into YCbCr
confidence value obtained for different test images are tabulated color space. Here we extract Cb and Cr part separately, as shown
in Table II. in Fig. 3 and 4.
The confidence values were obtained for ten test subjects, of
which eight subjects were recognized, as those images were in
the train database with a confidence value above the threshold
value and the other two subjects were not recognized
successfully as their confidence values fall below the threshold.

ISBN: 978-988-98671-0-2 WCECS 2008

Proceedings of the World Congress on Engineering and Computer Science 2008
WCECS 2008, October 22 - 24, 2008, San Francisco, USA

Fig 7. Template face (model) used to verify the existence of

Fig 3. Cb part of input image faces in skin regions.

Thus by template matching a rectangle is drawn for the detected

face boundary as shown in the Fig. 8.

50

100

150

Fig 4. Cr part of input image

200
From the Gaussian fitted skin color model of the input image
the likelihood of skin regions are obtained, highlighting the skin 250

regions as shown in Fig. 5.

50 100 150 200 250 300 350

Fig 8. Face boundary for the given image

The retrieval of human faces requires detection of human
faces in the subjects image. Thus this method segments the skin
regions out and locates faces using template matching in order to
detect frontal human faces.
The detected face image is given as input of dimension
100X100 to the Eigen face recognition system which obtains the
PCA projected values of reduced dimension 100X 1, which is
used to recognize the face image by preserving the Euclidean
Fig 5. Likelihood of skin regions in the given image
structure of a person. The KNN classifier is used to classify the
Using adaptive threshold skin regions are highlighted test image based on their Euclidean structure obtained using
effectively, as shown in Fig. 6. PCA. The test sample to the classifier and the classified output is
shown in Fig. 9 and 10.

Fig 6. Image after thresholding

The template face as shown in the Fig. 7 is obtained by

template matching. Fig 9. Test sample to the classifier

ISBN: 978-988-98671-0-2 WCECS 2008

Proceedings of the World Congress on Engineering and Computer Science 2008
WCECS 2008, October 22 - 24, 2008, San Francisco, USA

[12] Ming-Hsuan Yang, David J. Kriegman, and Narendra Ahuja, Detecting

faces in Images: A survey, IEEE Transactions on Pattern Analysis and
Machine Intelligence, pp. 34-58, January 2002.
[13] Vinod Pathangay and B. Yegnanarayana, "Use of vertical face profiles for
text dependent audio-visual biometric person authentication," in Indian
Conference on Computer vision, Graphics and Image processing,
Ahmedabad, India, pp. 6166, December 16-18 2002.

Fig 10. Classifier output [14] Alex Pentland, Baback Moghaddam, and Thad Starner (1994),
View-Based and Modular Eigenspaces for Face Recognition, IEEE Conf.
VI. CONCLUSION on Computer Vision and Pattern Recognition, MIT Media Laboratory
Tech. Report No. 245.
The proposed DR algorithm localizes the face from the given
input image using the skin color detection method where the
skin regions are segmented and face is located using template
matching. The detected face image is projected using Eigen face
analysis and classified using the K nearest neighborhood (KNN)
classifier. This algorithm is efficient as it can be integrated with
the output from multi-modal sensors and thus can be used as part
of multi-sensor data fusion.

ACKNOWLEDGMENT
Authors thank Tata Consultancy Services for funding this
project.

REFERENCES

[1] M.Belkin and P.Niyogi, Laplacian Eigenmaps and Spectral Techniques

for Embedding and Clustering, Proc. Conf. Advances in Neural
Information Processing System 15, 2001.
[2] Deng Cai, Xiaofei He, Jiawei Han, Senior Member, IEEE, and Hong-Jiang
Zhang, Fellow, IEEE, Orthogonal Laplacianfaces for Face Recognition,
IEEE Trans. On Image Processing, Vol. 15, No. 11, Nov. 2006.
[3] Rafael C.Gonzalez , Richard E.Woods, Digital Image Processing,
Prentice Hall Of India, 2005.
[4] M. A. Turk and A. P. Pentland, Face recognition using eigenfaces, IEEE
Computer Society Conference on Computer Vision and Pattern
Recognition, Proceedings CVPR 91.
[5] John Wright, Allen Yang, Arvind Ganesh, Shankar Sastry, and Yi Ma.
Robust face recognition via sparse representation, To appear in IEEE
Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2008.
[6] Turk.M, and Pentland.A, Eigenvalues for recognition, Journal of
Cognitive Neuroscience, vol. 13, no. 1, pp.71-86, 1991.
[7] P. Sinha, B. Balas, Y. Ostrovsky, and R. Russell, Face recognition by
humans: Nineteen results all computer vision researchers should know
about, Proceedings of the IEEE, vol. 94, no. 11, pp. 19481962, 2006.
[8] X. He, S. Yan, Y. Hu, P. Niyogi, and H. Zhang, Face recognition using
Laplacianfaces, IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 27, no. 3, pp. 328340, 2005.

[9] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, Face detection in

color images, IEEE Transactions on Pattern Analysis and Machine
Intelligence, pp. 696706, May 2002.
[10] J. Cai & A. Goshtasby & C. Yu, "Detecting Human Faces in Color Images",
Wright State University, University of Illinois, 1998.
[11] C. Garcia and G. Tziritas, Face detection using quantized skin color
region merging and wavelet packet analysis, IEEE Transactions on
Multimedia Vol.1, No. 3, pp. 264277, September 1999.

ISBN: 978-988-98671-0-2 WCECS 2008