Face Recognition Using Eigen Faces

Summary written by Vishwanath Sindagi

Summary: The paper addresses the problem of different the eigen faces and then choosing the face label that mini-
face processing tasks such as detection , recognition and mizes the distance to the projection of this new face.
tracking, with a specific emphasis on face recognition. This Datasets, Experiments and Results: The authors collected
work is inspired by Sirovich and Kirby [5], where a set of a new dataset consisting of 2500 images with 16 subjects
faces is efficiently represented using principal component covering different imaging conditions such as illumination,
analysis. pose, view etc. Average classification accuracy is used to
Related work: The authors categorize prior work primar- report the performance. Two sets of experiments are con-
ily into 2 categories: (i) Approaches [1, 4, 7] that focus on ducted: (i) Effects of varying lighting, size and head orien-
detecting individual features such as eyes, nose, mouth etc. tation is analzyed. The paper reports average classification
and by defining a face model that encodes the positional accuracy of 96% for lighting variation, 85% over orienta-
relationship between these features. Such methods are not tion variation and 64% over size variation. (ii) Effect of
easily extensible to multiple views and, it has been demon- varying the acceptance threshold is varied.
strated that individual features and their relationships do not Strengths: The paper provides new insights about the
provide sufficient representation to account for the perfor- lower dimensional manifold along which the face images
mance of human-level recognition. (ii) Connectionist ap- lie. They connect this insight nicely to the process of encod-
proaches [2, 3, 6] that capture configurational nature of the ing/decoding face images, which enabled them to develop
task. These methods train neural networks via backprop- a method that is based on dimensionality reduction. Fur-
agation. However, it was unclear how these methods will ther, the method uses raw intensity pixels for recognition
scale to larger problems. with minimal processing. Since the method is entirely data
Approach: A typical face image of size 256× 256 de- driven, no prior knowledge of geometry is required. Also,
scribes a 65,536 dimensional vector, or equivalently, a point the paper addresses several implementation issues like pro-
in 65,536 dimensional space. Therefore, a set of images, viding neat tricks on making the computation of eigen vec-
map to a set of points in this huge 65,536 dimensional tors faster. The method is easy to implement and the train-
space. Face images, typically being similar in overall con- ing is fully automated. Overall, the authors demonstrated
figuration and having high correlation among neighbouring strong performance in terms of accuracy and computational
pixels, will not be randomly distributed in this huge space. efficiency.
Rather, these set of images will lie on a lower-dimensional Weaknesses: First, since the method is based on PCA, it
sub-space or manifold. Based on this arguments and some suffers from general drawbacks of PCA like assumption
insights from the information theory approach of coding of linearity. Also, PCA does not consider label informa-
and decoding faces, the authors aim to map images to lower tion while training, limiting its discriminatory power. The
dimensional space by extracting most relevant information method is efficient only when the number of face classes is
in a face image. This is achieved by capturing the variation larger than the dimensions of the face space. Also, as the
in a collection of face images. Mathematically, this amounts number of face classes increase, there could be increased
to finding the principal components of the distribution of overlap when representing the face images with the same
faces, or the eigenvectors of the covariance matrix of the set face space, thus lowering the recognition rate. From the ex-
of face images. These eigen vectors characterize the varia- periments, it can be observed that the dataset size is small as
tion between face images. Each face image in the training compared to the current trends. Additionally, the method is
set can be represented exactly in terms of a linear combi- very sensitive to little variations in scale and large variations
nation of eigenfaces. Since, only a few eigenvectors/faces in pose and background variation.
capture most of the variation in the dataset, the proposed Reflections: While the authors take a different approach
method can be implemented efficiently. as compared to existing feature/configurational approaches,
Specifically, the proposed approach involves transforma- a more hybrid approach which can leverage advantages of
tion of face images into a set of characteristic feature images both methods would be more beneficial. Also, based on
(called as eigen faces), which are the principal components the discussion in weaknesses section, obvious approaches
of the initial training set of face images. New faces are rec- to improve the current algorithm would be to incorporate
ognized by projecting them into the sub-space spanned by non-linearity by using kernel methods or by incorporating

1
label information during training by using linear discrimi-
nant analysis framework.

References
[1] W. Bledsoe. The model method in facial recognition,
panoramic research inc., palo alto. Technical report, CA,
Technical Report, Technical Report PRI: 15, 1964.
[2] M. K. Fleming and G. W. Cottrell. Categorization of faces
using unsupervised feature extraction. In Neural Networks,
1990., 1990 IJCNN International Joint Conference on, pages
65–70. IEEE, 1990.
[3] G. E. Hinton and J. A. Anderson. Storage and processing
of information in distributed associative memory systems. In
Parallel models of associative memory, pages 139–180. Psy-
chology Press, 2014.
[4] T. Kanade. Picture processing system by computer complex
and recognition of human faces. 1974.
[5] L. Sirovich and M. Kirby. Low-dimensional procedure for the
characterization of human faces. Josa a, 4(3):519–524, 1987.
[6] T. Stonham. Practical face recognition and verification with
wisard. In Aspects of face processing, pages 426–441.
Springer, 1986.
[7] A. L. Yuille, P. W. Hallinan, and D. S. Cohen. Feature ex-
traction from faces using deformable templates. International
journal of computer vision, 8(2):99–111, 1992.