Abstract
Methods for face verification works reasonably well on face images with standardized (frontal) face positions and good spatial resolution. However such methods have significant challenges on poor resolution images, poor lighting conditions and not standard (frontal) face positions. In this paper, we survey the capability of existing face restoration and verification methods, with the aim of understanding how useful face restoration methods are for face verification. We propose a qualitative and quantitative comparison benchmark, and apply it on eight methods for face restoration and six methods for face verification, on several real-world low-quality images from a surveillance context, and outline observed advantages and limitations. Experiments shows that each restoration method can affect each face verification method differently, with fewer than the half of face restoration methods helping face verification. Interestingly, some face restoration methods with less good qualitative evaluation helped face verification the most. Experiments also show that face verification works less good if the resolution decreases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans PAMI:2037–2041
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans PAMI:1–10
Campisi P, Egiazarian K (2001) Blind image deconvolution: theory and applications. CRC Press
Carey S, Diamond R (1977) From piecemeal to configurational representation of faces. PubMed - NCBI
Champandard A J (2016) Neural Enhance: super resolution for images using deep learning. https://github.com/alexjc/neural-enhance
Chen Y, Tai Y, Liu X, Shen C, Yang J (2018) FSRNet: End-to-end learning face super-resolution with facial priors. In: Proceedings of IEEE CVPR, pp 2492–2501
Dong C, Loy C C, He K, Tang X (2016feb) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38 (2):295–307
Horé A, Ziou D (2010) Image quality metrics: Psnr vs. ssim. 20th Int Conf Pattern Recogn:2366–2369
Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE CVPR, pp 5197–5206
Jia Z, Zhao J, Wang H, Xiong Z, Finn A (2015) A two-step face hallucination approach for video surveillance applications. Multimed Tools Appl 74(6):1845–1862
Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of ECCV
Keys R (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160
Kim J, Lee J K, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE CVPR
Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: Proceedings of IEEE CVPR
Learned-Miller E, Huang G B, Chowdhury AHL, Hua G (2016) Labeled faces in the wild: A survey. In: Kawulok M, Celebi M E, Smolka B (eds) Advances in Face Detection and Facial Image Analysis. Springer, pp 189–248
Ledig C, Theis L, Huszar F, Caballero J, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE CVPR
Liao S, Lei Z, Yi D, Li SZ (2014) A benchmark study of large-scale unconstrained face recognition. IEEE Int Joint Conf Biomet:1–8
Liu W, Wen Y, Yu Z, Li M, Raj B, Song L (2017) Sphereface: Deep hypersphere embedding for face recognition. CVPR
Liu Z, Luo P, Wang X, Tang X (December 2015) Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV)
Matsushita Y, Kawasaki H, Ono S, Ikeuchi K (2014) Simultaneous deblur and super-resolution technique for video sequence captured by hand-held video camera. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp 4562–4566
Ning X, Li W, Tang B, He H (2018) Buldp: Biomimetic uncorrelated locality discriminant projection for feature extraction in face recognition. IEEE Trans Image Process 27:2575–2586
Ning X, Nan X, Yu L, Zhang L (2020) Multi-view frontal face image generation: A survey. Concurrency Computat Pract Exper - Wiley Online Library, pp 1–18
Praveenbalaji D, Srinivas R, Roopa S, Suresh M, Gayathri A (2020) Id photo verification by face recognition. 6th International Conference on Advanced Computing & Communication Systems (ICACCS), pp 1449–1453
Redmon J, Farhadi A (2018) YOLOv3: An incremental improvement. arXiv:1804.02767v1[cs.CV]
Sajjadi M S M, Vemulapalli R, Brown M (2018) Frame-recurrent video super-resolution. CoRR arXiv:1801.04590
Schroff F, Kalenchenko D, Philbin J (2015) Facenet a unified embedding for face recognition and clustering. CVPR:815–823
Sensalire M, Ogao P, Telea A (2008) Classifying desirable features of software visualization tools for corrective maintenance. In: Proceedings of ACM SoftVis
Shi W, Caballero J, Huszar F, Totz J, Aitken A P, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of IEEE CVPR
Sobiecki A, Neves LAP, Giraldi GA, Gatts GJF, Thomaz CE (2011) Segmentacao e restauracao digital para eliminacao de artefatos em imagens frontais de face. VII Workshop de Visão Computacional, Curitiba
Sobiecki A, Neves LAP, Thomaz CE (2010) To a better digitalization and visualization of frontal face photographs. IV European Conference on Computational Mechanics, Paris
Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. CVPR:1891–1898
Tao X, Gao H, Liao R, Wang J, Jia J (2017) Detail-revealing deep video super-resolution. In: Proceedings of IEEE ICCV
Turk M, Pentland A (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7.2:179–188
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3.1:71–86
van Vliet S, Sobiecki A, Telea A (2018) Joint brightness and tone stabilization of capsule endoscopy videos. In: Proceedings of VISAPP
Wang F, Xiang X, Cheng J, Yuille AL (2017) Normface: L 2 hypersphere embedding for face verification. Proceedings of the 25th ACM International Conference on Multimedia, pp 1041–1049
Wang L, Dong-Chen H (2006) Texture classification using texture spectrum. IEEE Trans PAMI:905–910
Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. ECCV:499–515
Wu X, He R, Sun Z, Tan T (2018) A light cnn for deep face representation with noisy labels. IEEE Trans Inf Forensic Secur 13(11):2884–2896
Yang M-H, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. IEEE TPAMI 24(1):34–58
Yuzhikov V (2015) SmartDeblur deconvolution software. http://smartdeblur.net
Zhang C, Zhang Z Y (2010) A survey of recent advances in face detection
Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503
Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of IEEE CVPR
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sobiecki, A., van Dijk, J., Folkertsma, H. et al. Does face restoration improve face verification?. Multimed Tools Appl 80, 32863–32883 (2021). https://doi.org/10.1007/s11042-021-11167-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11167-6