[Paper Review] A Survey of Face Recognition

Xinyi Wang, Jianteng Peng+4 more|arXiv (Cornell University)|Dec 26, 2022

Face recognition and analysis4 citations

TL;DR

This survey provides a comprehensive overview of face recognition (FR) technologies, covering its evolution from traditional methods like Eigenface and LBP to modern deep learning-based approaches. It details FR pipelines, loss functions, backbone architectures, and evaluates the impact of model size and data distribution through controlled experiments, offering practical insights for industrial deployment and benchmarking in real-world scenarios.

ABSTRACT

Recent years witnessed the breakthrough of face recognition with deep convolutional neural networks. Dozens of papers in the field of FR are published every year. Some of them were applied in the industrial community and played an important role in human life such as device unlock, mobile payment, and so on. This paper provides an introduction to face recognition, including its history, pipeline, algorithms based on conventional manually designed features or deep learning, mainstream training, evaluation datasets, and related applications. We have analyzed and compared state-of-the-art works as many as possible, and also carefully designed a set of experiments to find the effect of backbone size and data distribution. This survey is a material of the tutorial named The Practical Face Recognition Technology in the Industrial World in the FG2023.

Motivation & Objective

• To provide a systematic review of face recognition (FR) technologies, from classical methods to state-of-the-art deep learning approaches.
• To analyze and compare over 100 FR algorithms across feature extraction, loss functions, and recognition pipelines.
• To investigate the impact of backbone architecture size and training data distribution on FR performance through controlled experiments.
• To serve as a foundational resource for the FG2023 tutorial 'The Practical Face Recognition Technology in the Industrial World'.
• To compile and evaluate major training and test datasets, benchmarks, and real-world applications of FR.

Proposed method

• Surveying and categorizing over 100 face recognition algorithms, including both conventional manually designed features (e.g., Eigenface, LBP, SVM-based) and deep learning-based methods.
• Detailing the standard deep learning pipeline for FR, including face detection, alignment, embedding extraction, and matching, with emphasis on training and inference stages.
• Analyzing key components such as loss functions (e.g., triplet loss, softmax loss), embedding techniques, and optimization strategies for large-scale ID recognition.
• Designing and conducting controlled experiments to evaluate the effects of backbone depth (e.g., ResNet variants) and data distribution (e.g., class imbalance, data quality) on recognition accuracy.
• Benchmarking models on standard datasets including MS-Celeb-1M, MegaFace, and FRVT, with a focus on 1:1 verification, 1:N identification, and morph attack detection.
• Evaluating face quality assessment metrics, including scalar and vector-based quality scoring, to understand their impact on recognition performance.

Experimental results

Research questions

• RQ1 How do different deep learning-based face recognition architectures perform under varying backbone sizes and data distributions?
• RQ2 What is the relative effectiveness of conventional feature-based methods (e.g., LBP, Fisherface) compared to modern deep learning approaches in real-world settings?
• RQ3 How do training data quality and class distribution affect the generalization and robustness of face recognition models?
• RQ4 What are the key performance differences in FR systems when evaluated on standardized benchmarks like FRVT, MegaFace, and MS-Celeb-1M?
• RQ5 How do face quality assessment metrics influence recognition performance, especially in low-quality or noisy image conditions?

Key findings

• Deep learning-based face recognition models significantly outperform classical methods like Eigenface and LBP in accuracy and robustness, especially under unconstrained conditions.
• Larger backbone architectures (e.g., deeper ResNets) consistently improve recognition performance, but with diminishing returns beyond a certain depth, particularly on large-scale datasets.
• Data distribution has a substantial impact on model generalization: models trained on class-balanced, high-quality data show better performance on unseen data and under distribution shift.
• Face quality assessment metrics—especially quality vectors scoring attributes like lighting, pose, and sharpness—help identify low-quality images that degrade recognition performance.
• FRVT benchmarks, particularly FRVT 1:N and FRVT MORPH, reveal that state-of-the-art models achieve FNMR < 0.01 under standard conditions, but performance drops significantly under morphed face attacks.
• The MS-Celeb-1M challenge demonstrated that models trained on large-scale, noisy datasets can still achieve high recognition recall, highlighting the robustness of modern deep metric learning.