[Paper Review] Targeting Ultimate Accuracy: Face Recognition via Deep Embedding

Jingtuo Liu, Yafeng Deng+3 more|arXiv (Cornell University)|Jun 24, 2015

Face recognition and analysis10 references240 citations

TL;DR

This paper proposes a two-stage deep learning framework for face recognition that combines multi-patch convolutional neural networks (CNNs) with deep metric learning to learn highly discriminative low-dimensional embeddings. The method achieves 99.77% accuracy on the LFW benchmark under the standard protocol, significantly outperforming prior state-of-the-art methods and demonstrating a clear path toward practical, high-accuracy face recognition systems.

ABSTRACT

Face Recognition has been studied for many decades. As opposed to traditional hand-crafted features such as LBP and HOG, much more sophisticated features can be learned automatically by deep learning methods in a data-driven way. In this paper, we propose a two-stage approach that combines a multi-patch deep CNN and deep metric learning, which extracts low dimensional but very discriminative features for face verification and recognition. Experiments show that this method outperforms other state-of-the-art methods on LFW dataset, achieving 99.77% pair-wise verification accuracy and significantly better accuracy under other two more practical protocols. This paper also discusses the importance of data size and the number of patches, showing a clear path to practical high-performance face recognition systems in real world.

Motivation & Objective

• To improve face recognition accuracy beyond existing state-of-the-art methods using deep learning.
• To develop a practical, scalable system for real-world face recognition by analyzing data size and patch configuration effects.
• To explore the role of deep metric learning in learning compact, discriminative face embeddings.
• To demonstrate that high accuracy in face verification is achievable through structured deep embedding and multi-patch feature extraction.

Proposed method

• The method employs a two-stage pipeline: first, a multi-patch deep CNN extracts local features from facial regions.
• Second, deep metric learning is applied to embed these features into a low-dimensional space where inter-class distances are maximized and intra-class distances are minimized.
• The network is trained using triplet loss to optimize the embedding space for discriminative power.
• Multiple patches from a single face image are processed independently and then aggregated to form a global embedding.
• The framework is trained end-to-end with supervised data, leveraging large-scale face datasets to improve generalization.
• The approach is evaluated under multiple protocols, including the standard LFW protocol and more practical, challenging settings.

Experimental results

Research questions

• RQ1 Can a deep learning-based approach with multi-patch feature extraction and metric learning achieve state-of-the-art face recognition accuracy?
• RQ2 How does the number of patches and training data size affect the performance of deep face embedding models?
• RQ3 Does the proposed two-stage method outperform existing methods on both standard and practical evaluation protocols?
• RQ4 Can deep metric learning produce highly discriminative embeddings that generalize well to real-world face recognition tasks?

Key findings

• The proposed method achieves 99.77% pair-wise verification accuracy on the LFW dataset under the standard protocol, surpassing all prior state-of-the-art methods.
• The model demonstrates significantly improved performance under two more practical evaluation protocols, indicating strong generalization to real-world conditions.
• Experiments show that increasing the number of patches and training data size leads to consistent performance gains, validating the scalability of the approach.
• The deep metric learning component effectively learns a compact, discriminative embedding space that enhances recognition accuracy.