[Paper Review] Learning Face Representation from Scratch

Yi Dong, Zhen Lei+2 more|arXiv (Cornell University)|Nov 28, 2014

Face recognition and analysis21 references1,651 citations

TL;DR

This paper introduces CASIA-WebFace, a large-scale, publicly available face dataset of ~500,000 images from 10,000 subjects collected via a semi-automatic web crawling and clustering pipeline. Using this dataset, the authors train an 11-layer CNN that achieves state-of-the-art performance on LFW (99.16% accuracy under BLUFR protocol) and YTF (92.24% accuracy), outperforming DeepFace and DeepID2 in supervised settings.

ABSTRACT

Pushing by big data and deep convolutional neural network (CNN), the performance of face recognition is becoming comparable to human. Using private large scale training datasets, several groups achieve very high performance on LFW, i.e., 97% to 99%. While there are many open source implementations of CNN, none of large scale face dataset is publicly available. The current situation in the field of face recognition is that data is more important than algorithm. To solve this problem, this paper proposes a semi-automatical way to collect face images from Internet and builds a large scale dataset containing about 10,000 subjects and 500,000 images, called CASIAWebFace. Based on the database, we use a 11-layer CNN to learn discriminative representation and obtain state-of-theart accuracy on LFW and YTF. The publication of CASIAWebFace will attract more research groups entering this field and accelerate the development of face recognition in the wild.

Motivation & Objective

• To address the lack of publicly available large-scale face datasets, which hinders reproducible research and fair comparison of face recognition algorithms.
• To develop a scalable, semi-automatic pipeline for collecting and annotating web-based face images using structured data from IMDb and clustering techniques.
• To establish a large-scale, non-overlapping training dataset that enables training of deep CNNs for face representation learning in the wild.
• To provide a public benchmark dataset that standardizes evaluation protocols on LFW and YTF, promoting fair and comparable research.
• To train a high-performance deep CNN baseline on CASIA-WebFace that outperforms existing methods on standard benchmarks.

Proposed method

• Crawled celebrity images from IMDb using names as queries, leveraging the site’s structured metadata to extract image URLs.
• Applied a face clustering algorithm to group faces in multi-person images, assigning identity labels based on name co-occurrence and proximity.
• Used edit distance on names to detect and remove potential overlaps with LFW, ensuring dataset independence.
• Conducted manual verification and correction of annotations to improve label accuracy.
• Trained an 11-layer deep convolutional neural network with ReLU, dropout, and a combined identification/verification loss function.
• Used the learned features for evaluation on LFW (standard and BLUFR protocols) and YTF, applying cosine similarity and post-processing with PCA or Joint Bayes.

Experimental results

Research questions

• RQ1 Can a semi-automatic pipeline effectively collect and annotate large-scale, diverse face images from the web while minimizing human effort?
• RQ2 Can a public, non-overlapping face dataset like CASIA-WebFace enable training of deep CNNs that generalize well to standard benchmarks such as LFW and YTF?
• RQ3 How does a deep CNN trained on CASIA-WebFace compare to state-of-the-art models like DeepFace and DeepID2 in terms of accuracy and robustness on unconstrained face recognition tasks?
• RQ4 To what extent does representation learning on CASIA-WebFace improve performance on low false alarm rate scenarios, such as those in surveillance applications?
• RQ5 Can a single deep network trained on CASIA-WebFace achieve performance comparable to ensemble models, especially when combined with simple post-processing techniques?

Key findings

• The proposed semi-automatic pipeline successfully collected 500,000 face images from 10,000 subjects with minimal manual effort, resulting in a high-quality, non-overlapping dataset.
• The 11-layer CNN trained on CASIA-WebFace achieved 99.16% accuracy on LFW under the BLUFR protocol, significantly outperforming HD-LBP-based methods (41.66% VR@FAR=0.1%).
• On the YTF dataset, the model achieved 92.24% accuracy in the supervised setting, surpassing DeepFace (91.4%) when using Joint Bayes post-processing.
• The performance gain from PCA and Joint Bayes post-processing on YTF was substantial, increasing accuracy from 88.00% (cosine only) to 92.24%.
• The results demonstrate that deep models significantly outperform wide models like HD-LBP, especially in low false alarm rate regimes critical for surveillance.
• The public release of CASIA-WebFace enables fair, reproducible benchmarking and accelerates research in unconstrained face recognition.