[Paper Review] Enhancing Convolutional Neural Networks for Face Recognition with Occlusion Maps and Batch Triplet Loss

Daniel Sáez Trigueros, Li Meng+1 more|University of Hertfordshire Research Archive (University of Hertfordshire)|Jul 25, 2017

Face recognition and analysis45 references75 citations

TL;DR

The paper proposes (1) occlusion map guided data augmentation to train CNNs that perform robustly under partial occlusions, and (2) a batch triplet loss that minimizes both mean separation and score variance to improve verification, showing gains on LFW and occluded face datasets.

ABSTRACT

Despite the recent success of convolutional neural networks for computer vision applications, unconstrained face recognition remains a challenge. In this work, we make two contributions to the field. Firstly, we consider the problem of face recognition with partial occlusions and show how current approaches might suffer significant performance degradation when dealing with this kind of face images. We propose a simple method to find out which parts of the human face are more important to achieve a high recognition rate, and use that information during training to force a convolutional neural network to learn discriminative features from all the face regions more equally, including those that typical approaches tend to pay less attention to. We test the accuracy of the proposed method when dealing with real-life occlusions using the AR face database. Secondly, we propose a novel loss function called batch triplet loss that improves the performance of the triplet loss by adding an extra term to the loss function to cause minimisation of the standard deviation of both positive and negative scores. We show consistent improvement in the Labeled Faces in the Wild (LFW) benchmark by applying both proposed adjustments to the convolutional neural network training.

Motivation & Objective

• Motivate robust face recognition under real-life occlusions and identify which face regions drive CNN decisions.
• Introduce occlusion-based data augmentation guided by occlusion sensitivity maps to balance feature learning across face regions.
• Propose batch triplet loss to improve generalization to unseen identities by controlling score distributions.

Proposed method

• Use a CNN architecture with 10 conv layers and one fully connected layer, trained as a classifier to obtain bottleneck features for verification.
• Construct binary occlusion maps by occluding image regions and averaging to identify high vs. low impact face regions.
• Augment training with occluders placed according to a probability distribution derived from occlusion maps to emphasize high-impact regions.
• Fine-tune bottleneck features with a novel batch triplet loss that combines mean separation (triplet loss term) and a variance term for positive/negative scores (Equation 9).
• Adopt online triplet sampling to select informative triplets (anchor, positive, negative) during training.

Experimental results

Research questions

• RQ1 Can occlusion-aware data augmentation improve CNN robustness to partial facial occlusions in unconstrained settings?
• RQ2 Does a batch triplet loss that minimizes both the margin between positive and negative distributions and their variances improve verification performance on standard benchmarks?
• RQ3 How do occlusion-aware training schemes compare to random occlusion augmentation in terms of generalization to real-life occlusions?

Key findings

• Occlusion-aware training with strategically placed occluders improves mean accuracy and reduces variability across occlusion locations compared to baseline and random occlusion schemes.
• Occlusion maps reveal central face regions as high-effect areas, and targeted augmentation helps the model learn discriminative features from outer face regions as well.
• On AR face database, models trained with occluded data outperform the baseline, especially at low false acceptance rates, with a medium occluder often offering best overall performance.
• On LFW, models fine-tuned with the batch triplet loss (model C) consistently outperform those trained with standard triplet loss (model B).
• Occlusion-aware fine-tuning combined with batch triplet loss yields improvements over baseline in both occluded and non-occluded evaluation scenarios.