[Paper Review] von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verification

Md. Abul Hasnat, Julien Bohné+3 more|arXiv (Cornell University)|Jun 13, 2017

Face recognition and analysis68 references61 citations

TL;DR

The paper introduces a vMF mixture model-based loss (vMFML) for learning directional CNN features, achieving state-of-the-art results on several face verification benchmarks.

ABSTRACT

A number of pattern recognition tasks, extit{e.g.}, face verification, can be boiled down to classification or clustering of unit length directional feature vectors whose distance can be simply computed by their angle. In this paper, we propose the von Mises-Fisher (vMF) mixture model as the theoretical foundation for an effective deep-learning of such directional features and derive a novel vMF Mixture Loss and its corresponding vMF deep features. The proposed vMF feature learning achieves the characteristics of discriminative learning, extit{i.e.}, compacting the instances of the same class while increasing the distance of instances from different classes. Moreover, it subsumes a number of popular loss functions as well as an effective method in deep learning, namely normalization. We conduct extensive experiments on face verification using 4 different challenging face datasets, extit{i.e.}, LFW, YouTube faces, CACD and IJB-A. Results show the effectiveness and excellent generalization ability of the proposed approach as it achieves state-of-the-art results on the LFW, YouTube faces and CACD datasets and competitive results on the IJB-A dataset.

Motivation & Objective

• Provide a theoretical foundation for learning directional deep features using a von Mises-Fisher mixture model.
• Develop the vMF Mixture Loss (vMFML) and integrate it with CNNs for discriminative feature learning.
• Demonstrate strong generalization across diverse face verification datasets (LFW, YTF, CACD, IJB-A).
• Show relationships between vMFML and existing loss functions and normalization methods.

Proposed method

• Model facial features as a mixture of von Mises-Fisher distributions (vMFMM) with equal class privilege.
• Derive the vMF Mixture Loss (vMFML) by replacing the softmax with a vMF-based posterior and cross-entropy objective.
• Represent class centers as mean directions mu with concentration kappa, and normalize features to unit length.
• Provide gradients and backpropagation rules for end-to-end CNN training with vMFML.
• Use a ResNet-based CNN (Res-27) to extract 512-d directional features and apply unit normalization.
• Discuss the theoretical connections between vMFML and popular losses (softmax variants, center loss, angular/large margins) and normalization schemes.

Experimental results

Research questions

• RQ1 Does the vMF Mixture Loss (vMFML) enable discriminative learning by concentrating features around class mean directions while separating classes on the hypersphere?
• RQ2 How does vMFML relate to and encompass other loss functions used in face recognition (e.g., softmax variants, center loss, angular margin losses)?
• RQ3 What is the performance of vMFML-trained CNNs on standard face verification benchmarks (LFW, YouTube Faces, CACD, IJB-A)?
• RQ4 Does the proposed approach generalize across datasets with pose, lighting, and age variations?

Key findings

• vMFML enables discriminative feature learning by concentrating same-class features around mu with a controllable kappa, while separating classes on the unit sphere.
• The method achieves state-of-the-art results on LFW, YouTube Faces, and CACD datasets.
• On IJB-A, the approach yields competitive performance ( TAR@FAR=0.001 = 85% reported in abstract).
• Only a single CNN model (trained with vMFML) is required, avoiding multiple loss terms and extra metric-learning steps.
• The approach integrates normalization through unit-norm features and mu, yielding efficient training and interpretable parameters (mu and kappa).
• The work shows that vMFML subsumes or relates to several existing FR losses and normalization strategies, providing a theoretical lens for understanding them.