[Paper Review] SeesawFaceNets: sparse and robust face verification model for mobile platform

Jintao Zhang|arXiv (Cornell University)|Aug 24, 2019

Face recognition and analysis61 references4 citations

TL;DR

SeesawFaceNets proposes a lightweight, sparse, and robust face verification model for mobile platforms by integrating a novel Seesaw-block with squeeze-and-excitation (SE) modules into a MobileNetV2-based architecture. The model achieves state-of-the-art accuracy on multiple benchmarks with 66% lower FLOPs (146M vs. 221M) and 54.2% fewer parameters (1.3M vs. 2.4M) than SOTA models like MobiFace, while maintaining high performance under low computational and memory constraints.

ABSTRACT

Deep Convolutional Neural Network (DCNNs) come to be the most widely used solution for most computer vision related tasks, and one of the most important application scenes is face verification. Due to its high-accuracy performance, deep face verification models of which the inference stage occurs on cloud platform through internet plays the key role on most prectical scenes. However, two critical issues exist: First, individual privacy may not be well protected since they have to upload their personal photo and other private information to the online cloud backend. Secondly, either training or inference stage is time-comsuming and the latency may affect customer experience, especially when the internet link speed is not so stable or in remote areas where mobile reception is not so good, but also in cities where building and other construction may block mobile signals. Therefore, designing lightweight networks with low memory requirement and computational cost is one of the most practical solutions for face verification on mobile platform. In this paper, a novel mobile network named SeesawFaceNets, a simple but effective model, is proposed for productively deploying face recognition for mobile devices. Dense experimental results have shown that our proposed model SeesawFaceNets outperforms the baseline MobilefaceNets, with only {\bf66\%}(146M VS 221M MAdds) computational cost, smaller batch size and less training steps, and SeesawFaceNets achieve comparable performance with other SOTA model e.g. mobiface with only {\bf54.2\%}(1.3M VS 2.4M) parameters and {\bf31.6\%}(146M VS 462M MAdds) computational cost, It is also eventually competitive against large-scale deep-networks face recognition on all 5 listed public validation datasets, with {\bf6.5\%}(4.2M VS 65M) parameters and {\bf4.35\%}(526M VS 12G MAdds) computational cost.

Motivation & Objective

• To address privacy and latency issues in cloud-based face verification by enabling on-device inference.
• To design a lightweight deep neural network that maintains high accuracy with minimal computational and memory costs on mobile platforms.
• To improve upon existing mobile face recognition models like MobileFaceNet through architectural innovations without increasing inference complexity.
• To achieve competitive performance on public benchmarks with significantly reduced model size and FLOPs.

Proposed method

• Introduces the Seesaw-block, a modified inverted residual block with a skip connection branch using 2×2 max pooling and 1×1 convolution to enhance feature reuse.
• Employs squeeze-and-excitation (SE) modules to adaptively recalibrate channel features, improving representation learning with minimal computational overhead.
• Adopts a mobile-friendly architecture based on MobileNetV2, using depthwise separable convolutions and inverted bottlenecks to reduce FLOPs and parameters.
• Trains the model end-to-end on face verification datasets using standard optimization with reduced batch size and training steps to lower resource demands.
• Applies model compression techniques such as parameter pruning and efficient block design to maintain performance under strict hardware constraints.
• Validates the architecture on multiple public benchmarks using standard evaluation protocols to ensure robustness and generalization.

Experimental results

Research questions

• RQ1 Can a sparse and efficient neural network architecture achieve competitive face verification accuracy on mobile devices with significantly reduced FLOPs and parameters?
• RQ2 How does the Seesaw-block improve feature learning and model efficiency compared to standard residual or inverted bottleneck blocks?
• RQ3 To what extent can the proposed model outperform existing SOTA lightweight face recognition models like MobileFaceNet and MobiFace under identical hardware and training constraints?
• RQ4 Does the integration of SE modules and Seesaw-blocks enhance robustness across diverse face verification datasets without increasing computational cost?
• RQ5 Can the model be trained efficiently on limited GPU memory with small batch sizes while maintaining high accuracy?

Key findings

• SeesawFaceNets achieves 66% lower FLOPs (146M vs. 221M MAdds) than MobileFaceNet while maintaining comparable or better accuracy across all five benchmarks.
• The model uses only 54.2% of the parameters (1.3M vs. 2.4M) and 31.6% of the FLOPs (146M vs. 462M MAdds) of MobiFace, yet performs competitively on LFW and other datasets.
• The deeper and wider variant, DW-SeesawFaceNet, achieves comparable performance to large-scale deep networks with 6.5% of the parameters (4.2M vs. 65M) and only 4.35% of the FLOPs (526M vs. 12G MAdds).
• The Seesaw-block with skip connection improves feature reuse and model accuracy without adding extra computation, as confirmed by ablation studies.
• The model trains efficiently with a batch size of 128 and requires only 22GB GPU memory, enabling training from scratch on consumer-grade hardware.
• On all five public datasets—LFW, AgeDB-30, CFP-FP, CPLFW, and CALFW—SeesawFaceNets consistently outperforms MobileFaceNet and other SOTA lightweight models in terms of accuracy-to-efficiency ratio.