[Paper Review] Joint Estimation of Age and Gender from Unconstrained Face Images using Lightweight Multi-task CNN for Mobile Applications

Jia‐Hong Lee, Yi-Ming Chan+2 more|arXiv (Cornell University)|Jun 6, 2018

Face recognition and analysis15 references3 citations

TL;DR

This paper proposes a lightweight multi-task CNN (LMTCNN) for joint age and gender estimation from unconstrained face images, leveraging depthwise separable convolutions and hard parameter sharing to reduce model size and inference time. The method achieves comparable accuracy to state-of-the-art models while reducing model size by up to 9x and inference time by over 95% on mobile devices.

ABSTRACT

Automatic age and gender classification based on unconstrained images has become essential techniques on mobile devices. With limited computing power, how to develop a robust system becomes a challenging task. In this paper, we present an efficient convolutional neural network (CNN) called lightweight multi-task CNN for simultaneous age and gender classification. Lightweight multi-task CNN uses depthwise separable convolution to reduce the model size and save the inference time. On the public challenging Adience dataset, the accuracy of age and gender classification is better than baseline multi-task CNN methods.

Motivation & Objective

• Develop a compact deep learning model for real-time age and gender estimation on mobile devices with limited computational resources.
• Address the challenge of deploying accurate age and gender classification systems in unconstrained, real-world environments common in mobile applications.
• Reduce model size and inference latency without sacrificing classification accuracy by combining multi-task learning and efficient convolutional operations.
• Enable simultaneous inference for age and gender using a single shared feature extractor, minimizing memory and computational overhead.
• Demonstrate the feasibility of deploying the model on low-end mobile hardware such as smartphones and smart robots.

Proposed method

• Employ a hard parameter sharing paradigm in a multi-task learning framework, where a single shared CNN backbone extracts features for both age and gender classification.
• Replace standard convolutions in the baseline model with depthwise separable convolutions to significantly reduce computational cost and model size.
• Use depthwise separable convolutions, which decompose a standard convolution into depthwise (1x1 kernel per input channel) and pointwise (1x1 kernel for feature combination) operations.
• Apply width multiplier and resolution multiplier to further control model complexity and trade-off between speed and accuracy.
• Construct the LMTCNN architecture by modifying the state-of-the-art Levi_Hassner CNN [8] to integrate depthwise separable convolutions and multi-task learning.
• Convert the trained model into a TensorFlow computational graph for deployment on Android-based mobile devices.

Experimental results

Research questions

• RQ1 Can a lightweight multi-task CNN achieve comparable age and gender classification accuracy to larger models while being suitable for mobile deployment?
• RQ2 To what extent can depthwise separable convolutions reduce model size and inference time without degrading performance on unconstrained face images?
• RQ3 How does hard parameter sharing across age and gender tasks affect model efficiency and generalization in a mobile setting?
• RQ4 What is the real-world inference speed and memory footprint of the proposed model on low-end mobile hardware compared to existing methods?
• RQ5 Can the model maintain high accuracy on the challenging in-the-wild Adience benchmark while being optimized for mobile inference?

Key findings

• The LMTCNN with width multiplier 2 for the first depthwise separable convolution and 1 for the second achieved 44.26% top-1 accuracy on age estimation and 85.16% on gender classification, outperforming the baseline Levi_Hassner CNN in gender accuracy while being significantly smaller.
• The model size of LMTCNN with width multiplier 1 was only 8.7 MB, representing a 9x reduction compared to the 70.8 MB of the Levi_Hassner CNN.
• Inference speed on the ASUS Zenbo was 204.7 ms/frame for LMTCNN-1-1 and 297.6 ms/frame for LMTCNN-2-1, compared to ~4800 ms/frame for the Levi_Hassner CNN, achieving a 95%+ speedup.
• The LMTCNN achieved a 70.78% top-2 accuracy on age estimation and 85.16% on gender classification with a model only 30 MB when width multiplier was set to 2.
• The system was successfully deployed on mobile devices including the ASUS Zenbo robot and ASUS Zenfone 3 smartphone, demonstrating real-time performance and low memory usage.
• The use of depthwise separable convolutions reduced computational cost by a factor of $ \frac{D_K^2 C_O}{D_K^2 + C_O} $, with greater speedup for models with higher channel counts.