[Paper Review] FP-Age: Leveraging Face Parsing Attention for Facial Age Estimation in the Wild

Yiming Lin, Jie Shen+2 more|arXiv (Cornell University)|Jun 21, 2021

Face recognition and analysis4 citations

TL;DR

This paper proposes FP-Age, a novel facial age estimation framework that leverages face parsing attention to explicitly focus on semantically informative facial regions—such as eyes and mouth—despite pose and expression variations in unconstrained environments. By integrating coarse and fine-grained features from a pre-trained face parsing network through a dedicated Face Parsing Attention module, FP-Age achieves state-of-the-art performance on IMDB-Clean and multiple benchmark datasets under both intra- and cross-dataset evaluation protocols.

ABSTRACT

Image-based age estimation aims to predict a person's age from facial images. It is used in a variety of real-world applications. Although end-to-end deep models have achieved impressive results for age estimation on benchmark datasets, their performance in-the-wild still leaves much room for improvement due to the challenges caused by large variations in head pose, facial expressions, and occlusions. To address this issue, we propose a simple yet effective method to explicitly incorporate facial semantics into age estimation, so that the model would learn to correctly focus on the most informative facial components from unaligned facial images regardless of head pose and non-rigid deformation. To this end, we design a face parsing-based network to learn semantic information at different scales and a novel face parsing attention module to leverage these semantic features for age estimation. To evaluate our method on in-the-wild data, we also introduce a new challenging large-scale benchmark called IMDB-Clean. This dataset is created by semi-automatically cleaning the noisy IMDB-WIKI dataset using a constrained clustering method. Through comprehensive experiment on IMDB-Clean and other benchmark datasets, under both intra-dataset and cross-dataset evaluation protocols, we show that our method consistently outperforms all existing age estimation methods and achieves a new state-of-the-art performance. To the best of our knowledge, our work presents the first attempt of leveraging face parsing attention to achieve semantic-aware age estimation, which may be inspiring to other high level facial analysis tasks. Code and data are available on \url{https://github.com/ibug-group/fpage}.

Motivation & Objective

• To improve facial age estimation performance in unconstrained, in-the-wild settings where pose, expression, and occlusion variations degrade model accuracy.
• To explicitly incorporate facial semantic information—such as eyes, nose, and mouth—into age estimation to guide attention toward more informative regions.
• To develop a clean, large-scale benchmark dataset for in-the-wild age estimation by semi-automatically removing noisy samples from IMDB-WIKI.
• To demonstrate that different facial parts contribute variably to age estimation, with the nose being the least informative.
• To establish a new state-of-the-art in age estimation by combining semantic feature learning with an attention mechanism in an end-to-end trainable framework.

Proposed method

• A pre-trained face parsing network is used to extract multi-scale semantic features from input facial images, providing both coarse and fine-grained representations of facial regions.
• A novel Face Parsing Attention (FPA) module is designed to dynamically weigh and attend to the most informative semantic features—particularly around eyes and mouth—by fusing them with high-level features.
• Low-level and high-level features are concatenated with learned face parsing masks to form a unified representation for age prediction.
• The FPA module uses a learnable attention mechanism that aggregates semantic-aware features, improving feature discrimination without requiring additional parsing annotations.
• The entire FP-Age network is trained end-to-end using standard age estimation loss, enabling joint optimization of parsing-aware attention and age prediction.
• A semi-automatic data cleaning pipeline based on constrained clustering is applied to the IMDB-WIKI dataset to generate the new IMDB-Clean benchmark, reducing noise and improving data quality.

Figure 1: FP-Age. A pre-trained face parsing framework [ 4 ] (top) is used to extract features of the target face in the input image. A lightweight network (bottom) aggregates low-level features, high-level features and face masks to predict the age. The shapes of tensors are labelled by the blocks

Experimental results

Research questions

• RQ1 Can explicitly modeling facial semantic parts through attention improve age estimation performance in unconstrained, in-the-wild settings?
• RQ2 How do different facial regions (e.g., eyes, mouth, nose) contribute to age estimation accuracy, and which are most informative?
• RQ3 Can a pre-trained face parsing network be effectively leveraged to improve age estimation without requiring additional annotations?
• RQ4 Does the proposed Face Parsing Attention mechanism lead to more robust and accurate age predictions across diverse datasets and evaluation protocols?
• RQ5 Can a cleaned, large-scale benchmark like IMDB-Clean lead to more reliable and generalizable performance gains in age estimation?

Key findings

• FP-Age achieves a new state-of-the-art mean absolute error (MAE) of 5.60 on the IMDB-Clean dataset under cross-dataset evaluation, significantly outperforming prior methods.
• On the FG-Net dataset, FP-Age achieves an MAE of 6.81 and a CS5 score of 48.49, surpassing all previous methods in cross-dataset evaluation.
• Statistical significance tests confirm that FP-Age’s performance is significantly better than eight competing methods (p-values corrected via Bonferroni correction were all < 1e-100).
• The nose region was found to be the least informative for age estimation, while eyes and mouth corners were consistently more critical for accurate predictions.
• The model demonstrates robustness across diverse datasets, achieving SOTA results on Morph, CACD, KANFace, and FG-Net under both intra- and cross-dataset protocols.
• The ablation study confirms that the Face Parsing Attention module significantly improves performance, validating its role in guiding attention to semantically relevant facial parts.

Figure 2: RoI Tanh-polar Transform [ 4 ] warps the whole image to a fix-sized representation in the Tanh-polar space with the given bounding box.