[Paper Review] CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation

Yuntao Shou, Wei Ai+3 more|arXiv (Cornell University)|Dec 4, 2023

Face recognition and analysis5 citations

TL;DR

This paper proposes CZL-CIAE, a CLIP-driven zero-shot age estimation framework that leverages contrastive vision-language features, a Fourier-based Transformer (FourierFormer) for spatial and channel-wise feature fusion, and a reversible error-correction module to improve age prediction accuracy on unseen age categories. The method achieves state-of-the-art performance across multiple benchmarks, reducing MAE by up to 25% compared to baseline CLIP in zero-shot settings with 30% unseen categories.

ABSTRACT

The age estimation task aims to predict the age of an individual by analyzing facial features in an image. The development of age estimation can improve the efficiency and accuracy of various applications (e.g., age verification and secure access control, etc.). In recent years, contrastive language-image pre-training (CLIP) has been widely used in various multimodal tasks and has made some progress in the field of age estimation. However, existing CLIP-based age estimation methods require high memory usage (quadratic complexity) when globally modeling images, and lack an error feedback mechanism to prompt the model about the quality of age prediction results. To tackle the above issues, we propose a novel CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation (CILF-CIAE). Specifically, we first introduce the CLIP model to extract image features and text semantic information respectively, and map them into a highly semantically aligned high-dimensional feature space. Next, we designed a new Transformer architecture (i.e., FourierFormer) to achieve channel evolution and spatial interaction of images, and to fuse image and text semantic information. Compared with the quadratic complexity of the attention mechanism, the proposed Fourierformer is of linear log complexity. To further narrow the semantic gap between image and text features, we utilize an efficient contrastive multimodal learning module that supervises the multimodal fusion process of FourierFormer through contrastive loss for image-text matching, thereby improving the interaction effect between different modalities. Finally, we introduce reversible age estimation, which uses end-to-end error feedback to reduce the error rate of age predictions. Through extensive experiments on multiple data sets, CILF-CIAE has achieved better age prediction results.

Motivation & Objective

• To address the poor performance of existing zero-shot age estimation methods on unseen age categories.
• To improve generalization in age estimation without requiring labeled training data for unseen age groups.
• To introduce an error-controllable feedback mechanism to reduce prediction error in age estimation.
• To leverage CLIP’s pre-trained vision-language alignment for zero-shot transfer to age estimation.
• To design a novel Fourier-based Transformer architecture that enhances spatial and channel-wise feature interactions.

Proposed method

• Utilizes CLIP’s image and text encoders to extract aligned visual and textual features in a shared embedding space.
• Introduces a FourierFormer architecture that replaces standard attention mechanisms with Fourier priors to model spatial and channel-wise interactions in image features.
• Performs contrastive learning between image and text features to align semantic representations for zero-shot transfer.
• Employs a reversible age estimation module that uses end-to-end error feedback to refine predictions and reduce regression error.
• Fuses CLIP-encoded image and text features through the FourierFormer to generate context-aware representations for age prediction.
• Trains the entire model end-to-end using a multi-task loss combining contrastive learning and age regression.

Experimental results

Research questions

• RQ1 Can CLIP’s pre-trained vision-language features be effectively leveraged for zero-shot age estimation without fine-tuning on labeled age data?
• RQ2 How does replacing standard attention with Fourier priors in a Transformer architecture affect feature representation in age estimation?
• RQ3 To what extent does an end-to-end error-correction mechanism improve age prediction accuracy in zero-shot settings?
• RQ4 How do the individual components (spatial interaction, channel evolution, error correction) contribute to overall performance on unseen age categories?
• RQ5 Does the proposed method generalize better than baseline CLIP and existing zero-shot age estimation models under high data sparsity (e.g., 30% unseen categories)?

Key findings

• The full CZL-CIAE model achieves the lowest MAE across all six datasets when 30% of test categories are unseen, with a mean MAE of 23.7 on MORPH-S1.
• The error correction module alone reduces MAE by 2.4 points on average compared to the baseline CLIP model when 20% of categories are unseen.
• The combination of spatial interaction and error correction modules yields the best performance, reducing MAE by 3.2 points compared to using only the channel evolution module.
• Ablation studies confirm that all three modules—spatial interaction, channel evolution, and error correction—contribute significantly to performance, with the full model outperforming the ablated versions in all settings.
• On the FGNET dataset with 30% unseen categories, CZL-CIAE achieves an MAE of 23.7, a 4.6-point improvement over the baseline CLIP model.
• The model maintains consistent performance gains across all test splits, demonstrating strong robustness to varying levels of domain shift in unseen age groups.