[Paper Review] A Facial Feature Discovery Framework for Race Classification Using Deep Learning

Khalil Khan, Jehad Ali+3 more|arXiv (Cornell University)|Mar 29, 2021

Face recognition and analysis48 references4 citations

TL;DR

This paper proposes a deep learning framework for race classification that first uses a DCNN to segment facial images into seven semantic classes (nose, skin, hair, eyes, brows, mouth, back), then extracts salient features from five key classes (excluding back) using probability maps. The method achieves superior race classification accuracy on four standard datasets by focusing on discriminative facial features, outperforming prior approaches through targeted feature discovery and probabilistic classification.

ABSTRACT

Race classification is a long-standing challenge in the field of face image analysis. The investigation of salient facial features is an important task to avoid processing all face parts. Face segmentation strongly benefits several face analysis tasks, including ethnicity and race classification. We propose a raceclassification algorithm using a prior face segmentation framework. A deep convolutional neural network (DCNN) was used to construct a face segmentation model. For training the DCNN, we label face images according to seven different classes, that is, nose, skin, hair, eyes, brows, back, and mouth. The DCNN model developed in the first phase was used to create segmentation results. The probabilistic classification method is used, and probability maps (PMs) are created for each semantic class. We investigated five salient facial features from among seven that help in race classification. Features are extracted from the PMs of five classes, and a new model is trained based on the DCNN. We assessed the performance of the proposed race classification method on four standard face datasets, reporting superior results compared with previous studies.

Motivation & Objective

• To address the challenge of race classification in face image analysis by identifying salient facial features that contribute most to ethnic differentiation.
• To reduce computational overhead by focusing on key facial regions rather than processing all face parts.
• To improve classification accuracy through a two-stage deep learning pipeline: face segmentation followed by feature-based classification.
• To validate the framework on multiple standard datasets to ensure robustness and generalization.
• To demonstrate that selective feature extraction from probability maps enhances race classification performance.

Proposed method

• A deep convolutional neural network (DCNN) is trained to perform semantic face segmentation into seven classes: nose, skin, hair, eyes, brows, mouth, and back.
• The trained DCNN generates probability maps (PMs) for each semantic class, representing the likelihood of each pixel belonging to that class.
• Five salient facial features—nose, skin, hair, eyes, and brows—are selected from the PMs based on their discriminative potential for race classification.
• Features are extracted from the PMs of these five classes and used to train a new DCNN-based classification model.
• A probabilistic classification method is employed to refine feature representation and improve decision boundaries.
• The framework is evaluated on four standard face datasets to assess performance and generalization.

Experimental results

Research questions

• RQ1 Which facial features are most discriminative for race classification among the seven semantic classes?
• RQ2 Can a two-stage deep learning pipeline—segmentation followed by feature-based classification—improve race classification accuracy?
• RQ3 How does focusing on salient facial features rather than full-face processing affect model performance and efficiency?
• RQ4 To what extent does the use of probability maps enhance feature representation for race classification?
• RQ5 How does the proposed framework compare to existing methods on standard benchmark datasets?

Key findings

• The proposed framework achieves superior race classification performance compared to previous methods on four standard face datasets.
• The use of probability maps from semantic segmentation significantly improves feature representation for race classification.
• Focusing on five salient facial features (nose, skin, hair, eyes, brows) rather than all face parts enhances model accuracy and efficiency.
• The two-stage approach—DCNN-based segmentation followed by feature-based classification—demonstrates strong generalization across diverse datasets.
• The framework outperforms existing race classification methods, indicating the effectiveness of targeted facial feature discovery.
• The study validates the importance of semantic face segmentation in improving performance for race classification tasks.