[Paper Review] Cartoon Face Recognition: A Benchmark Dataset

Yi Zheng, Yifan Zhao+5 more|arXiv (Cornell University)|Jul 31, 2019

Face recognition and analysis42 references4 citations

TL;DR

This paper introduces iCartoonFace, the largest and most comprehensive benchmark dataset for cartoon face recognition, comprising 389,678 images of 5,013 characters with rich annotations. It proposes a multi-task domain adaptation framework that leverages human face data to improve cartoon face recognition and detection, achieving state-of-the-art performance with a Rank@1 accuracy of 84.34% using joint learning and domain regularization.

ABSTRACT

Recent years have witnessed increasing attention in cartoon media, powered by the strong demands of industrial applications. As the first step to understand this media, cartoon face recognition is a crucial but less-explored task with few datasets proposed. In this work, we first present a new challenging benchmark dataset, consisting of 389,678 images of 5,013 cartoon characters annotated with identity, bounding box, pose, and other auxiliary attributes. The dataset, named iCartoonFace, is currently the largest-scale, high-quality, richannotated, and spanning multiple occurrences in the field of image recognition, including near-duplications, occlusions, and appearance changes. In addition, we provide two types of annotations for cartoon media, i.e., face recognition, and face detection, with the help of a semi-automatic labeling algorithm. To further investigate this challenging dataset, we propose a multi-task domain adaptation approach that jointly utilizes the human and cartoon domain knowledge with three discriminative regularizations. We hence perform a benchmark analysis of the proposed dataset and verify the superiority of the proposed approach in the cartoon face recognition task. We believe this public availability will attract more research attention in broad practical application scenarios.

Motivation & Objective

• To address the lack of large-scale, high-quality datasets for cartoon face recognition in industrial and academic applications.
• To investigate the feasibility of transferring knowledge from human face recognition to cartoon face recognition.
• To develop a semi-automatic annotation pipeline that leverages human face data to improve labeling efficiency and accuracy for cartoon media.
• To establish a benchmark for both cartoon face recognition and face detection tasks under diverse real-world conditions.
• To evaluate the effectiveness of multi-task domain adaptation in bridging the domain gap between human and cartoon faces.

Proposed method

• Proposes iCartoonFace, a large-scale dataset with 389,678 images of 5,013 cartoon characters, annotated with identity, bounding box, 3D pose, gender, and other attributes.
• Develops a semi-automatic labeling algorithm that uses pre-trained human face detectors and classifiers as prior knowledge to accelerate and improve cartoon face annotation.
• Introduces a multi-task domain adaptation framework that jointly trains on human and cartoon face data using three discriminative regularizations: classification loss, unknown identity rejection loss, and domain alignment loss.
• Employs a shared backbone (e.g., DenseNet-169) with task-specific heads for face recognition and detection, enabling knowledge transfer between domains.
• Applies focal loss with ArcFace (F-ArcFace) to handle class imbalance and improve recognition performance on hard samples.
• Uses data augmentation with varying facial region expansion rates (0.0 to 1.0) to study the impact of contextual information on recognition accuracy.

Experimental results

Research questions

• RQ1 Can knowledge from human face recognition datasets be effectively transferred to improve cartoon face recognition?
• RQ2 How does the inclusion of contextual information (e.g., hair, accessories) affect cartoon face recognition performance?
• RQ3 What is the optimal configuration of multi-task domain adaptation losses for joint learning on human and cartoon face domains?
• RQ4 To what extent does semi-automatic annotation using human face data improve labeling efficiency and accuracy for cartoon datasets?
• RQ5 How does the performance of state-of-the-art models vary across different data augmentation and domain adaptation strategies on the iCartoonFace benchmark?

Key findings

• The proposed multi-task domain adaptation model with all three losses (classification, unknown identity rejection, and domain alignment) achieves the highest Rank@1 accuracy of 84.34% on iCartoonFace, outperforming baseline models.
• Including human face data (e.g., from WiderFace) in the training process improved cartoon face detection accuracy from 91.0% to 92.4%.
• Expanding the facial region by 0.75× (75% larger than original) yielded the best recognition performance, with Rank@1 reaching 78.29% and Rank@5 at 89.95%.
• F-ArcFace outperformed other state-of-the-art models, achieving 83.96% Rank@1, demonstrating the benefit of focal loss in handling hard samples in cartoon face recognition.
• The ablation study confirmed that domain alignment loss is critical for performance gain, as it significantly improves generalization across domains.
• The semi-automatic labeling pipeline successfully reduced annotation cost while maintaining high-quality annotations, enabling the creation of a large-scale, richly annotated dataset.