[论文解读] InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity

Hee Jung Ryu, Hartwig Adam+1 more|arXiv (Cornell University)|Dec 1, 2017

Face recognition and analysis参考文献 13被引用 81

一句话总结

InclusiveFaceNet 从保留的数据集中学习种族和性别表示，并将其转移到人脸属性检测，在提升少数群体子组的准确性同时保持人口统计隐私。

ABSTRACT

We demonstrate an approach to face attribute detection that retains or improves attribute detection accuracy across gender and race subgroups by learning demographic information prior to learning the attribute detection task. The system, which we call InclusiveFaceNet, detects face attributes by transferring race and gender representations learned from a held-out dataset of public race and gender identities. Leveraging learned demographic representations while withholding demographic inference from the downstream face attribute detection task preserves potential users' demographic privacy while resulting in some of the best reported numbers to date on attribute detection in the Faces of the World and CelebA datasets.

研究动机与目标

• Motivate the need to improve face attribute detection fairness across race and gender subgroups.
• Propose a twofold transfer learning approach to incorporate demographic representations without run-time demographic inference.
• Demonstrate that leveraging demographic diversity can improve overall and subgroup-specific accuracy.
• Show that the method achieves competitive or superior results on public datasets (FotW and CelebA).

提出的方法

• Use transfer learning from a face recognition model (FaceNet) to extract demographic representations for race and gender from a held-out dataset with uniform distribution.
• Freeze the base recognition representations and train separate but coupled classifiers for race and gender (diversity classifier) prior to learning the attribute detector.
• Transfer the learned race and gender representations to the downstream face attribute detection network (InclusiveFaceNet) in a twofold transfer learning setup.
• Train a multihead face attribute detector with transferred race and gender heads, while not inferring demographic attributes at deployment time.
• Evaluate using accuracy per demographic subgroup and Average False Rate (AFR) to assess fairness across subgroups.
• Compare against baselines and prior art to establish state-of-the-art performance across datasets.

实验结果

研究问题

• RQ1 Can learning race and gender representations from a held-out demographic dataset improve face attribute detection across race and gender subgroups?
• RQ2 Does transferring demographic representations improve overall attribute detection accuracy without requiring demographic inference at run time?
• RQ3 How does InclusiveFaceNet perform on Faces of the World (FotW) and CelebA datasets compared to prior methods?

主要发现

• The twofold transfer learning approach matches or improves baseline attribute detection, establishing new state-of-the-art performance across gender boundaries on CelebA and FotW.
• Transfer of race representations yields accuracy gains for multiple attributes and subgroups, including a notable improvement in smiling detection on FotW.
• On CelebA, race representations improve accuracy for 23 of 40 attributes, with some attributes showing up to 0.9% absolute gains; some attributes show small losses.
• Overall, 35 out of 40 CelebA attributes are maintained or improved with race transfer, and 10 attributes reach new state-of-the-art accuracy.
• FotW results show the proposed model with race and gender transfer outperforming prior art, with the best model achieving 90.96% accuracy on smiling with race transfer.
• The method preserves demographic privacy by not inferring demographic attributes at deployment time.