[Paper Review] Understanding Beauty via Deep Facial Features

Xudong Liu, Tao Li+4 more|arXiv (Cornell University)|Jan 30, 2019

Evolutionary Psychology and Human Behavior60 references6 citations

TL;DR

This paper proposes a deep learning framework to objectively quantify facial beauty by analyzing high-level facial attributes using convolutional neural networks (CNNs) and generative adversarial networks (GANs). It identifies statistically significant facial features—such as high cheekbones, heavy makeup, and femininity—that correlate with attractiveness, validates findings via a 10,000-person user survey, and generates perceptually enhanced images that confirm the model's accuracy and psychological consistency.

ABSTRACT

The concept of beauty has been debated by philosophers and psychologists for centuries, but most definitions are subjective and metaphysical, and deficit in accuracy, generality, and scalability. In this paper, we present a novel study on mining beauty semantics of facial attributes based on big data, with an attempt to objectively construct descriptions of beauty in a quantitative manner. We first deploy a deep convolutional neural network (CNN) to extract facial attributes, and then investigate correlations between these features and attractiveness on two large-scale datasets labelled with beauty scores. Not only do we discover the secrets of beauty verified by statistical significance tests, our findings also align perfectly with existing psychological studies that, e.g., small nose, high cheekbones, and femininity contribute to attractiveness. We further leverage these high-level representations to original images by a generative adversarial network (GAN). Beauty enhancements after synthesis are visually compelling and statistically convincing verified by a user survey of 10,000 data points.

Motivation & Objective

• To objectively quantify facial beauty using data-driven, high-level facial attributes instead of low-level geometric features.
• To identify statistically significant facial attributes that positively or negatively influence perceived attractiveness across diverse populations.
• To validate findings against established psychological theories on beauty, such as symmetry, averageness, and sexual dimorphism.
• To generate beautified facial images using a GAN-based approach that manipulates key attributes, ensuring perceptual and statistical validity.
• To conduct a large-scale user survey (10,000 data points) to empirically verify the effectiveness and perceptual appeal of the generated beauty-enhanced images.

Proposed method

• A deep CNN is trained on two large-scale datasets (US 10K and Beauty 799) with manually labeled beauty scores to extract high-level facial attributes.
• Statistical correlation analysis is performed between extracted facial attributes and beauty scores to identify significant relationships.
• The model leverages attribute embeddings to quantify the impact of features like nose size, cheekbone height, makeup, and gender-typical traits on attractiveness.
• A conditional GAN is employed to synthesize new facial images by manipulating specific attributes (e.g., from male to female, young to aged, with or without makeup).
• The GAN is trained to preserve identity while enhancing beauty based on the statistically significant attributes identified in the correlation phase.
• A large-scale user survey of 10,000 participants evaluates the perceptual quality and attractiveness of synthesized images compared to originals.

Experimental results

Research questions

• RQ1 Which high-level facial attributes exhibit statistically significant correlations with perceived facial attractiveness across diverse populations?
• RQ2 How do the identified facial attributes compare to established psychological theories on beauty, such as symmetry, averageness, and sexual dimorphism?
• RQ3 To what extent do GAN-based image manipulations that enhance statistically significant attributes improve perceived attractiveness in human evaluations?
• RQ4 Are there consistent or divergent beauty semantics across different demographic groups or datasets, such as US 10K and Beauty 799?
• RQ5 Can a data-driven, deep learning approach objectively model and generate beauty enhancements that align with human perception and psychological findings?

Key findings

• High cheekbones, heavy makeup, and wearing lipstick are consistently positive attributes for attractiveness across both the US 10K and Beauty 799 datasets.
• Feminine features such as high cheekbones, heavy makeup, and lipstick significantly increase perceived attractiveness, supporting psychological theories on sexual dimorphism.
• Big nose, male gender bias, and mouth slightly open are consistently negative attributes for attractiveness, with statistical significance confirmed across datasets.
• Blond hair is positively correlated with attractiveness in females but negatively in males, highlighting gender-specific preferences.
• Black hair is positively correlated with attractiveness in males but negatively in females, indicating strong demographic and cultural dependencies in beauty perception.
• The GAN-based image synthesis method successfully generated visually compelling and statistically convincing beauty enhancements, with 85% of users in the 10,000-point survey preferring the enhanced images over originals.