[Paper Review] A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images

John Leuner|arXiv (Cornell University)|Feb 27, 2019

Evolutionary Psychology and Human BehaviorPsychology25 references17 citations

TL;DR

This replication study confirms that machine learning models can predict sexual orientation from facial images with high accuracy using deep neural networks (68% male, 77% female) and facial morphology (62% male, 72% female). It further demonstrates that even highly blurred images—containing only low-level color and brightness features—can predict orientation (63% male, 72% female), indicating that grooming, lighting, and image processing artifacts, rather than facial structure alone, contribute significantly to model performance.

ABSTRACT

Recent research used machine learning methods to predict a person's sexual orientation from their photograph (Wang and Kosinski, 2017). To verify this result, two of these models are replicated, one based on a deep neural network (DNN) and one on facial morphology (FM). Using a new dataset of 20,910 photographs from dating websites, the ability to predict sexual orientation is confirmed (DNN accuracy male 68%, female 77%, FM male 62%, female 72%). To investigate whether facial features such as brightness or predominant colours are predictive of sexual orientation, a new model based on highly blurred facial images was created. This model was also able to predict sexual orientation (male 63%, female 72%). The tested models are invariant to intentional changes to a subject's makeup, eyewear, facial hair and head pose (angle that the photograph is taken at). It is shown that the head pose is not correlated with sexual orientation. While demonstrating that dating profile images carry rich information about sexual orientation these results leave open the question of how much is determined by facial morphology and how much by differences in grooming, presentation and lifestyle. The advent of new technology that is able to detect sexual orientation in this way may have serious implications for the privacy and safety of gay men and women.

Motivation & Objective

• To replicate and validate the findings of Wang and Kosinski (2017) that machine learning can predict sexual orientation from facial images.
• To investigate whether facial morphology, grooming, or image processing artifacts are the primary drivers of prediction accuracy.
• To assess model robustness to intentional alterations such as makeup, facial hair, eyewear, and head pose.
• To determine whether low-level visual features like brightness, hue, and saturation in blurred images carry predictive information about sexual orientation.
• To evaluate the implications of such models for privacy and safety, especially for LGBTQ+ individuals.

Proposed method

• Collected a new dataset of 20,910 facial images from online dating profiles, focusing on white individuals from the U.S.
• Trained a deep neural network (DNN) model on cropped facial images to extract high-level features for sexual orientation prediction.
• Developed a facial morphology (FM) classifier using geometric features such as facial contour, eye and nose positions, and face shape.
• Created a third model trained on highly blurred images (1×1 and 5×5 pixel resolution) to isolate low-level color and brightness information.
• Conducted controlled experiments altering makeup, facial hair, eyewear, and head pose to test model invariance.
• Used area under the ROC curve (AUC) as the primary metric to evaluate model performance across all studies.

Experimental results

Research questions

• RQ1 Can machine learning models replicate the high prediction accuracy of sexual orientation from facial images reported by Wang and Kosinski?
• RQ2 To what extent do low-level image features such as brightness, hue, and saturation in blurred images contribute to prediction accuracy?
• RQ3 Are the models robust to intentional changes in appearance such as makeup, facial hair, eyewear, and head pose?
• RQ4 Is there a correlation between head pose and sexual orientation, or is the prediction invariant to pose variation?
• RQ5 To what extent are differences in grooming, lifestyle, or image processing artifacts responsible for model performance, rather than biological facial morphology?

Key findings

• The DNN model achieved an AUC of 0.68 for males and 0.77 for females, replicating and confirming the results of Wang and Kosinski.
• The facial morphology model achieved an AUC of 0.62 for males and 0.72 for females, indicating that geometric facial features alone are predictive.
• A model trained on 5×5 pixel blurred images achieved an AUC of 0.63 for males and 0.72 for females, proving that low-level color and brightness features are predictive.
• The models remained robust to intentional alterations such as makeup, facial hair, eyewear, and head pose, indicating invariance to these changes.
• No significant correlation was found between head pose and sexual orientation, suggesting pose does not influence model predictions.
• Composite images and color distribution analyses revealed that straight females and gay males tend to have brighter faces and more vivid lip colors, suggesting grooming and lighting differences are key predictors.