[Paper Review] AI-based BMI Inference from Facial Images: An Application to Weight Monitoring

Hera Siddiqui, Ajita Rattani+2 more|arXiv (Cornell University)|Oct 14, 2020

Cardiovascular Health and Risk Factors4 citations

TL;DR

This paper proposes and evaluates five deep learning-based CNN architectures—VGG19, ResNet50, DenseNet, MobileNet, and lightCNN—for estimating Body Mass Index (BMI) from facial images using transfer learning and regression. The key finding is that ResNet50 achieves the lowest Mean Absolute Error (MAE) of 1.04 on the Bollywood dataset, demonstrating the effectiveness of residual connections in BMI inference from facial features.

ABSTRACT

Self-diagnostic image-based methods for healthy weight monitoring is gaining increased interest following the alarming trend of obesity. Only a handful of academic studies exist that investigate AI-based methods for Body Mass Index (BMI) inference from facial images as a solution to healthy weight monitoring and management. To promote further research and development in this area, we evaluate and compare the performance of five different deep-learning based Convolutional Neural Network (CNN) architectures i.e., VGG19, ResNet50, DenseNet, MobileNet, and lightCNN for BMI inference from facial images. Experimental results on the three publicly available BMI annotated facial image datasets assembled from social media, namely, VisualBMI, VIP-Attributes, and Bollywood datasets, suggest the efficacy of the deep learning methods in BMI inference from face images with minimum Mean Absolute Error (MAE) of $1.04$ obtained using ResNet50.

Motivation & Objective

• To investigate and compare the performance of multiple deep CNN architectures for BMI inference from facial images.
• To evaluate the efficacy of pre-trained CNNs (VGG19, ResNet50, DenseNet, MobileNet, lightCNN) in estimating BMI using transfer learning.
• To analyze the impact of different regression techniques (SVR, Ridge Regression) on BMI prediction accuracy.
• To assess model generalization across diverse facial image datasets with varying demographics and image quality.
• To identify the most effective architecture for low-latency, on-device deployment in mobile health applications.

Proposed method

• Transfer learning is employed by using ImageNet-pretrained CNNs (VGG19, ResNet50, DenseNet, MobileNet, lightCNN) to extract deep features from facial images.
• Pre-trained CNNs are fine-tuned or used as feature extractors, with the final layer replaced to regress BMI as a continuous value.
• Two regression models—Support Vector Regression (SVR) and Ridge Regression (RR)—are applied to the extracted deep features for BMI prediction.
• Experiments are conducted on three public datasets: VisualBMI, VIP-Attributes, and Bollywood, each with diverse demographics and image conditions.
• Model performance is evaluated using Mean Absolute Error (MAE), with separate metrics computed for overall, male, and female subsets.
• End-to-end training of a custom CNN is also performed for comparison, using smooth L1 loss for regression.

Experimental results

Research questions

• RQ1 Which deep CNN architecture yields the lowest BMI prediction error when applied to facial images across diverse datasets?
• RQ2 How do different regression techniques (SVR vs. Ridge Regression) affect BMI inference performance when combined with deep features?
• RQ3 How does dataset variability—such as lighting, pose, makeup, and demographic diversity—affect BMI prediction accuracy?
• RQ4 What is the performance gap between pre-trained CNNs and a custom-trained CNN in BMI estimation from facial images?
• RQ5 To what extent do ResNet and DenseNet architectures outperform others due to architectural advantages like residual connections and feature reuse?

Key findings

• ResNet50 achieved the lowest Mean Absolute Error (MAE) of 1.04 on the Bollywood dataset when combined with Ridge Regression, outperforming all other models.
• On the VIP-Attributes dataset, ResNet50 achieved an MAE of 1.13 with Ridge Regression, the lowest among all models, indicating strong performance on celebrity images with lower BMI variance.
• DenseNet and ResNet50 consistently outperformed VGG19, MobileNet, and lightCNN across all datasets, particularly when using Ridge Regression.
• The use of Ridge Regression generally yielded lower MAE than Support Vector Regression across all models and datasets.
• The custom CNN architecture achieved significantly higher MAE (ranging from 4.12 to 6.65) compared to pre-trained models, indicating overfitting due to limited training data.
• No significant gender-based differences in prediction error were observed, suggesting consistent performance across male and female subjects.