[Paper Review] Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression

Xinyao Wang, Liefeng Bo+1 more|arXiv (Cornell University)|Apr 16, 2019

Face recognition and analysis71 references41 citations

TL;DR

Introduces Adaptive Wing loss for heatmap regression in face alignment, plus Weighted Loss Map and CoordConv with boundary information to improve landmark localization; achieves state-of-the-art on COFW, 300W, and WFLW datasets.

ABSTRACT

Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks. Code will be made publicly available at https://github.com/protossw512/AdaptiveWingLoss.

Motivation & Objective

• Motivate and analyze loss functions for heatmap regression in facial landmark localization.
• Propose Adaptive Wing loss that adapts to ground-truth pixel intensities to focus on foreground and difficult background pixels.
• Introduce a Weighted Loss Map to balance foreground and background pixel contributions during training.
• Incorporate boundary information and CoordConv with boundary coordinates to improve coordinate-aware learning.
• Demonstrate state-of-the-art performance on standard benchmarks and validate transferability to related heatmap regression tasks.

Proposed method

• Analyze ideal loss properties for heatmap regression using influence arguments and ground-truth pixel intensities.
• Define Adaptive Wing (AWing) loss with an adaptive nonlinear region controlled by parameters (omega, theta, epsilon, alpha) and a smooth transition to a linear regime.
• Introduce a Weighted Loss Map that uses a dilated ground-truth heatmap to emphasize foreground and difficult background pixels (weight W).
• Add an auxiliary boundary prediction channel to capture global boundary information and fuse it via CoordConv with boundary-encoded coordinates (B_x, B_y) to enhance landmark localization.
• Leverage a stacked Hourglass (HG) backbone with coordinate encoding and boundary-aware channels; train with RMSProp and standard data augmentation; evaluate on COFW, 300W, and WFLW.

Experimental results

Research questions

• RQ1 Can a loss function that adapts its curvature to ground-truth heatmap pixel values improve heatmap regression for facial landmarks?
• RQ2 Does balancing foreground and difficult background pixels via a weighted loss map enhance convergence and localization accuracy?
• RQ3 Does incorporating boundary information and CoordConv encoding improve landmark localization accuracy?
• RQ4 How does the proposed Adaptive Wing loss perform on standard face-alignment benchmarks compared to prior methods?
• RQ5 Is the Adaptive Wing loss beneficial for other heatmap regression tasks such as human pose estimation?

Key findings

• AWing loss improves heatmap regression quality and landmark localization, outperforming prior methods on COFW, 300W, and WFLW datasets.
• On COFW, the approach reduces the 10% NME failure rate from 3.73% to 0.99%.
• On 300W, AWing achieves state-of-the-art results across common, challenging, and full test sets with notably lower NME and higher AUC metrics.
• On WFLW, AWing delivers the best results across subsets, significantly reducing failure rates and increasing AUC; overall NME and AUC improvements are substantial.
• The Adaptive Wing loss also demonstrates benefits in a human pose estimation task (LSP), improving PCK@0.2 over MSE baselines.
• Ablation studies show the combination AW + Weighted Loss Map + Boundary integration + CoordConv + boundary coordinates yields progressive gains, with AW alone contributing the most significant improvement.