[Paper Review] Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection

Xuanyi Dong, Yi Yang|arXiv (Cornell University)|Aug 6, 2019

Face recognition and analysis47 references7 citations

TL;DR

This paper proposes TS³, a teacher-student framework that improves semi-supervised facial landmark detection by using a teacher to filter high-quality pseudo-labels generated by two student detectors. The students iteratively refine predictions on unlabeled data, while the teacher dynamically evaluates and selects reliable pseudo-labels, enabling state-of-the-art performance with only 30% labeled data on 300-W and AFLW benchmarks.

ABSTRACT

Facial landmark detection aims to localize the anatomically defined points of human faces. In this paper, we study facial landmark detection from partially labeled facial images. A typical approach is to (1) train a detector on the labeled images; (2) generate new training samples using this detector's prediction as pseudo labels of unlabeled images; (3) retrain the detector on the labeled samples and partial pseudo labeled samples. In this way, the detector can learn from both labeled and unlabeled data to become robust. In this paper, we propose an interaction mechanism between a teacher and two students to generate more reliable pseudo labels for unlabeled data, which are beneficial to semi-supervised facial landmark detection. Specifically, the two students are instantiated as dual detectors. The teacher learns to judge the quality of the pseudo labels generated by the students and filter out unqualified samples before the retraining stage. In this way, the student detectors get feedback from their teacher and are retrained by premium data generated by itself. Since the two students are trained by different samples, a combination of their predictions will be more robust as the final prediction compared to either prediction. Extensive experiments on 300-W and AFLW benchmarks show that the interactions between teacher and students contribute to better utilization of the unlabeled data and achieves state-of-the-art performance.

Motivation & Objective

• To address the high annotation cost of facial landmark detection by leveraging unlabeled data in a semi-supervised manner.
• To reduce error accumulation from low-quality pseudo-labels in self-training frameworks.
• To improve robustness and generalization in facial landmark detection without requiring extensive hyper-parameter tuning.
• To enable effective collaboration between multiple student detectors through mutual regularization and ensemble prediction.

Proposed method

• Two student detectors (CPM and Hourglass) generate pseudo-labels for unlabeled facial images in an iterative self-training loop.
• A teacher network is trained to evaluate the quality of the pseudo-labels produced by the students using a learned criterion based on prediction consistency and confidence.
• The teacher filters out low-quality pseudo-labels, retaining only high-confidence, consistent predictions for retraining the students.
• The students are retrained using a combination of real labeled data and the filtered pseudo-labeled data, with predictions from both students ensembled to improve final output.
• The teacher is adaptively updated during training using the improved student predictions, enabling dynamic feedback and improved selection over time.
• The framework avoids overfitting by utilizing validation sets to guide the selection of high-quality pseudo-labels.

Experimental results

Research questions

• RQ1 Can a teacher network effectively filter low-quality pseudo-labels generated by student detectors in semi-supervised facial landmark detection?
• RQ2 How does the interaction between multiple students improve performance compared to single-model self-training?
• RQ3 Can the proposed teacher-student interaction mechanism achieve state-of-the-art performance with minimal hyper-parameter tuning?
• RQ4 To what extent can the model generalize when only 30% of the training data is labeled?

Key findings

• On the 300-W benchmark, TS³ achieves a mean error of 5.64 NME using only 10% labeled data, outperforming single-model baselines (CPM: 8.28 NME, HG: 6.25 NME).
• With 20% labeled data, TS³ reduces NME to 5.03 on 300-W, demonstrating faster convergence and better generalization than SPL and SPaCo.
• On the AFLW benchmark, TS³ achieves state-of-the-art performance in semi-supervised settings, even outperforming fully supervised models when only 30% of labels are used.
• The ablation study confirms that the interaction between two students improves performance by 30% over CPM and 9% over HG, highlighting the benefit of ensemble and mutual regularization.
• The teacher’s quality filtering mechanism prevents the model from falling into local minima, as seen in SPL, which degrades after round 4 due to poor pseudo-labels.
• The framework is robust to data imbalance and requires no additional hyper-parameter tuning beyond standard training, unlike co-training strategies that scale quadratically with the number of models.