[Paper Review] Improving Fast Segmentation With Teacher-student Learning

Jiafeng Xie, Bing Shuai+3 more|arXiv (Cornell University)|Oct 19, 2018

Advanced Neural Network Applications16 references51 citations

TL;DR

The paper introduces a teacher-student learning framework to boost fast segmentation models by transferring zero-order and first-order knowledge from a heavy teacher to a fast student, using both labeled and unlabeled data without extra inference cost.

ABSTRACT

Recently, segmentation neural networks have been significantly improved by demonstrating very promising accuracies on public benchmarks. However, these models are very heavy and generally suffer from low inference speed, which limits their application scenarios in practice. Meanwhile, existing fast segmentation models usually fail to obtain satisfactory segmentation accuracies on public benchmarks. In this paper, we propose a teacher-student learning framework that transfers the knowledge gained by a heavy and better performed segmentation network (i.e. teacher) to guide the learning of fast segmentation networks (i.e. student). Specifically, both zero-order and first-order knowledge depicted in the fine annotated images and unlabeled auxiliary data are transferred to regularize our student learning. The proposed method can improve existing fast segmentation models without incurring extra computational overhead, so it can still process images with the same fast speed. Extensive experiments on the Pascal Context, Cityscape and VOC 2012 datasets demonstrate that the proposed teacher-student learning framework is able to significantly boost the performance of student network.

Motivation & Objective

• Motivate the need for fast yet accurate segmentation in real-time or resource-constrained settings.
• Propose a teacher-student framework to regularize fast student learning using teacher-derived knowledge.
• Extend the framework to leverage unlabeled data by generating pseudo-ground-truth from the teacher.
• Demonstrate gains on multiple benchmarks (Pascal Context, Cityscapes, VOC 2012) without increasing inference cost.

Proposed method

• Define a student S (fast) and a fixed teacher T (heavy) and optimize L = L_S + r(S,T).
• Regularize S with zero-order knowledge via probability loss L_p between teacher and student outputs.
• Regularize S with first-order knowledge via consistency loss L_c on boundary information between teacher and student outputs.
• Distill knowledge on fine-annotated data and extend to unlabeled data by using teacher-generated pseudo labels as ground truth.
• On unlabeled data, train with L = L_LabeledData + λ L_unlabeledData to jointly optimize both data regimes.

Experimental results

Research questions

• RQ1 Can a teacher network’s knowledge improve a fast segmentation model without increasing inference cost?
• RQ2 Does combining zero-order (probability) and first-order (consistency) knowledge enhance student learning beyond using either alone?
• RQ3 Can unlabeled data further boost performance via teacher-generated supervision without manual annotations?
• RQ4 How does the method perform across standard segmentation benchmarks and with different teacher/student backbones?

Key findings

• On Pascal Context, the Enhanced MobileNet-1.0-DeepLab-v2 (with L_p, L_c, and unlabeled data) achieves 43.8% mIoU at 46.5 FPS, compared to the base 40.9% mIoU.
• In ablations, L_p alone raises mIoU to 42.3% and adding L_c to 42.8%; unlabeled data yields a further 1.0% gain to 43.8% mIoU.
• Using a fixed high-capacity teacher (ResNet-101 DeepLab-v2) and MobileNet-based student, the student reaches 71.9% mIoU on Cityscapes validation (enhanced from 67.3%), while maintaining fast inference (20.6 FPS).
• On VOC 2012 validation, the enhanced MobileNet-1.0-DeepLab-v2 achieves 69.6% mIoU, outperforming the base student by 2.3%.
• Across three datasets, the framework consistently improves student performance without additional computational overhead.
• The improvement magnitude generally increases with larger teacher-student performance gaps, indicating effective knowledge transfer.