[Paper Review] Discriminability Distillation in Group Representation Learning

Manyuan Zhang, Guanglu Song+2 more|arXiv (Cornell University)|Aug 25, 2020

Video Surveillance and Tracking Methods76 references4 citations

TL;DR

This paper proposes Discriminability Distillation Learning (DDL), a lightweight post-processing method that enhances group representation learning by distilling discriminability scores from a pre-trained model using embedded class centroids. By identifying elements with high discriminability—measured as the ratio of intra-class to hardest-negative inter-class distances—DDL enables efficient, explainable feature aggregation, achieving state-of-the-art performance across face recognition, person re-identification, and action recognition with minimal computational overhead.

ABSTRACT

Learning group representation is a commonly concerned issue in tasks where the basic unit is a group, set, or sequence. Previously, the research community tries to tackle it by aggregating the elements in a group based on an indicator either defined by humans such as the quality and saliency, or generated by a black box such as the attention score. This article provides a more essential and explicable view. We claim the most significant indicator to show whether the group representation can be benefited from one of its element is not the quality or an inexplicable score, but the discriminability w.r.t. the model. We explicitly design the discrimiability using embedded class centroids on a proxy set. We show the discrimiability knowledge has good properties that can be distilled by a light-weight distillation network and can be generalized on the unseen target set. The whole procedure is denoted as discriminability distillation learning (DDL). The proposed DDL can be flexibly plugged into many group-based recognition tasks without influencing the original training procedures. Comprehensive experiments on various tasks have proven the effectiveness of DDL for both accuracy and efficiency. Moreover, it pushes forward the state-of-the-art results on these tasks by an impressive margin.

Motivation & Objective

• To address the lack of explainability and efficiency in existing group representation learning methods that rely on attention or quality scores.
• To identify a more fundamental and interpretable indicator for selecting representative elements in a group.
• To develop a light-weight distillation network that can efficiently estimate discriminability without retraining the base model.
• To improve recognition accuracy and computational efficiency in group-based tasks such as set-to-set face recognition, video-based person re-identification, and action recognition.
• To enable generalization of discriminability knowledge across unseen data distributions without modifying the original network.

Proposed method

• Defining discriminability as the ratio of an element’s distance to its class centroid over its distance to the closest hard negative centroid, using embeddings from a pre-trained model.
• Training a light-weight Discriminability Distillation Network (DDNet) that regresses regularized discriminability scores from raw input images.
• Applying the DDNet during inference to score all elements in a group, then aggregating features using these discriminability weights.
• Enabling a filtering strategy that selects only high-discriminability elements to reduce computational cost while preserving performance.
• Using the same DDNet architecture across tasks by replacing 2D convolutions with 3D convolutions for video-based applications.
• Integrating DDL as a plug-in module that does not require retraining the base network, ensuring flexibility and compatibility.

Experimental results

Research questions

• RQ1 Can discriminability, defined via inter- and intra-class distances, serve as a more explainable and effective indicator than attention or quality scores for group representation learning?
• RQ2 Can a lightweight distillation network accurately and efficiently regress discriminability scores from raw inputs without retraining the base model?
• RQ3 Does the distilled discriminability knowledge generalize well to unseen data and diverse group-based recognition tasks?
• RQ4 Can DDL improve recognition accuracy while reducing computational cost in tasks like video-based person re-identification and action recognition?
• RQ5 How does DDL compare to dense sampling, random sampling, and uniform sampling in terms of accuracy and efficiency on untrimmed and trimmed video datasets?

Key findings

• DDL improved mAP by 3.6% over average pooling and outperformed attention mechanisms and RNN-based aggregation on set-to-set face recognition.
• On the Mars dataset, DDL achieved 77.7% mAP and 84.0% CMC-1, surpassing the previous state-of-the-art by 1.0% mAP and 2.2% CMC-1.
• On ActivityNet-1.2, DDL achieved a 4% accuracy gain over random or uniform sampling with only 9 clips, and outperformed dense sampling (5x more clips) by 2.49%.
• On Kinetics-700, DDL improved accuracy by 1.84% over random sampling and 2.18% over uniform sampling, while achieving 6x speedup compared to dense sampling.
• The DDNet was highly efficient, requiring only a fraction of the computation of the base model, and enabled significant inference speedups through selective feature extraction.
• The method generalized well across diverse tasks, including face recognition, person re-identification, and action recognition, without modifying the base network.