[Paper Review] Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression

Dong Wang, Lei Zhou+3 more|arXiv (Cornell University)|Mar 15, 2018

Anomaly Detection Techniques and Applications5 references43 citations

TL;DR

This paper introduces a subspace-clustering based filter pruning method that reduces CNN size by clustering feature maps and pruning associated filters, improving compression with competitive accuracy after fine-tuning across VGG-16 and ResNet-50, and extending to object detection and pose estimation tasks.

ABSTRACT

While the research on convolutional neural networks (CNNs) is progressing quickly, the real-world deployment of these models is often limited by computing resources and memory constraints. In this paper, we address this issue by proposing a novel filter pruning method to compress and accelerate CNNs. Our work is based on the linear relationship identified in different feature map subspaces via visualization of feature maps. Such linear relationship implies that the information in CNNs is redundant. Our method eliminates the redundancy in convolutional filters by applying subspace clustering to feature maps. In this way, most of the representative information in the network can be retained in each cluster. Therefore, our method provides an effective solution to filter pruning for which most existing methods directly remove filters based on simple heuristics. The proposed method is independent of the network structure, thus it can be adopted by any off-the-shelf deep learning libraries. Experiments on different networks and tasks show that our method outperforms existing techniques before fine-tuning, and achieves the state-of-the-art results after fine-tuning.

Motivation & Objective

• Motivate CNN compression to address limited compute and memory resources in real-world deployments.
• Propose a filter pruning framework that leverages linear relationships in feature-map subspaces to retain representative information.
• Make the pruning method independent of network architecture for broad applicability.
• Demonstrate effectiveness across multiple networks (VGG-16, ResNet-50) and tasks (image classification, object detection, pose estimation).
• Show that subspace clustering-based pruning outperforms existing pruning strategies, especially before fine-tuning.

Proposed method

• Cluster input feature maps of a layer using subspace clustering to form ci' clusters (ci' ≤ ci).
• Compute average filters per cluster to form pruned filters across the layer and the next layer's channel axes.
• Reconstruct the pruned layer outputs by solving a linear least-squares problem to minimize reconstruction error (Frobenius norm).
• Iteratively prune layer-by-layer in a feedforward manner, with fine-tuning after pruning all target layers.
• Handle two pruning strategies for architectures: single-path networks (e.g., VGG) and multi-path networks (e.g., ResNet) with constraints on residual blocks."
• Theoretical framing uses a self-expressiveness model for subspace clustering: X = XC with diag(C)=0 and minimize ||C||1 to derive an affinity matrix for spectral clustering; subsequent steps form cluster indices to guide pruning (Equations (1)-(4)).

Experimental results

Research questions

• RQ1 Can linear relationships in feature-map subspaces be exploited to identify representative filters for pruning?
• RQ2 Does subspace-clustering-based pruning preserve accuracy better than traditional importance-based pruning methods before and after fine-tuning?
• RQ3 Is the proposed method architecture-agnostic and applicable to both single-path and multi-path CNNs?
• RQ4 How does the method perform on diverse tasks such as image classification, object detection, and pose estimation when used as backbone compression?
• RQ5 What are the trade-offs between pruning extent (speed-up) and accuracy across different models?

Key findings

• The method consistently outperforms several filter-selection baselines in single-layer pruning across VGG-16, preserving more information for reconstruction.
• Layer-by-layer pruning with subspace clustering achieves speed-ups (e.g., 2x, 4x, 5x) with smaller accuracy loss than competing methods without fine-tuning, and similar or better results after fine-tuning.
• On ResNet-50, the approach yields a 2x acceleration with smaller accuracy drop than a leading baseline, both without and with fine-tuning.
• For object detection with Faster R-CNN (VGG-16 backbone), 2x and 4x pruning cause minimal mAP loss (0.2 and 1.0 points, respectively) compared to baselines.
• For CMU-Pose on MSCOCO-14, 2x and 4x pruning incur modest mAP declines (0.8 and 1.9 points, respectively).
• Across tasks, the method demonstrates strong generalization in accelerating convolutional backbones—achieving notable reductions in computation and memory with limited degradation in performance after fine-tuning.