[论文解读] Revisiting the Importance of Individual Units in CNNs via Ablation

Bolei Zhou, Yiyou Sun+2 more|arXiv (Cornell University)|Jun 7, 2018

Adversarial Robustness in Machine Learning参考文献 19被引用 89

一句话总结

移除单个 CNN 单元通常对总体准确率影响较小，但对某些类别可能造成较大下降，表明这些单元针对特定类别子集具有专门性。该研究将单元属性与类别特定重要性联系起来，并检验旋转、批量归一化和 dropout 的影响。

ABSTRACT

We revisit the importance of the individual units in Convolutional Neural Networks (CNNs) for visual recognition. By conducting unit ablation experiments on CNNs trained on large scale image datasets, we demonstrate that, though ablating any individual unit does not hurt overall classification accuracy, it does lead to significant damage on the accuracy of specific classes. This result shows that an individual unit is specialized to encode information relevant to a subset of classes. We compute the correlation between the accuracy drop under unit ablation and various attributes of an individual unit such as class selectivity and weight L1 norm. We confirm that unit attributes such as class selectivity are a poor predictor for impact on overall accuracy as found previously in recent work \cite{morcos2018importance}. However, our results show that class selectivity along with other attributes are good predictors of the importance of one unit to individual classes. We evaluate the impact of random rotation, batch normalization, and dropout to the importance of units to specific classes. Our results show that units with high selectivity play an important role in network classification power at the individual class level. Understanding and interpreting the behavior of these units is necessary and meaningful.

研究动机与目标

• Motivate a detailed analysis of how individual CNN units contribute to visual recognition beyond overall accuracy.
• Quantify the impact of ablating single units on both overall and per-class accuracy across large-scale datasets.
• Investigate correlations between unit attributes (selectivity, correlation, L1 norm, concept alignment) and their class-specific importance.
• Examine how training regularizers like batch normalization and dropout affect unit interpretability and class-specific contributions.

提出的方法

• Perform unit ablations by zeroing a unit's weights and biases and measuring validation accuracy drops.
• Compute two types of drops: overall accuracy drop and per-class accuracy drop (max class accuracy drop per unit).
• Rank units by max class accuracy drop to assess class-specific information carried by each unit.
• Analyze correlations between unit attributes (L1 norm, class correlation, class selectivity, concept IoU) and both overall and max class accuracy drops.
• Compare ablations to random rotations of unit directions to distinguish directional phenomena from random directions.
• Evaluate effects of batch normalization and dropout on unit-level interpretability and class-specific contributions.

实验结果

研究问题

• RQ1 Do single-unit ablations significantly harm overall CNN accuracy or primarily affect a subset of classes?
• RQ2 Which unit attributes best predict a unit’s importance to per-class accuracy and to overall generalization?
• RQ3 How do regularizers like batch normalization and dropout influence the class-specific information carried by units?
• RQ4 Are the effects of ablation attributable to meaningful unit directions rather than random rotations in representation space?

主要发现

• A single unit ablation commonly yields small overall accuracy drops but can cause large drops for certain classes (e.g., >10% for some classes).
• Units tend to specialize, with removed units harming specific classes more than others, indicating alignment with single directions in representation space.
• Max class accuracy drop correlates negatively with several unit attributes, especially class selectivity and class correlation, signaling that more class-aligned units have larger per-class impact.
• L1 norm of a unit correlates with both overall and per-class drops, suggesting pruning-relevant weights relate to unit importance.
• Concept Alignment (IoU with visual concepts) is a strong predictor for which class a unit most affects, outperforming other single-attribute predictors in predicting per-unit class impact.
• Random rotations of unit directions generally show weaker class-specific effects than true unit directions, indicating specialization beyond random directions