[Paper Review] Understanding the Impact of Label Granularity on CNN-based Image Classification

Zhuo Chen, Ruizhou Ding+2 more|arXiv (Cornell University)|Jan 21, 2019

Machine Learning and Data Classification19 references4 citations

TL;DR

This paper demonstrates that training CNNs with fine-grain labels—such as 'Persian cat' instead of 'cat'—improves both optimization and generalization, leading to higher accuracy on coarse-grain classification tasks. Surprisingly, a CNN trained on only 40% of the data with fine-grain labels outperforms one trained on the full dataset with coarse-grain labels, highlighting significant data efficiency gains.

ABSTRACT

In recent years, supervised learning using Convolutional Neural Networks (CNNs) has achieved great success in image classification tasks, and large scale labeled datasets have contributed significantly to this achievement. However, the definition of a label is often application dependent. For example, an image of a cat can be labeled as "cat" or perhaps more specifically "Persian cat." We refer to this as label granularity. In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as "Persian cat" can improve CNN accuracy on classifying coarse-grain classes, in this case "cat." The experimental results show that training CNNs with fine-grain labels improves both network's optimization and generalization capabilities, as intuitively it encourages the network to learn more features, and hence increases classification accuracy on coarse-grain classes under all datasets considered. Moreover, fine-grain labels enhance data efficiency in CNN training. For example, a CNN trained with fine-grain labels and only 40% of the total training data can achieve higher accuracy than a CNN trained with the full training dataset and coarse-grain labels. These results point to two possible applications of this work: (i) with sufficient human resources, one can improve CNN performance by re-labeling the dataset with fine-grain labels, and (ii) with limited human resources, to improve CNN performance, rather than collecting more training data, one may instead use fine-grain labels for the dataset. We further propose a metric called Average Confusion Ratio to characterize the effectiveness of fine-grain labeling, and show its use through extensive experimentation. Code is available at https://github.com/cmu-enyac/Label-Granularity.

Motivation & Objective

• To investigate how label granularity affects CNN performance in image classification tasks.
• To determine whether training with fine-grain labels improves accuracy and data efficiency compared to coarse-grain labeling.
• To propose a quantitative metric for predicting the effectiveness of fine-grain labeling.
• To explore practical applications of fine-grain labeling under varying human and data resource constraints.

Proposed method

• Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet datasets comparing training with coarse-grain vs. fine-grain labels.
• Training CNNs using both full and reduced (e.g., 40%) training data to evaluate data efficiency.
• Defining and computing the Average Confusion Ratio (ACR) to measure label hierarchy quality and predict performance gains.
• Introducing controlled noise in fine-grain labels via a randomness factor to test robustness of fine-grain labeling benefits.
• Varying the number of coarse-grain classes in CIFAR-100 to study diminishing returns of fine-grain labeling.
• Analyzing test accuracy improvements (ΔA^test) and correlating them with ACR values across multiple experimental settings.

Experimental results

Research questions

• RQ1 Does training with fine-grain labels improve classification accuracy on coarse-grain classes compared to training with coarse-grain labels?
• RQ2 How does fine-grain labeling affect data efficiency in CNN training?
• RQ3 What factors determine the effectiveness of fine-grain labeling in improving CNN performance?
• RQ4 Can a quantitative metric predict the performance gain from using fine-grain labels?
• RQ5 How does the number of coarse-grain classes influence the benefit of fine-grain labeling?

Key findings

• A CNN trained with fine-grain labels and only 40% of the CIFAR-10 training data achieved higher test accuracy (99.20%) than a CNN trained with the full dataset and coarse-grain labels (98.42%).
• Fine-grain labeling improved both training accuracy (optimization) and test accuracy (generalization) across all datasets tested, including CIFAR-10, CIFAR-100, and ImageNet subsets.
• The improvement from fine-grain labeling decreased as the number of coarse-grain classes increased, with negligible gains when the number of coarse-grain classes exceeded 15 in CIFAR-100.
• The Average Confusion Ratio (ACR) metric strongly correlated with performance gains, with higher ACR values predicting greater improvements in test accuracy.
• When fine-grain labels were corrupted with increasing randomness, the performance gain from fine-grain training diminished, indicating that label quality is critical.
• Fine-grain labeling led to faster convergence and better feature learning, enhancing the network's ability to generalize to coarse-grain classes.