[Paper Review] Fine-grained visual recognition with salient feature detection

Hui Feng, Shanshan Wang+1 more|arXiv (Cornell University)|Aug 12, 2018

Advanced Neural Network Applications14 references4 citations

TL;DR

This paper proposes FP-CNN, a fine-grained visual recognition method that detects salient object parts using part annotations, extracts discriminative features from individual parts, and combines them for improved classification. It achieves 88.20% accuracy on the CUB200-2011 dataset, outperforming state-of-the-art methods by up to 7.2 percentage points.

ABSTRACT

Computer vision based fine-grained recognition has received great attention in recent years. Existing works focus on discriminative part localization and feature learning. In this paper, to improve the performance of fine-grained recognition, we try to precisely locate as many salient parts of object as possible at first. Then, we figure out the classification probability that can be obtained by using separate parts for object classification. Finally, through extracting efficient features from each part and combining them, then feeding to a classifier for recognition, an improved accuracy over state-of-art algorithms has been obtained on CUB200-2011 bird dataset.

Motivation & Objective

• To improve fine-grained visual recognition accuracy by precisely localizing salient object parts.
• To analyze the contribution of individual parts (e.g., head, wing, leg) to classification performance.
• To develop a method that combines features from multiple parts to achieve superior recognition accuracy.
• To outperform existing state-of-the-art methods on the CUB200-2011 benchmark.

Proposed method

• A deep neural network is trained to detect and localize salient parts using part annotations to generate labeled part images.
• Part regions are generated based on keypoint annotations (e.g., beak, crown, wing) using minimal rectangles or square envelopes.
• Features are extracted from each detected part using a fine-tuned ResNet-50 model.
• Features from multiple parts are combined incrementally using a classifier (libSVM) to evaluate performance gain.
• The final model uses features from the head, wing, and breast—selected based on individual part performance—for optimal recognition.
• The method is evaluated on the CUB200-2011 dataset with input images resized to 224×224.

Experimental results

Research questions

• RQ1 Which object parts contribute most significantly to fine-grained classification accuracy?
• RQ2 How does the performance of individual parts compare when used in isolation for classification?
• RQ3 Can incremental combination of features from multiple parts improve overall recognition accuracy beyond using full images or single parts?
• RQ4 Does the proposed salient part detection and feature combination method surpass existing state-of-the-art approaches on CUB200-2011?

Key findings

• The head region alone achieves 77.02% classification accuracy, outperforming all other individual parts and approaching the performance of full-image recognition (78.92%).
• The wing and breast parts each achieve approximately 50% accuracy when used independently, indicating they provide moderate but useful discriminative information.
• The leg and tail parts yield the lowest individual accuracies at 31.72% and 29.48%, respectively, indicating minimal discriminative power for classification.
• The incremental combination of features from the head, wing, and breast achieves the highest accuracy of 88.23%, slightly outperforming the combination of all parts.
• The final FP-CNN model achieves 88.20% accuracy on the CUB200-2011 test set, surpassing the best existing method (Mask-CNN at 87.30%) by 0.9 percentage points.
• The method outperforms weakly supervised approaches by 4.1 to 7.2 percentage points, demonstrating strong performance even without full supervision.