[論文レビュー] Learning Deep Bilinear Transformation for Fine-grained Image Representation

Heliang Zheng, Jianlong Fu+2 more|arXiv (Cornell University)|Nov 9, 2019

Advanced Neural Network Applications参考文献 33被引用数 77

ひとこと要約

この論文は、Deep Bilinear Transformation (DBT) ブロックを提案します。これはセマンティックにグルーピングされたチャネル内のグループ内バイリニア相互作用を学習し、CNN における深いスタックを可能にすることで、高度な識別を実現しつつ計算量を削減します。

ABSTRACT

Bilinear feature transformation has shown the state-of-the-art performance in learning fine-grained image representations. However, the computational cost to learn pairwise interactions between deep feature channels is prohibitively expensive, which restricts this powerful transformation to be used in deep neural networks. In this paper, we propose a deep bilinear transformation (DBT) block, which can be deeply stacked in convolutional neural networks to learn fine-grained image representations. The DBT block can uniformly divide input channels into several semantic groups. As bilinear transformation can be represented by calculating pairwise interactions within each group, the computational cost can be heavily relieved. The output of each block is further obtained by aggregating intra-group bilinear features, with residuals from the entire input features. We found that the proposed network achieves new state-of-the-art in several fine-grained image recognition benchmarks, including CUB-Bird, Stanford-Car, and FGVC-Aircraft.

研究の動機と目的

• Improve fine-grained image recognition by learning richer bilinear features without prohibitive computation.
• Integrate semantic information into bilinear pooling to focus on discriminative feature channels.
• Enable deep stacking of bilinear transformations within standard CNN backbones.
• Maintain or reduce feature dimensionality while capturing second-order interactions.

提案手法

• Introduce semantic grouping to partition channels into G semantic groups.
• Compute intra-group bilinear interactions within each group to form group bilinear features.
• Aggregate intra-group bilinear features across groups with group index encoding to preserve order.
• Incorporate a residual-like shortcut to fuse original and bilinear features and ease optimization.
• Stack DBT blocks in CNNs (e.g., ResNet backbones) with minimal additional parameters and ~5M FLOPs per block.

実験結果

リサーチクエスチョン

• RQ1 Can semantic grouping enable efficient intra-group bilinear interactions suitable for deep CNNs?
• RQ2 Does grouping channels by semantic parts improve discriminative bilinear representations for fine-grained tasks?
• RQ3 What is the impact of group index encoding and shortcut connections on learning and accuracy?
• RQ4 How does DBTNet compare to existing bilinear pooling methods on standard fine-grained benchmarks?
• RQ5 Is DBT effective when integrated into deeper networks and on large-scale datasets?

主な発見

• DBTNet achieves new state-of-the-art on CUB-200-2011, Stanford-Car, and FGVC-Aircraft datasets.
• DBTNet-50 (2k) reaches 87.5 on CUB, 94.1 on Stanford-Car, and 91.2 on Aircraft.
• DBTNet-101 (2k) reaches 88.1 on CUB, 94.5 on Stanford-Car, and 91.6 on Aircraft.
• Compared with Compact Bilinear and Kernel Pooling, DBT generally improves accuracy across datasets.
• On iNaturalist-2017, DBTNet-50 outperforms ResNet-50 by 2.1 percentage points; on ImageNet, DBTNet-50 beats ResNet-50 by 1.6 points.