[Paper Review] Revisiting Feature Alignment for One-stage Object Detection

Yuntao Chen, Chenxia Han+2 more|arXiv (Cornell University)|Aug 5, 2019

Advanced Neural Network ApplicationsComputer Science22 references63 citations

TL;DR

The paper introduces RoIConv to align features with bounding boxes in one-stage detectors, enabling a Fully Convolutional AlignDet that uses learned anchors and achieves state-of-the-art COCO results without sacrificing speed.

ABSTRACT

Recently, one-stage object detectors gain much attention due to their simplicity in practice. Its fully convolutional nature greatly reduces the difficulty of training and deployment compared with two-stage detectors which require NMS and sorting for the proposal stage. However, a fundamental issue lies in all one-stage detectors is the misalignment between anchor boxes and convolutional features, which significantly hinders the performance of one-stage detectors. In this work, we first reveal the deep connection between the widely used im2col operator and the RoIAlign operator. Guided by this illuminating observation, we propose a RoIConv operator which aligns the features and its corresponding anchors in one-stage detection in a principled way. We then design a fully convolutional AlignDet architecture which combines the flexibility of learned anchors and the preciseness of aligned features. Specifically, our AlignDet achieves a state-of-the-art mAP of 44.1 on the COCO test-dev with ResNeXt-101 backbone.

Motivation & Objective

• Identify the misalignment between anchor boxes and convolutional features in one-stage detectors.
• Propose a principled feature alignment method to bridge this gap.
• Develop a fully convolutional detector that leverages learned anchors with exact alignment.
• Demonstrate state-of-the-art performance on COCO with minimal anchor design.

Proposed method

• Reveal the link between im2col and RoIAlign to motivate a new RoIConv operator.
• Define RoIConv that uses anchor-induced offsets to align features with anchors at all locations.
• Construct a Fully Convolutional AlignDet with a Dense Proposal Module (DPM) and an Aligned Detection Module (ADM).
• Integrate RoIConv into the ADM to enable precise, plug-and-play alignment in one-stage detectors.
• Conduct ablations to compare RoIConv with vanilla conv, deformable conv, and anchor-guided deformable conv.

Experimental results

Research questions

• RQ1 Can explicit feature alignment between anchors and backbone features improve one-stage detectors without sacrificing speed?
• RQ2 Does RoIConv provide principled, location-aware alignment beyond multi-scale features?
• RQ3 How do learned anchors interact with alignment to affect detection accuracy on COCO?

Key findings

• RoIConv provides effective and efficient feature alignment, improving single-scale RetinaNet by 5.0 AP.
• AlignDet with ADM achieves 37.9 AP (ResNet-50) and 39.8 AP (ResNet-101) with 1 anchor, outperforming RetinaNet with multiple anchors.
• AlignDet with ResNet-101 FPN and 1 anchor reaches 39.8 AP and 52.8 APl on COCO minival, surpassing the RetinaNet baseline.
• Compared to RetinaNet, AlignDet improves AP by 5.5–5.3 points on minival for ResNet backbones.
• RoIConv designs with larger kernels and higher output channels yield higher AP, with 7x7 and 1024 channels achieving notable gains.
• AlignDet on COCO test-dev with ResNeXt-101-FPN backbone attains 44.1 AP, surpassing RetinaNet by 3.3 AP while maintaining speed.