[Paper Review] Scale-Aware Trident Networks for Object Detection

Yanghao Li, Yuntao Chen+2 more|arXiv (Cornell University)|Jan 7, 2019

Advanced Neural Network ApplicationsComputer Science44 references90 citations

TL;DR

TridentNet introduces scale-specific feature maps via parallel branches with shared weights and different receptive fields, trained with scale-aware sampling, achieving state-of-the-art single-model COCO results (e.g., 46.8–48.4 AP depending on setup) without extra parameters.

ABSTRACT

Scale variation is one of the key challenges in object detection. In this work, we first present a controlled experiment to investigate the effect of receptive fields for scale variation in object detection. Based on the findings from the exploration experiments, we propose a novel Trident Network (TridentNet) aiming to generate scale-specific feature maps with a uniform representational power. We construct a parallel multi-branch architecture in which each branch shares the same transformation parameters but with different receptive fields. Then, we adopt a scale-aware training scheme to specialize each branch by sampling object instances of proper scales for training. As a bonus, a fast approximation version of TridentNet could achieve significant improvements without any additional parameters and computational cost compared with the vanilla detector. On the COCO dataset, our TridentNet with ResNet-101 backbone achieves state-of-the-art single-model results of 48.4 mAP. Codes are available at https://git.io/fj5vR.

Motivation & Objective

• Investigate how receptive field size affects detection under scale variation.
• Propose a multi-branch Trident architecture with shared weights to create scale-specific features.
• Develop a scale-aware training scheme to specialize each branch to object scales.
• Offer a fast inference variant that maintains performance with no extra parameters.
• Evaluate TridentNet against state-of-the-art detectors on COCO and analyze ablations.

Proposed method

• Construct TridentNets by replacing certain backbone blocks with trident blocks that share parameters but use different dilation rates.
• Use weight sharing across branches to keep total parameters equal to baseline.
• Apply scale-aware training by selecting RoIs whose scale falls within predefined valid ranges per branch.
• During inference, fuse outputs from branches via NMS to produce final detections; include a fast TridentNet Fast variant using a single major branch.
• Provide a fast approximation that uses the middle branch for inference while preserving performance.

Experimental results

Research questions

• RQ1 How does varying the receptive field affect detection performance across object scales?
• RQ2 Can a multi-branch, scale-aware network achieve uniform representational power across scales without increasing parameters?
• RQ3 What is the impact of weight sharing and scale-aware training on overall detection accuracy compared to single-branch baselines?

Key findings

• Increasing receptive field helps large objects but can hurt small objects, motivating scale-aware, branch-specific processing.
• Three-branch TridentNet with shared weights and scale-aware training improves AP over baselines (e.g., from 37.9 to 40.6 on COCO minival with ResNet-101).
• With deformable backbones, TridentNet achieves higher AP (e.g., 41.8 vs 39.9 baseline).
• TridentNet with three branches reaches 46.6–46.8 AP on COCO test-dev without image pyramid, and 48.4 AP with image pyramid (single-model, ResNet-101-Deformable, TridentNet*).
• A fast inference variant (TridentNet Fast) using a major branch yields near-identical performance with no extra parameters or computation.