[Paper Review] Cascade R-CNN: Delving into High Quality Object Detection

Zhaowei Cai, Nuno Vasconcelos|arXiv (Cornell University)|Dec 3, 2017

Advanced Neural Network Applications25 references393 citations

TL;DR

Cascade R-CNN introduces a multi-stage detector with progressively higher IoU thresholds, trained recursively and applied during inference, to achieve high-quality object detection across architectures on COCO.

ABSTRACT

In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference-time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R-CNN is shown to surpass all single-model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R-CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code will be made available at https://github.com/zhaoweicai/cascade-rcnn.

Motivation & Objective

• Motivate the challenge that fixed IoU thresholds create trade-offs between positives and hard false positives in detection.
• Propose a cascaded detection architecture where later stages are specialized for higher IoU (quality) levels.
• Show that cascaded training and inference mitigate overfitting and distribution mismatch between training and testing proposals.
• Demonstrate that Cascade R-CNN provides consistent gains across different backbone detectors on COCO, with modest computational overhead.

Proposed method

• Introduce a cascade of detectors trained with increasing IoU thresholds u^t to progressively suppress close false positives.
• Use a cascaded bounding box regression where f_T ◦ f_{T-1} ◦ ... ◦ f_1 refines proposals stage by stage with stage-specific training distributions.
• Normalize the regression targets Δ=(δx, δy, δw, δh) with stage-specific mean/variance to stabilize multi-task learning.
• Train each stage with a loss L(x^t,g)=L_cls(h_t(x^t),y^t)+λ[y^t≥1]L_loc(f_t(x^t,b^t),g) to ensure stage-wise quality growth.
• Apply the same cascade during inference to align hypothesis quality with detector capability at each stage.
• Demonstrate that cascaded distributions keep positive samples roughly constant in size, reducing overfitting and improving high-IoU performance.
• Show compatibility with multiple baseline two-stage detectors (Faster R-CNN, R-FCN, FPN) without bespoke bells-and-whistles.

Experimental results

Research questions

• RQ1 Can a cascaded detector architecture improve high-IoU (high-quality) detections compared to single-threshold detectors?
• RQ2 Does training and inference with progressively higher IoU thresholds reduce overfitting and mismatch between proposal quality and detector quality?
• RQ3 Is Cascade R-CNN effective across different baseline detectors and backbones on COCO?
• RQ4 What is the impact of cascade depth and stage-specific IoU thresholds on localization and detection performance?

Key findings

• The Cascade R-CNN outperforms all single-model detectors on COCO with a vanilla implementation.
• Gains are consistent across baseline detectors and backbones, typically in the range of 2 to 4 AP points.
• Cascade regression improves localization for higher-IoU hypotheses, unlike single regressor or iterative BBox approaches which degrade at high IoU.
• An integral-loss multi-head setup is outperformed by the cascade approach for all metrics.
• Three-stage Cascade R-CNN provides the best overall balance; a fourth stage yields diminishing returns and can hurt overall AP but boosts high-IoU performance.
• The approach generalizes across Faster R-CNN, R-FCN, and FPN backbones, maintaining gains with increasing detector strength.