[Paper Review] SaccadeNet: A Fast and Accurate Object Detector

Shiyi Lan, Zhou Ren+3 more|arXiv (Cornell University)|Mar 26, 2020

Advanced Neural Network Applications32 references4 citations

TL;DR

SaccadeNet is a fast and accurate object detector inspired by human saccadic eye movements, which sequentially attends to object center and corner keypoints for coarse-to-fine localization. It achieves 40.4% mAP at 28 FPS and 30.5% mAP at 118 FPS on MS COCO, outperforming all existing real-time detectors by jointly predicting center and corner features without grouping overhead.

ABSTRACT

Object detection is an essential step towards holistic scene understanding. Most existing object detection algorithms attend to certain object areas once and then predict the object locations. However, neuroscientists have revealed that humans do not look at the scene in fixed steadiness. Instead, human eyes move around, locating informative parts to understand the object location. This active perceiving movement process is called extit{saccade}. %In this paper, Inspired by such mechanism, we propose a fast and accurate object detector called extit{SaccadeNet}. It contains four main modules, the \cenam, the \coram, the \atm, and the \aggatt, which allows it to attend to different informative object keypoints, and predict object locations from coarse to fine. The \coram~is used only during training to extract more informative corner features which brings free-lunch performance boost. On the MS COCO dataset, we achieve the performance of 40.4\% mAP at 28 FPS and 30.5\% mAP at 118 FPS. Among all the real-time object detectors, %that can run faster than 25 FPS, our SaccadeNet achieves the best detection performance, which demonstrates the effectiveness of the proposed detection mechanism.

Motivation & Objective

• To develop a real-time object detector that mimics human visual saccades by sequentially attending to informative object keypoints.
• To improve detection accuracy without sacrificing inference speed by leveraging joint prediction of center and corner features.
• To eliminate the need for time-consuming grouping algorithms used in keypoint-based detectors by predicting center and corners simultaneously.
• To investigate the impact of feature fusion from center and corner keypoints on localization and classification performance.
• To demonstrate that active, multi-stage attention to keypoint hierarchies enhances detection accuracy in real-time settings.

Proposed method

• Introduces a four-module architecture: Center Attentive Module, Corner Attentive Module, Attention Transitive Module, and Aggregation Attentive Module.
• Uses the Center Attentive Module to predict object center and category, followed by the Attention Transitive Module to predict rough corner locations.
• Employs the Corner Attentive Module during training to enhance boundary feature learning, improving localization accuracy with no inference cost.
• Applies the Aggregation Attentive Module to fuse features from center and corner keypoints, refining bounding box predictions.
• Jointly regresses center and corner locations to avoid grouping steps, maintaining high inference speed.
• Uses bilinear interpolation to compute diagonal and mid-edge points for improved keypoint supervision.

Experimental results

Research questions

• RQ1 Can a detector that mimics human saccadic eye movements improve both speed and accuracy in object detection?
• RQ2 Does joint prediction of center and corner keypoints eliminate the need for post-processing grouping while maintaining high performance?
• RQ3 How effective is feature fusion from center and corner keypoints in improving localization accuracy?
• RQ4 Does iterative refinement of bounding boxes through Aggregation-Attn improve detection performance at the cost of speed?
• RQ5 Can corner features enhance localization without degrading classification performance?

Key findings

• SaccadeNet achieves 40.4% mAP at 28 FPS and 30.5% mAP at 118 FPS on MS COCO using DLA-34 and ResNet-18 backbones, respectively, outperforming all prior real-time detectors.
• The Aggregation Attentive Module significantly improves localization, especially at high IoU thresholds (e.g., mAP@90 increases from 18.96% to 21.07% when both center and corner features are fused).
• Using corner features alone improves mAP@90 by 1.35 percentage points compared to using only center features, demonstrating the importance of boundary-aware features.
• Iterative refinement with multiple Aggregation-Attn modules improves mAP@90 from 18.96% to 21.32%, but at the cost of reduced speed, justifying the use of a single module for efficiency.
• Adding an Aggregation-Attn-Cls module for classification refinement degrades performance (mAP@50 drops from 71.02% to 52.26%), indicating corner features contribute little to classification.
• The Corner Attentive Module provides a free-lunch performance boost during training, enhancing localization without increasing inference time.