[Paper Review] DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

Hao Zhang, Feng Li+6 more|arXiv (Cornell University)|Mar 7, 2022

Advanced Neural Network Applications747 citations

TL;DR

DINO improves DETR with contrastive denoising training, mixed query initialization, and a look-forward-two mechanism, achieving state-of-the-art end-to-end object detection on COCO with smaller models and data footprints. It demonstrates strong scalability across backbones and pretraining regimes.

ABSTRACT

We present DINO ( extbf{D}ETR with extbf{I}mproved de extbf{N}oising anch extbf{O}r boxes), a state-of-the-art end-to-end object detector. % in this paper. DINO improves over previous DETR-like models in performance and efficiency by using a contrastive way for denoising training, a mixed query selection method for anchor initialization, and a look forward twice scheme for box prediction. DINO achieves $49.4$AP in $12$ epochs and $51.3$AP in $24$ epochs on COCO with a ResNet-50 backbone and multi-scale features, yielding a significant improvement of $ extbf{+6.0}$ extbf{AP} and $ extbf{+2.7}$ extbf{AP}, respectively, compared to DN-DETR, the previous best DETR-like model. DINO scales well in both model size and data size. Without bells and whistles, after pre-training on the Objects365 dataset with a SwinL backbone, DINO obtains the best results on both COCO exttt{val2017} ($ extbf{63.2}$ extbf{AP}) and exttt{test-dev} ( extbf{$ extbf{63.3}$AP}). Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results. Our code will be available at \url{https://github.com/IDEACVR/DINO}.

Motivation & Objective

• Motivate end-to-end object detection without hand-crafted components like NMS or anchors.
• Improve DETR-like models' training efficiency and accuracy through novel training and query strategies.
• Demonstrate scalability across backbones and large-scale pretraining while reducing data/parameter requirements.

Proposed method

• Formulates detector queries as dynamic 4D anchor boxes within a Transformer framework and uses deformable attention.
• Introduces Contrastive DeNoising Training to create positive/negative samples for robust one-to-one matching.
• Proposes Mixed Query Selection to initialize positional queries from encoder features while keeping content queries learnable.
• Implements Look Forward Twice to allows gradients from later layers to refine earlier layer parameters for better box prediction.

Experimental results

Research questions

• RQ1 Can contrastive denoising improve one-to-one matching and reduce duplicates in DETR-like models?
• RQ2 Does mixed query initialization enhance spatial priors without compromising content query learning?
• RQ3 Will a look-forward-twice scheme improve early-layer box predictions by leveraging later-layer refined information?
• RQ4 How does DINO perform with smaller backbones and with large-scale pretraining compared to prior DETR-like models?
• RQ5 Is end-to-end Transformer detection competitive with or superior to classical detectors on COCO under varying data and model scales?

Key findings

• DINO achieves 49.0 AP (4-scale) and 49.4 AP (5-scale) on COCO val2017 with ResNet-50 in 12 epochs, and 51.3 AP (5-scale) in 24 epochs, outperforming previous DETR-like models.
• With SwinL backbone pre-trained on Objects365, DINO reaches 63.2 AP on COCO val2017 and 63.3 AP on test-dev, setting a new SOTA end-to-end DETR-like result.
• DINO-5scale outperforms DINO-4scale by +0.4 AP in val2017 and reaches higher AP in test-dev while using more scales; both demonstrate strong scalability.
• Ablations show that CDN (contrastive denoising) and look-forward-twice provide gains over optimized DN-DETR and pure query selection, with the full DINO configuration yielding the best results.
• DINO reduces model size to 1/15 of SwinV2-G and lowers pretraining data needs (1/60 for backbone, 1/5 for detection data) while achieving superior COCO scores.
• On small objects, DINO shows notable gains (+7.5 AP at 12 epochs) compared to baselines.