[Paper Review] YOLO9000: Better, Faster, Stronger

Joseph Redmon, Ali Farhadi|arXiv (Cornell University)|Dec 25, 2016

Advanced Neural Network Applications13 references435 citations

TL;DR

YOLO9000 jointly trains a real-time detector that can detect over 9000 object categories by combining detection data with large-scale classification data using a WordTree hierarchical labeling scheme.

ABSTRACT

We introduce YOLO9000, a state-of-the-art, real-time object detection system that can detect over 9000 object categories. First we propose various improvements to the YOLO detection method, both novel and drawn from prior work. The improved model, YOLOv2, is state-of-the-art on standard detection tasks like PASCAL VOC and COCO. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At 40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like Faster RCNN with ResNet and SSD while still running significantly faster. Finally we propose a method to jointly train on object detection and classification. Using this method we train YOLO9000 simultaneously on the COCO detection dataset and the ImageNet classification dataset. Our joint training allows YOLO9000 to predict detections for object classes that don't have labelled detection data. We validate our approach on the ImageNet detection task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite only having detection data for 44 of the 200 classes. On the 156 classes not in COCO, YOLO9000 gets 16.0 mAP. But YOLO can detect more than just 200 classes; it predicts detections for more than 9000 different object categories. And it still runs in real-time.

Motivation & Objective

• Improve YOLO to achieve higher recall and localization while maintaining speed.
• Develop a method to train detectors using both detection and classification data.
• Create a scalable label space enabling detection beyond a small fixed set of classes.

Proposed method

• Improve YOLO to YOLOv2 with batch normalization, high-resolution classifier, anchor boxes, and dimension priors.
• Use multi-scale training to enable variable input sizes and speed-accuracy tradeoffs.
• Replace location prediction with direct bounding box coordinates relative to grid cells for stability.
• Add a passthrough layer to fuse fine-grained features from earlier layers.
• Introduce Darknet-19 as the base network and train for detection with a three-prior setup.
• Propose WordTree hierarchical classification to merge ImageNet and COCO labels and enable joint training for detection and classification.
• Train YOLO9000 by blending COCO detection data with ImageNet classification data and propagate labels through the hierarchy.

Experimental results

Research questions

• RQ1 Can a single real-time detector be trained to recognize thousands of object categories using a combination of detection and classification data?
• RQ2 Does hierarchical labeling (WordTree) improve integration of multiple datasets and classes without mutual exclusion issues?
• RQ3 What architectural and training strategies yield state-of-the-art speed-accuracy tradeoffs for both detection and large-vocabulary classification?
• RQ4 How well can a detector trained with weakly labeled classification data generalize to detection tasks for unseen classes?
• RQ5 What is the impact of multi-scale training and feature fusion on small-object localization and overall mAP?

Key findings

• YOLOv2 achieves state-of-the-art speed-accuracy on VOC 2007 with 76.8 mAP at 67 FPS (and 78.6 mAP at 40 FPS).
• YOLOv2 outperforms Faster R-CNN with ResNet and SSD on VOC 2007 in speed-accuracy tradeoffs.
• YOLO9000 attains 19.7 mAP on ImageNet detection while only having detection data for 44 of 200 classes; 16.0 mAP on the 156 classes unseen in COCO.
• YOLO9000 can detect over 9000 object categories in real time by joint training on COCO and ImageNet via WordTree.
• Dimension priors learned by k-means improve recall and IOU alignment versus hand-picked priors.
• Joint training enables robust cross-dataset generalization with a hierarchical label structure.