[Paper Review] SOLOv2: Dynamic, Faster and Stronger.

Xinlong Wang, Rufeng Zhang+3 more|arXiv (Cornell University)|Mar 23, 2020

Advanced Neural Network ApplicationsComputer Science88 citations

TL;DR

SOLOv2 proposes a dynamic, fast, and accurate instance segmentation framework by decoupling the mask head into learnable kernel and feature branches, enabling location-conditioned mask prediction. It introduces Matrix NMS for one-shot, parallel non-maximum suppression, achieving 31.3 FPS inference speed and 37.1% AP on COCO, outperforming prior methods in speed-accuracy trade-off.

ABSTRACT

In this work, we aim at building a simple, direct, and fast instance segmentation framework with strong performance. We follow the principle of the SOLO method of Wang et al. SOLO: segmenting objects by locations. Importantly, we take one step further by dynamically learning the mask head of the object segmenter such that the mask head is conditioned on the location. Specifically, the mask branch is decoupled into a mask kernel branch and mask feature branch, which are responsible for learning the convolution kernel and the convolved features respectively. Moreover, we propose Matrix NMS (non maximum suppression) to significantly reduce the inference time overhead due to NMS of masks. Our Matrix NMS performs NMS with parallel matrix operations in one shot, and yields better results. We demonstrate a simple direct instance segmentation system, outperforming a few state-of-the-art methods in both speed and accuracy. A light-weight version of SOLOv2 executes at 31.3 FPS and yields 37.1% AP. Moreover, our state-of-the-art results in object detection (from our mask byproduct) and panoptic segmentation show the potential to serve as a new strong baseline for many instance-level recognition tasks besides instance segmentation. Code is available at: this https URL

Motivation & Objective

• To develop a simple, direct, and fast instance segmentation framework with strong performance.
• To improve mask prediction by dynamically conditioning the mask head on object location.
• To reduce inference time overhead from non-maximum suppression (NMS) in mask prediction.
• To establish a new strong baseline for instance-level recognition tasks, including detection and panoptic segmentation.

Proposed method

• Decomposes the mask head into a mask kernel branch and a mask feature branch, enabling location-aware mask prediction.
• Learns convolution kernels and feature maps separately, allowing dynamic adaptation based on object location.
• Introduces Matrix NMS, a one-shot, parallel matrix operation-based NMS method to accelerate inference.
• Applies the same architecture to object detection and panoptic segmentation as byproducts, leveraging shared features.
• Uses a simple, end-to-end training paradigm without complex head designs or post-processing.

Experimental results

Research questions

• RQ1 Can a simple, direct instance segmentation framework achieve state-of-the-art performance in both speed and accuracy?
• RQ2 How can mask prediction be made more adaptive to object location through dynamic head learning?
• RQ3 Can NMS inference overhead be significantly reduced without sacrificing detection quality?
• RQ4 Can the same architecture serve as a strong baseline for multiple instance-level recognition tasks?

Key findings

• SOLOv2 achieves 37.1% AP on COCO instance segmentation with a lightweight version running at 31.3 FPS.
• Matrix NMS reduces NMS inference time overhead through one-shot, parallel matrix operations.
• The decoupled mask head design improves mask quality by enabling location-conditioned predictions.
• The method sets state-of-the-art performance in object detection and panoptic segmentation using the same architecture.