[Paper Review] SOLO: Segmenting Objects by Locations

Xinlong Wang, Tao Kong+3 more|arXiv (Cornell University)|Dec 10, 2019

Advanced Neural Network ApplicationsComputer Science30 references37 citations

TL;DR

SOLO reframes instance segmentation as two pixel-level classification tasks by assigning each grid cell an object center location and size, enabling direct, box-free, one-shot mask prediction. It achieves competitive accuracy with Mask R-CNN and surpasses prior single-shot methods.

ABSTRACT

We present a new, embarrassingly simple approach to instance segmentation in images. Compared to many other dense prediction tasks, e.g., semantic segmentation, it is the arbitrary number of instances that have made instance segmentation much more challenging. In order to predict a mask for each instance, mainstream approaches either follow the 'detect-thensegment' strategy as used by Mask R-CNN, or predict category masks first then use clustering techniques to group pixels into individual instances. We view the task of instance segmentation from a completely new perspective by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size, thus nicely converting instance mask segmentation into a classification-solvable problem. Now instance segmentation is decomposed into two classification tasks. We demonstrate a much simpler and flexible instance segmentation framework with strong performance, achieving on par accuracy with Mask R-CNN and outperforming recent singleshot instance segmenters in accuracy. We hope that this very simple and strong framework can serve as a baseline for many instance-level recognition tasks besides instance segmentation.

Motivation & Objective

• Reconsider how to distinguish object instances by location and size.
• Introduce instance categories based on center locations and feature pyramid levels.
• Develop an end-to-end, single-shot framework that outputs masks and classes without post-processing.
• Leverage CoordConv to embed spatial information into CNNs.
• Demonstrate strong performance on COCO compared to existing methods.

Proposed method

• Divide an image into an S x S grid; each cell predicts a semantic category and an instance mask for the object whose center lies in that cell.
• Use a Feature Pyramid Network (FPN) to handle objects at different scales by assigning them to different feature levels.
• Attach two prediction heads (category and mask) per FPN level with shared weights; masks are conditioned on grid location.
• Incorporate CoordConv by concatenating pixel coordinates to input features to enable spatially varying predictions.
• Train with a combined loss L = L_cate + λ L_mask, where L_mask uses Dice loss for stable mask optimization.

Experimental results

Research questions

• RQ1 Can instance segmentation be performed directly in a single shot without bounding-box proposals or pixel-wise clustering?
• RQ2 Does encoding instance location and object size into an instance category enable accurate per-pixel mask predictions?
• RQ3 What impact do grid size, FPN levels, and CoordConv have on segmentation accuracy?
• RQ4 How does SOLO compare to state-of-the-art two-stage and one-stage methods on COCO?
• RQ5 What is the potential efficiency gain of a decoupled variant of SOLO?

Key findings

• SOLO achieves 37.8% mask AP with ResNet-101-FPN, competitive with Mask R-CNN (37.8% vs 37.8% in table; note: adjusted values in table).
• SOLO outperforms prior single-shot methods and approaches or matches two-stage methods on COCO test-dev.
• Decoupled SOLO (X and Y branches) yields 40.5 AP with DCN-101-FPN, and reduces memory usage.
• CoordConv significantly boosts AP (up to ~3.6 points) over standard convolutions.
• Dice loss provides best mask AP and training stability among tested loss functions.
• With larger grids and multi-level FPN, SOLO reaches 35.8 AP on COCO-val, demonstrating scalability across object sizes.