[Paper Review] Mask Encoding for Single Shot Instance Segmentation

Rufeng Zhang, Zhi Tian+3 more|arXiv (Cornell University)|Mar 26, 2020

Advanced Neural Network Applications35 references4 citations

TL;DR

This paper proposes MEInst, a novel single-shot instance segmentation framework that encodes 2D instance masks into compact, fixed-dimensional vectors using dictionary learning (e.g., PCA), enabling efficient regression-based mask prediction within one-stage detectors like FCOS. The method achieves 36.9% mask AP on COCO with a single ResNeXt-101-FPN model and single-scale testing, outperforming prior one-stage methods and demonstrating strong performance on small objects while maintaining real-time inference capability.

ABSTRACT

To date, instance segmentation is dominated by twostage methods, as pioneered by Mask R-CNN. In contrast, one-stage alternatives cannot compete with Mask R-CNN in mask AP, mainly due to the difficulty of compactly representing masks, making the design of one-stage methods very challenging. In this work, we propose a simple singleshot instance segmentation framework, termed mask encoding based instance segmentation (MEInst). Instead of predicting the two-dimensional mask directly, MEInst distills it into a compact and fixed-dimensional representation vector, which allows the instance segmentation task to be incorporated into one-stage bounding-box detectors and results in a simple yet efficient instance segmentation framework. The proposed one-stage MEInst achieves 36.4% in mask AP with single-model (ResNeXt-101-FPN backbone) and single-scale testing on the MS-COCO benchmark. We show that the much simpler and flexible one-stage instance segmentation method, can also achieve competitive performance. This framework can be easily adapted for other instance-level recognition tasks. Code is available at: https://git.io/AdelaiDet

Motivation & Objective

• To address the challenge of compactly representing instance masks in one-stage instance segmentation, which has hindered performance compared to two-stage methods like Mask R-CNN.
• To enable efficient and accurate mask prediction by distilling 2D masks into fixed-dimensional representation vectors, reducing design and computational complexity.
• To demonstrate that a simple, one-stage framework with mask coefficient regression can achieve competitive accuracy without relying on complex architectures or multi-stage pipelines.
• To show that mask encoding via dictionary learning (e.g., PCA) enables robust, noise-resistant, and easily reconstructable mask representations.
• To establish a generalizable framework compatible with various one-stage detectors, including anchor-free and anchor-based paradigms.

Proposed method

• The method encodes each 2D instance mask into a compact, fixed-dimensional vector using dictionary learning techniques such as PCA, sparse coding, or autoencoders.
• The mask representation is learned from a dataset of instance masks, exploiting the low intrinsic dimensionality of natural masks to reduce redundancy.
• A parallel mask branch is added to one-stage detectors (e.g., FCOS) to regress the fixed-dimensional mask coefficients, enabling end-to-end training.
• The reconstructed mask is obtained by decoding the coefficient vector using the learned dictionary, allowing for high-fidelity mask reconstruction.
• The framework is compatible with both anchor-based and anchor-free detectors, requiring minimal architectural modifications.
• The method avoids contour-based representations that suffer from 'hollow decay' artifacts, especially on disjointed or complex shapes.

Experimental results

Research questions

• RQ1 Can a compact, fixed-dimensional vector representation of instance masks achieve competitive segmentation accuracy in a one-stage detector?
• RQ2 Does mask encoding via dictionary learning (e.g., PCA) enable better reconstruction fidelity and robustness compared to parametric contour-based methods?
• RQ3 Can a simple one-stage instance segmentation framework with mask coefficient regression outperform existing one-stage methods in both accuracy and inference speed?
• RQ4 How does the performance of the proposed method compare to two-stage methods like Mask R-CNN, especially on small and large objects?
• RQ5 To what extent can the mask encoding framework be generalized across different one-stage detector architectures?

Key findings

• MEInst achieves 36.9% mask AP on COCO with a single ResNeXt-101-FPN model and single-scale testing, outperforming most prior one-stage methods.
• The method improves by 11.8% in AP50 and 16.5% in AP75 over ESE-Seg, a state-of-the-art contour-based method, demonstrating superior accuracy and detail preservation.
• MEInst surpasses PolarMask in accuracy with similar computational complexity, due to lower reconstruction error and more effective mask representation.
• For small objects (area < 32²), MEInst achieves 38.0% mask AP compared to Mask R-CNN’s 35.3%, indicating a significant advantage in handling small instances.
• On large objects, MEInst achieves 46.7% mask AP compared to Mask R-CNN’s 53.5%, indicating a performance gap due to limited capacity of the compact vector representation for high-detail masks.
• With data augmentation (multi-scale), MEInst reaches 38.2% mask AP on COCO test-dev, demonstrating strong scalability and potential for further improvement.