[论文解读] Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts

Sagi Eppel, Alán Aspuru‐Guzik|arXiv (Cornell University)|Aug 24, 2019

Industrial Vision Systems and Defect Detection被引用 4

一句话总结

本文提出了一种生成器-评估器-选择器网络用于全景图像分割，通过将分割生成与选择过程解耦，实现了对未见物体类别的类别无关部件级分割。通过使用 Pointer Net 生成器和独立的评估器对分割结果进行排序与选择，该方法在 COCO 全景分割数据集上取得了具有竞争力的性能，并在无需微调的情况下有效将新类别物体拆分为部件。

ABSTRACT

In machine learning and other fields, suggesting a good solution to a problem is usually a harder task than evaluating the quality of such a solution. This asymmetry is the basis for a large number of selection oriented methods that use a generator system to guess a set of solutions and an evaluator system to rank and select the best solutions. This work examines the use of this approach to the problem of panoptic image segmentation and class agnostic parts segmentation. The generator/evaluator approach for this case consists of two independent convolutional neural nets: a generator net that suggests variety segments corresponding to objects, stuff and parts regions in the image, and an evaluator net that chooses the best segments to be merged into the segmentation map. The result is a trial and error evolutionary approach in which a generator that guesses segments with low average accuracy, but with wide variability, can still produce good results when coupled with an accurate evaluator. The generator consists of a Pointer net that receives an image and a point in the image, and predicts the region of the segment containing the point. Generating and evaluating each segment separately is essential in this case since it demands exponentially fewer guesses compared to a system that guesses and evaluates the full segmentation map in each try. The classification of the selected segments is done by an independent region-specific classification net. This allows the segmentation to be class agnostic and hence, capable of segmenting unfamiliar categories that were not part of the training set. The method was examined on the COCO Panoptic segmentation benchmark and gave results comparable to those of the basic semantic segmentation and Mask-RCNN methods. In addition, the system was used for the task of splitting objects of unseen classes (that did not appear in the training set) into parts.

研究动机与目标

• 为解决全景分割中对不熟悉物体类别（尤其是部件级别）的分割挑战。
• 通过将生成与评估过程解耦，克服一次性生成完整分割图的困难。
• 通过类别无关的分割方法，实现在推理阶段对未见物体类别的零样本部件级分割。
• 通过利用可独立选择高质量分割结果的评估器，提升分割生成的鲁棒性与多样性。

提出的方法

• 基于 Pointer Net 的生成器网络接收图像和一个点作为输入，预测包含该点的分割掩码。
• 每个分割独立生成与评估，与完整图生成相比，搜索空间呈指数级减少。
• 评估器网络根据质量对生成的分割进行排序，选择最优候选分割纳入最终分割结果。
• 一个独立的区域特定分类网络为选定的分割分配语义标签，实现类别无关的推理。
• 该框架通过在推理阶段生成并评估未见物体类别的部件，支持零样本部件拆分。
• 该方法避免了完整分割图的端到端训练，而是采用带有评估器反馈的试错式进化策略。

实验结果

研究问题

• RQ1 生成器-评估器-选择框架是否能在减少归纳偏置的前提下，优于端到端的分割模型？
• RQ2 类别无关的分割系统在推理阶段对未见物体类别的泛化能力如何？
• RQ3 独立的分割生成与评估在多大程度上提升了分割输出的鲁棒性与多样性？
• RQ4 该框架是否能在无需微调的情况下，有效将新颖未见物体类别拆分为有意义的部件？
• RQ5 解耦方法在 COCO 全景分割等标准基准上的性能是否保持竞争力？

主要发现

• 该方法在 COCO 全景分割基准上的性能与最先进语义分割和 Mask R-CNN 方法相当。
• 系统成功地对未见类别的物体进行分割与拆分，展示了零样本泛化能力。
• 使用独立评估器即使在生成器输出低精度但多样化预测时，也能实现高质量分割的选择。
• 独立生成与评估分割显著减少了与完整分割图搜索相比所需的猜测次数。
• 尽管是类别无关的，该框架仍保持强大性能，支持在训练中未见过的类别进行推理。
• 该方法展示了在复杂分割任务中，基于进化思想的试错策略的有效性。