[Paper Review] Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization

Weiguang Zhao, Yuyao Yan+4 more|arXiv (Cornell University)|Jul 22, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes PBNet, a novel 3D point cloud instance segmentation framework that uses point-wise binarization to separate adjacent objects by classifying points into high-density (HPs) and low-density (LPs) groups. By clustering HPs first to resolve overlapping instances and then using neighbor voting to assign LPs, PBNet achieves superior performance, ranking first on the ScanNetV2 benchmark with an mAP of 54.3.

ABSTRACT

Instance segmentation on point clouds is crucially important for 3D scene understanding. Most SOTAs adopt distance clustering, which is typically effective but does not perform well in segmenting adjacent objects with the same semantic label (especially when they share neighboring points). Due to the uneven distribution of offset points, these existing methods can hardly cluster all instance points. To this end, we design a novel divide-and-conquer strategy named PBNet that binarizes each point and clusters them separately to segment instances. Our binary clustering divides offset instance points into two categories: high and low density points (HPs vs. LPs). Adjacent objects can be clearly separated by removing LPs, and then be completed and refined by assigning LPs via a neighbor voting method. To suppress potential over-segmentation, we propose to construct local scenes with the weight mask for each instance. As a plug-in, the proposed binary clustering can replace traditional distance clustering and lead to consistent performance gains on many mainstream baselines. A series of experiments on ScanNetV2 and S3DIS datasets indicate the superiority of our model. In particular, PBNet ranks first on the ScanNetV2 official benchmark challenge, achieving the highest mAP. Code will be available publicly at https://github.com/weiguangzhao/PBNet.

Motivation & Objective

• Address the challenge of segmenting adjacent 3D objects with the same semantic label that share neighboring points, a limitation of traditional distance clustering.
• Overcome incomplete instance clustering caused by uneven offset point distribution in existing methods.
• Develop a plug-in compatible binary clustering strategy that enhances performance across multiple mainstream baselines without architectural overhaul.
• Suppress over-segmentation in large objects like sofas and tables through a local scene mechanism with weight masks.
• Achieve state-of-the-art performance on benchmark datasets with minimal hyperparameter sensitivity.

Proposed method

• Binarize each point in the offset branch into high-density (HP) or low-density (LP) points based on local point density to separate adjacent instances.
• Cluster HPs first using a distance-based grouping strategy to resolve overlapping instances with the same semantic label.
• Reconstruct and refine incomplete instances by assigning LPs via a neighbor voting mechanism using features from neighboring HPs.
• Construct local scenes around each predicted instance by searching for surrounding instances to improve context awareness.
• Generate instance-specific weight masks using global features and local scene encoding to guide the network in focusing on primary objects.
• Combine global features and refined local features to predict final instance masks, with a mask loss to optimize segmentation quality.

Experimental results

Research questions

• RQ1 Can point-wise binarization into high- and low-density points improve the separation of adjacent 3D instances with the same semantic label?
• RQ2 Does neighbor voting on low-density points effectively complete and refine incomplete instance clusters caused by sparse or uneven offset points?
• RQ3 Can a local scene mechanism with weight masks suppress over-segmentation in large objects like sofas and tables?
• RQ4 To what extent can the proposed binary clustering strategy be used as a plug-in to improve performance on existing SOTA baselines?
• RQ5 How sensitive is the method to hyperparameters, and can it maintain robustness with minimal tuning?

Key findings

• PBNet achieves the highest mAP of 54.3 on the ScanNetV2 official benchmark, ranking first among all methods evaluated.
• The ablation study shows that the proposed mask loss increases mAP by 7.7% and AP50 by 3.2% compared to the baseline.
• The neighbor voting mechanism for LPs improves mAP by 2.6 percentage points compared to the baseline without voting.
• The local scene mechanism with weight masks boosts mAP by 4.3 percentage points over the baseline without refinement.
• The method introduces only three hyperparameters (rd, dθ, K), which are empirically stable and less sensitive than those in prior SOTA methods.
• Despite a 150–250ms latency increase over HAIS, PBNet achieves a significant mAP gain, making the trade-off favorable for accuracy-critical applications.