[Paper Review] GaIA: Graphical Information Gain based Attention Network for Weakly Supervised Point Cloud Semantic Segmentation

Min Seok Lee, Seok Woo Yang+1 more|arXiv (Cornell University)|Oct 2, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes GaIA, a graphical information gain-based attention network for weakly supervised point cloud semantic segmentation that reduces epistemic uncertainty by leveraging relative entropy between target points and their neighborhoods to identify reliable features. It further introduces ArcPoint loss to optimize high-entropy unlabeled points by enforcing angular margin in hypersphere space, achieving state-of-the-art performance on S3DIS and ScanNet-v2 under sparse annotation.

ABSTRACT

While point cloud semantic segmentation is a significant task in 3D scene understanding, this task demands a time-consuming process of fully annotating labels. To address this problem, recent studies adopt a weakly supervised learning approach under the sparse annotation. Different from the existing studies, this study aims to reduce the epistemic uncertainty measured by the entropy for a precise semantic segmentation. We propose the graphical information gain based attention network called GaIA, which alleviates the entropy of each point based on the reliable information. The graphical information gain discriminates the reliable point by employing relative entropy between target point and its neighborhoods. We further introduce anchor-based additive angular margin loss, ArcPoint. The ArcPoint optimizes the unlabeled points containing high entropy towards semantically similar classes of the labeled points on hypersphere space. Experimental results on S3DIS and ScanNet-v2 datasets demonstrate our framework outperforms the existing weakly supervised methods. We have released GaIA at https://github.com/Karel911/GaIA.

Motivation & Objective

• To reduce epistemic uncertainty in weakly supervised point cloud semantic segmentation, which is critical for improving model reliability despite sparse annotations.
• To address the limitation of standard softmax loss in optimizing high-entropy unlabeled points by introducing a margin-based loss that prioritizes uncertain samples.
• To improve feature representation by propagating reliable information from low-entropy points to high-entropy neighbors using relative entropy-based graphical information gain.
• To achieve better semantic segmentation performance without full supervision by focusing on uncertainty reduction and targeted optimization of ambiguous points.

Proposed method

• Graphical information gain computes relative entropy between the entropy of a target point and its neighborhood to identify reliable points, which are used to update uncertain features.
• The method uses a Siamese network branch to compute entropy per point and applies a neighborhood-based entropy comparison to detect reliable points near decision boundaries.
• ArcPoint loss introduces an anchor-based additive angular margin to the softmax loss, explicitly optimizing high-entropy unlabeled points toward semantically similar embeddings of labeled points on the hypersphere.
• Feature representations of unlabeled points with high entropy are refined by propagating credible information from reliable neighbors, improving embedding quality.
• The framework jointly optimizes the segmentation head and feature encoder using cross-entropy loss with ArcPoint regularization, focusing on uncertainty reduction.
• Training is conducted under sparse annotation (e.g., 1% or 1 point per class), simulating realistic weak supervision.

Experimental results

Research questions

• RQ1 Can reducing epistemic uncertainty through relative entropy improve weakly supervised point cloud semantic segmentation?
• RQ2 How does graphical information gain help identify reliable points for uncertainty reduction in sparse annotation settings?
• RQ3 Does an angular margin-based loss like ArcPoint outperform standard softmax in optimizing high-entropy unlabeled points?
• RQ4 Is there a trade-off between reducing epistemic uncertainty and increasing false predictions with high reliability, and how does it vary across classes?
• RQ5 How does the proposed method compare quantitatively and qualitatively to existing weakly supervised baselines on S3DIS and ScanNet-v2?

Key findings

• GaIA achieved state-of-the-art performance on S3DIS and ScanNet-v2 under 1% and 1-point-per-class annotation, outperforming existing weakly supervised methods.
• The graphical information gain effectively reduced epistemic uncertainty by identifying reliable points through relative entropy, leading to improved feature representation.
• ArcPoint loss showed a short-tail distribution of angular distances, indicating better optimization of unlabeled points compared to standard softmax loss.
• Despite uncertainty reduction, a trade-off emerged: false predictions with high reliability increased under 1% annotation, particularly in classes like column, sofa, bookcase, and board.
• The number of false predictions with high reliability remained consistently high across training steps in specific classes, indicating persistent misclassification despite low uncertainty.
• Qualitative results on S3DIS showed GaIA’s predictions were closer to ground truth than the baseline, especially in complex regions like furniture and cluttered areas.