[Paper Review] A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds

Xiaoyuan Luo, Shaolei Liu+3 more|arXiv (Cornell University)|Apr 12, 2021

Human Pose and Action Recognition32 references11 citations

TL;DR

This paper proposes a learnable self-supervised task for unsupervised domain adaptation (UDA) in point cloud processing, using a neural network to learn nonlinear transformations that destroy and reconstruct parts of point clouds. By sharing an encoder between a main classification task and this reconstruction-based self-supervision, the model learns transferable local features, achieving state-of-the-art performance on PointDA-10 and PointSegDA benchmarks.

ABSTRACT

Deep neural networks have achieved promising performance in supervised point cloud applications, but manual annotation is extremely expensive and time-consuming in supervised learning schemes. Unsupervised domain adaptation (UDA) addresses this problem by training a model with only labeled data in the source domain but making the model generalize well in the target domain. Existing studies show that self-supervised learning using both source and target domain data can help improve the adaptability of trained models, but they all rely on hand-crafted designs of the self-supervised tasks. In this paper, we propose a learnable self-supervised task and integrate it into a self-supervision-based point cloud UDA architecture. Specifically, we propose a learnable nonlinear transformation that transforms a part of a point cloud to generate abundant and complicated point clouds while retaining the original semantic information, and the proposed self-supervised task is to reconstruct the original point cloud from the transformed ones. In the UDA architecture, an encoder is shared between the networks for the self-supervised task and the main task of point cloud classification or segmentation, so that the encoder can be trained to extract features suitable for both the source and the target domain data. Experiments on PointDA-10 and PointSegDA datasets show that the proposed method achieves new state-of-the-art performance on both classification and segmentation tasks of point cloud UDA. Code will be made publicly available.

Motivation & Objective

• Address the high cost of manual annotation in supervised 3D point cloud learning by enabling domain generalization without target domain labels.
• Overcome the limitations of hand-crafted self-supervised tasks in existing UDA methods for point clouds.
• Improve feature transferability between source and target domains by learning a nonlinear transformation for destruction-reconstruction.
• Achieve state-of-the-art performance on point cloud UDA benchmarks using a shared encoder in a multitask learning framework.

Proposed method

• Propose a learnable nonlinear transformation using a deep neural network to destroy a part of a point cloud, preserving semantic content while generating complex variants.
• Design a destruction-reconstruction self-supervised task where the model learns to reconstruct the original point cloud from the transformed version.
• Integrate the self-supervised task into a multitask UDA framework with a shared encoder and separate heads for the main task (classification/segmentation) and auxiliary task (reconstruction).
• Apply multi-region transformation to encourage focus on local, more transferable features during training.
• Use pseudo-labels from the target domain to train the auxiliary network, enabling joint optimization on source and target data.
• Train the encoder via adversarial learning to produce robust, domain-invariant features suitable for both domains.

Experimental results

Research questions

• RQ1 Can a learnable, neural-based transformation outperform hand-crafted destruction methods in self-supervised UDA for point clouds?
• RQ2 Does integrating a learnable reconstruction task improve feature transferability across domain shifts in point cloud classification and segmentation?
• RQ3 How does the multi-region transformation strategy affect performance and local feature learning in UDA?
• RQ4 Can the learned transformation serve as an effective data augmentation strategy beyond domain adaptation?

Key findings

• The proposed method achieves a new state-of-the-art average classification accuracy of 66.5% on the PointDA-10 benchmark when using PointNet as the encoder.
• With DGCNN as the encoder, the method achieves a new SoTA average accuracy of 70.0% on PointDA-10.
• The learned transformation improves generalization, achieving 63.0% average accuracy as a data augmentation strategy, outperforming baseline and prior methods in most settings.
• The multi-region transformation strategy improves accuracy by 1.0% over single-region learning, demonstrating its effectiveness in focusing on local features.
• Even without domain adaptation, the self-supervised task alone achieves 63.5% average accuracy, surpassing baseline and prior methods like PointDAN and DefRec.
• The method outperforms existing SOTA approaches on both classification and segmentation tasks in the PointSegDA benchmark, confirming its broad applicability.