[Paper Review] GPA-Net:No-Reference Point Cloud Quality Assessment with Multi-task Graph Convolutional Network

Ziyu Shan, Qi Yang+5 more|arXiv (Cornell University)|Oct 29, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes GPA-Net, a no-reference point cloud quality assessment method using a multi-task graph convolutional network with a novel graph convolution kernel (GPAConv) that captures structural and textural perturbations. It integrates distortion type and degree prediction as auxiliary tasks and employs a coordinate normalization module to achieve shift, scale, and rotation invariance, outperforming state-of-the-art NR-PCQA metrics on two independent datasets.

ABSTRACT

With the rapid development of 3D vision, point cloud has become an increasingly popular 3D visual media content. Due to the irregular structure, point cloud has posed novel challenges to the related research, such as compression, transmission, rendering and quality assessment. In these latest researches, point cloud quality assessment (PCQA) has attracted wide attention due to its significant role in guiding practical applications, especially in many cases where the reference point cloud is unavailable. However, current no-reference metrics which based on prevalent deep neural network have apparent disadvantages. For example, to adapt to the irregular structure of point cloud, they require preprocessing such as voxelization and projection that introduce extra distortions, and the applied grid-kernel networks, such as Convolutional Neural Networks, fail to extract effective distortion-related features. Besides, they rarely consider the various distortion patterns and the philosophy that PCQA should exhibit shifting, scaling, and rotational invariance. In this paper, we propose a novel no-reference PCQA metric named the Graph convolutional PCQA network (GPA-Net). To extract effective features for PCQA, we propose a new graph convolution kernel, i.e., GPAConv, which attentively captures the perturbation of structure and texture. Then, we propose the multi-task framework consisting of one main task (quality regression) and two auxiliary tasks (distortion type and degree predictions). Finally, we propose a coordinate normalization module to stabilize the results of GPAConv under shift, scale and rotation transformations. Experimental results on two independent databases show that GPA-Net achieves the best performance compared to the state-of-the-art no-reference PCQA metrics, even better than some full-reference metrics in some cases.

Motivation & Objective

• To address the limitations of existing no-reference point cloud quality assessment (NR-PCQA) methods that rely on preprocessing like voxelization or projection, which introduce distortions.
• To develop a feature extraction mechanism that effectively captures distortion-related features in irregular point cloud structures, overcoming the limitations of grid-based CNNs.
• To incorporate multi-task learning for joint prediction of quality, distortion type, and distortion degree, enhancing feature representation.
• To ensure shift, scale, and rotation invariance in quality assessment through a coordinate normalization module, aligning with human perception.
• To achieve superior performance on NR-PCQA benchmarks without requiring reference point clouds.

Proposed method

• Proposes GPAConv, a novel graph convolution kernel that computes both direction and length similarities between neighboring points to capture structural and textural perturbations.
• Designs a multi-task learning framework with one main task (quality regression) and two auxiliary tasks (distortion type and degree prediction) to improve feature learning.
• Introduces a coordinate normalization module that reorients local coordinate systems using the Local Reference Frame (LRF) to ensure invariance under rotation, scaling, and translation.
• Applies t-SNE visualization to demonstrate that GPA-Net’s feature space better separates point clouds with distinct content and distortions compared to baselines.
• Employs a patch-based graph construction strategy, where each patch is processed independently with a learned graph structure to preserve local geometry.
• Uses max-pooling over graph features followed by fully connected layers for final quality prediction.

Experimental results

Research questions

• RQ1 Can a graph convolutional network with a novel kernel design effectively extract distortion-relevant features from irregular point clouds without preprocessing?
• RQ2 Does multi-task learning with auxiliary tasks (distortion type and degree) improve the performance of no-reference PCQA?
• RQ3 Can a coordinate normalization module based on LRF achieve robustness to rotation, scaling, and translation in point cloud quality assessment?
• RQ4 How does GPA-Net compare to state-of-the-art NR-PCQA methods in terms of SROCC and PLCC on independent benchmarks?
• RQ5 To what extent does the model generalize to point clouds with diverse content and distortion patterns?

Key findings

• GPA-Net achieves the highest SROCC of 0.886 and PLCC of 0.886 on the SJTU-PCQA dataset, outperforming all state-of-the-art NR-PCQA metrics.
• On the TCG-PCQA dataset, GPA-Net achieves SROCC of 0.875 and PLCC of 0.886, demonstrating consistent superiority across independent benchmarks.
• When rotation invariance is removed, GPA-Net’s performance degrades significantly—SROCC drops to 0.837 under 180° rotation around the y-axis—highlighting the effectiveness of the normalization module.
• The model performs best when local z and x axes are reoriented to global y and z axes, with SROCC of 0.875 and PLCC of 0.886, indicating optimal axis alignment for feature stability.
• t-SNE visualization confirms that GPA-Net’s feature space better separates point clouds with different content (e.g., 'Redandblack' vs. 'Soldier') compared to other encoders.
• The optimal configuration for the first GPA layer uses 25,000 keypoints and 100 neighbors, achieving the best trade-off between coverage and overlap in patch construction.