[Paper Review] Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric

Zicheng Zhang, Wei Sun+5 more|arXiv (Cornell University)|Aug 30, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes a no-reference point cloud quality assessment method that evaluates visual quality by analyzing moving camera videos of point clouds. By extracting spatial (static) and temporal (dynamic) quality-aware features using a ResNet50 and SlowFast R50 model, the method achieves state-of-the-art performance, with SRCC up to 0.8611 on SJTU-PCQA, rivaling full-reference methods and outperforming existing no-reference approaches.

ABSTRACT

Point cloud is one of the most widely used digital representation formats for three-dimensional (3D) contents, the visual quality of which may suffer from noise and geometric shift distortions during the production procedure as well as compression and downsampling distortions during the transmission process. To tackle the challenge of point cloud quality assessment (PCQA), many PCQA methods have been proposed to evaluate the visual quality levels of point clouds by assessing the rendered static 2D projections. Although such projection-based PCQA methods achieve competitive performance with the assistance of mature image quality assessment (IQA) methods, they neglect that the 3D model is also perceived in a dynamic viewing manner, where the viewpoint is continually changed according to the feedback of the rendering device. Therefore, in this paper, we evaluate the point clouds from moving camera videos and explore the way of dealing with PCQA tasks via using video quality assessment (VQA) methods. First, we generate the captured videos by rotating the camera around the point clouds through several circular pathways. Then we extract both spatial and temporal quality-aware features from the selected key frames and the video clips through using trainable 2D-CNN and pre-trained 3D-CNN models respectively. Finally, the visual quality of point clouds is represented by the video quality values. The experimental results reveal that the proposed method is effective for predicting the visual quality levels of the point clouds and even competitive with full-reference (FR) PCQA methods. The ablation studies further verify the rationality of the proposed framework and confirm the contributions made by the quality-aware features extracted via the dynamic viewing manner. The code is available at https://github.com/zzc-1998/VQA_PC.

Motivation & Objective

• To address the limitation of existing projection-based no-reference PCQA methods that rely only on static 2D views.
• To investigate whether dynamic viewing behavior—simulating moving cameras—improves point cloud quality assessment.
• To develop a no-reference PCQA framework that leverages video quality assessment (VQA) to model both spatial and temporal distortions in point clouds.
• To validate the effectiveness of using multiple camera pathways to capture diverse dynamic views for improved quality prediction.
• To ensure computational efficiency while maintaining high accuracy for practical deployment.

Proposed method

• The method generates video sequences by rotating a virtual camera along four distinct circular pathways around the point cloud.
• Spatial quality features are extracted from key frames using a trainable 2D-CNN (ResNet50) to capture static distortions like noise and geometric shifts.
• Temporal quality features are extracted from video clips using a pre-trained 3D-CNN (SlowFast R50) to model motion-related artifacts such as abrupt view transitions.
• The final quality score is derived from a fusion of spatial and temporal features, enabling the model to assess both static and dynamic visual quality.
• The framework is trained end-to-end using a regression head to predict subjective quality scores without requiring a pristine reference point cloud.
• The method is evaluated on three benchmark datasets: SJTU-PCQA, WPC, and LSPCQA-I, using SRCC and PLCC as primary metrics.

Figure 1: Illustration of human habits of observing point clouds via static and dynamic views. The static views focus on the single rendered images while the dynamic views focus on the frames (usually generated by rotation).

Experimental results

Research questions

• RQ1 Can modeling dynamic viewing behavior through moving camera videos improve no-reference point cloud quality assessment compared to static projection-based methods?
• RQ2 How do spatial and temporal features extracted from video sequences contribute to the overall quality prediction performance?
• RQ3 What is the relative contribution of different camera pathways in capturing diverse quality-related visual content?
• RQ4 How does the proposed method compare in performance and efficiency to existing no-reference and full-reference PCQA methods?
• RQ5 Can the proposed VQA-based framework achieve performance comparable to full-reference methods without access to the original point cloud?

Key findings

• The proposed method achieves an SRCC of 0.8611 and PLCC of 0.8702 on the SJTU-PCQA dataset, outperforming all existing no-reference PCQA methods.
• The method achieves competitive performance with full-reference PCQA methods, indicating that dynamic video-based assessment captures perceptual quality more effectively than static projections.
• Ablation studies confirm that both spatial and temporal features are essential, with the temporal branch significantly improving performance over static-only baselines.
• The model using all four camera pathways (P7) achieves the highest performance, and removing any single pathway leads to measurable degradation, confirming the value of multi-pathway video capture.
• Despite including video generation time, the proposed method maintains competitive computational efficiency, with an average inference time of 13.15 seconds per point cloud—faster than most model-based PCQA methods.
• The pathway θ_A contributes the most to performance, as shown by the lowest performance in the P6 model (which excludes θ_A), indicating pathway-specific sensitivity to distortion types.

Figure 3: The framework of the proposed method, which includes the video capture process, the feature extraction module, and the feature regression module.