[Paper Review] DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Yongcheng Liu, Bin Fan+4 more|arXiv (Cornell University)|Sep 9, 2019

3D Shape Modeling and AnalysisEngineering60 references32 citations

TL;DR

DensePoint proposes a permutation-invariant generalized convolution (PConv) and a densely connected architecture to learn multi-level, multi-scale contextual representations directly from irregular point clouds, achieving state-of-the-art results across classification, retrieval, segmentation, and normal estimation.

ABSTRACT

Point cloud processing is very challenging, as the diverse shapes formed by irregular points are often indistinguishable. A thorough grasp of the elusive shape requires sufficiently contextual semantic information, yet few works devote to this. Here we propose DensePoint, a general architecture to learn densely contextual representation for point cloud processing. Technically, it extends regular grid CNN to irregular point configuration by generalizing a convolution operator, which holds the permutation invariance of points, and achieves efficient inductive learning of local patterns. Architecturally, it finds inspiration from dense connection mode, to repeatedly aggregate multi-level and multi-scale semantics in a deep hierarchy. As a result, densely contextual information along with rich semantics, can be acquired by DensePoint in an organic manner, making it highly effective. Extensive experiments on challenging benchmarks across four tasks, as well as thorough model analysis, verify DensePoint achieves the state of the arts.

Motivation & Objective

• Motivate the need for sufficiently contextual semantic information to recognize complex 3D shapes formed by irregular points.
• Develop a generalized convolution operator that is permutation invariant and supports local connectivity and weight sharing on irregular point clouds.
• Propose DensePoint, a dense-connection architecture that aggregates multi-level, multi-scale context for richer representations.
• Enhance expressive power with ePConv (grouped filters) while keeping a narrow channel design for efficiency.
• Demonstrate state-of-the-art performance on shape classification, retrieval, part segmentation, and normal estimation.

Proposed method

• Introduce PConv, a generalized convolution on point clouds defined as f_N(x) = rho({phi(f_xn) for x_n in N(x)}), where phi is a shared transformation and rho is a symmetric aggregation.
• Define PPool to perform learnable pooling by downsampling points and applying PConv on local neighborhoods.
• Adopt a DensePoint architecture inspired by dense connections, where each layer receives inputs from all preceding layers and passes its output to all subsequent layers, enabling densely contextual representations.
• Introduce ePConv (enhanced PConv) with filter grouping to widen the effective capacity while keeping narrow output channels, using a grouped phi and a combining psi to integrate group outputs.
• Use a stage-based network with three PPools and multiple DensePoints for classification, and four DensePoints with feature propagation for per-point tasks.
• Experiment with design choices: narrowness k, aggregation function (max pooling recommended), group number Ng, and stages where DensePoint is applied.

Experimental results

Research questions

• RQ1 Can a convolution operator for point clouds be made permutation invariant and capable of efficient local pattern learning?
• RQ2 Does a densely connected, multi-level context learning framework improve shape understanding over traditional hierarchical point-cloud networks?
• RQ3 How do design choices like narrow channel width, grouped filtering, and stage placement affect performance and efficiency across tasks?
• RQ4 What performance gains can be achieved on classification, retrieval, segmentation, and normal estimation with DensePoint compared to prior methods?

Key findings

• DensePoint achieves state-of-the-art performance on four tasks: classification, shape retrieval, part segmentation, and normal estimation.
• On ModelNet40 classification with 1k points, DensePoint reaches 93.2% accuracy (outperforming PointNet++ and other point-based methods).
• DensePoint substantially improves retrieval results with mAP of 88.5% on ModelNet40 using 1k points.
• In ShapeNet part segmentation, DensePoint achieves 84.2% class mIoU and 86.4% instance mIoU, surpassing leading point-based methods.
• For normal estimation on ModelNet40, DensePoint attains an error of 0.149 (cosine loss), outperforming prior methods and showing notable gains over PointNet++ (0.29) and others.
• Ablation studies confirm the effectiveness of dense contextual learning, ePConv with filter grouping, and the max-pooling aggregation strategy, with DensePoint consistently outperforming baseline and concatenation variants.