[Paper Review] Joint Pose and Shape Estimation of Vehicles from LiDAR Data

Hunter Goforth, Xiaoyan Hu+2 more|arXiv (Cornell University)|Sep 8, 2020

Robot Manipulation and Learning38 references4 citations

TL;DR

This paper proposes a joint learning framework for estimating vehicle pose and shape from sparse LiDAR scans using a shared encoder-decoder architecture. By training pose and shape prediction jointly with a novel combined loss, the method achieves superior accuracy and efficiency over independent estimation, generalizing well to real-world Argoverse data despite being trained only on synthetic data.

ABSTRACT

We address the problem of estimating the pose and shape of vehicles from LiDAR scans, a common problem faced by the autonomous vehicle community. Recent work has tended to address pose and shape estimation separately in isolation, despite the inherent connection between the two. We investigate a method of jointly estimating shape and pose where a single encoding is learned from which shape and pose may be decoded in an efficient yet effective manner. We additionally introduce a novel joint pose and shape loss, and show that this joint training method produces better results than independently-trained pose and shape estimators. We evaluate our method on both synthetic data and real-world data, and show superior performance against a state-of-the-art baseline.

Motivation & Objective

• To address the limitation of prior work that treats pose and shape estimation separately, despite their strong interdependence.
• To develop a unified deep learning framework that shares representation between pose and shape prediction for improved performance.
• To train the model exclusively on synthetic LiDAR data while maintaining strong generalization to real-world scenarios.
• To reduce model redundancy and improve inference efficiency by sharing a single encoder for both tasks.
• To demonstrate that joint training with a combined loss yields better results than independent training of pose and shape heads.

Proposed method

• A shared encoder processes raw, unaligned LiDAR point clouds into a single latent representation, reducing redundancy compared to separate encodings.
• The shape decoder reconstructs a complete, uniformly sampled 3D point cloud of the vehicle from the shared code, based on the PCN architecture.
• The pose decoder is a multi-layer perceptron (1024, 512, 512, 3) that regresses 2D translation (x, y) and heading angle θ from the shared code.
• A novel joint loss function combines shape completion loss (e.g., Chamfer Distance) and pose regression loss to optimize both tasks simultaneously.
• The network is trained in two stages: first, the encoder and shape decoder are pre-trained on synthetic data; then, the pose decoder is fine-tuned with a frozen encoder before end-to-end joint training.
• Synthetic data is generated using high-resolution CAD models and simulated LiDAR sensors to create paired partial and complete scans for supervision.

Experimental results

Research questions

• RQ1 Can joint learning of pose and shape from partial LiDAR scans improve performance over independent estimation?
• RQ2 Does sharing a single encoder between pose and shape heads reduce redundancy and improve stability?
• RQ3 Can a model trained only on synthetic LiDAR data generalize effectively to real-world autonomous vehicle data?
• RQ4 How does the proposed joint loss compare to independent training in terms of accuracy and robustness?
• RQ5 To what extent does the method maintain consistent estimation under varying occlusion and point density in real-world conditions?

Key findings

• The jointly trained shared-encoder model outperforms independently trained pose and shape estimators on both synthetic and real-world data.
• The method achieves state-of-the-art performance on the Argoverse dataset, with shape estimates closer to ground truth in 12 out of 15 real-world vehicle tracks.
• Pose estimation remains accurate even without RGB input, with heading angle prediction performing well across diverse real-world conditions.
• The model generalizes robustly to real-world data despite being trained only on synthetic data, demonstrating strong zero-shot generalization.
• The architecture maintains consistent shape and pose predictions across large temporal gaps with significant occlusion and variable point density, as shown in qualitative results.
• The joint training regime with the combined loss leads to improved performance over stage-wise pre-training, indicating benefit from end-to-end optimization.