[논문 리뷰] Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects

Yang Xiao, Xuchong Qiu외 3명|arXiv (Cornell University)|2019. 06. 12.

Robot Manipulation and Learning참고 문헌 59인용 수 36

한 줄 요약

일반적인 범주에 구애받지 않는 포즈 추정 방법으로, 주어진 3D 모델에 대해 3D 객체의 포즈를 조건화하여 추가 학습 없이도 보지 못한 객체 카테고리에 대한 포즈 추정이 가능하다. 이 방법은 표준 벤치마크에서 성능을 향상시키고 새로운 객체 및 데이터셋에 대한 강한 일반화를 보여준다.

ABSTRACT

Most deep pose estimation methods need to be trained for specific object instances or categories. In this work we propose a completely generic deep pose estimation approach, which does not require the network to have been trained on relevant categories, nor objects in a category to have a canonical pose. We believe this is a crucial step to design robotic systems that can interact with new objects in the wild not belonging to a predefined category. Our main insight is to dynamically condition pose estimation with a representation of the 3D shape of the target object. More precisely, we train a Convolutional Neural Network that takes as input both a test image and a 3D model, and outputs the relative 3D pose of the object in the input image with respect to the 3D model. We demonstrate that our method boosts performances for supervised category pose estimation on standard benchmarks, namely Pascal3D+, ObjectNet3D and Pix3D, on which we provide results superior to the state of the art. More importantly, we show that our network trained on everyday man-made objects from ShapeNet generalizes without any additional training to completely new types of 3D objects by providing results on the LINEMOD dataset as well as on natural entities such as animals from ImageNet.

연구 동기 및 목표

• 사전 정의된 카테고리나 인스턴스 외부의 물체에 대해 강인한 포즈 추정을 촉진한다(현장에서의 물체들).
• 목표 물체의 3D 모델에 포즈 추정을 조건으로 하는 심층 네트워크를 제안한다.
• 형태 조건 포즈 추정이 알려진 카테고리의 정확도를 향상시키고 새로운 객체로 일반화함을 보여준다.
• 3D 형상을 나타내는 포인트 클라우드와 다중 시야 렌더링 모두 포즈 예측을 위한 형상 정보를 인코딩하는 데 사용될 수 있음을 보인다.

제안 방법

• 두 가지 분기 네트워크 처리: (1) RGB 이미지 through a CNN (ResNet-18) and (2) 3D shape through either PointNet or multi-view rendered images.
• A mixed classification-and-regression loss predicts Euler-angle bins and intra-bin offsets for azimuth, elevation, and in-plane rotation.
• Angles are discretized into L_theta bins with corresponding classification scores and regression offsets (Huber loss).
• Data augmentation includes shape-rotation perturbations to reduce overfitting to canonical orientations.
• Training uses Adam with staged learning rates; synthetic ShapeNet data used for training with SUN397 backgrounds; testing on Pascal3D+, ObjectNet3D, Pix3D, and LINEMOD.
• Shape encoders: (a) PointNet for point clouds; (b) multi-view CNN using rendered views around the object; weights shared across viewpoints.

실험 결과

연구 질문

• RQ1 Can a deep pose estimator learn category-free viewpoint estimation conditioned on a 3D object model?
• RQ2 Does incorporating exact or approximate 3D shape information boost pose estimation performance for known categories?
• RQ3 How well does the method generalize to novel categories and completely unseen object types?
• RQ4 What is the impact of using multi-view shape representations versus single-view or point-cloud encodings?

주요 결과

• Using 3D shape information (point cloud or multi-view renderings) significantly improves pose estimation over a shape-less baseline across datasets.
• Multi-view representations generally outperform point-cloud encodings for the shape input.
• The method achieves competitive or superior results on Pascal3D+, ObjectNet3D, and Pix3D, even when trained only on synthetic data.
• The approach provides meaningful coarse pose estimates on LINEMOD without object-specific training, enabling effective downstream refinement (e.g., DeepIM).
• Randomizing object shape orientation during training reduces overfitting to a canonical pose and improves robustness to unseen shapes.