[Paper Review] PartAfford: Part-level Affordance Discovery from 3D Objects

Chao Xu, Yixin Chen+4 more|arXiv (Cornell University)|Feb 28, 2022

Robot Manipulation and Learning13 citations

TL;DR

This paper introduces PartAfford, a novel task for discovering part-level affordances in 3D objects using only sparse affordance labels and geometric regularization, without dense supervision. The method employs a slot-attention encoder for unsupervised part abstraction and a multi-branch decoder for reconstruction, affordance prediction, and cuboidal primitive regularization, achieving strong generalization on cross-category and challenging examples with a newly constructed dataset of 25,000+ 3D objects annotated with 24 affordance categories.

ABSTRACT

Understanding what objects could furnish for humans-namely, learning object affordance-is the crux to bridge perception and action. In the vision community, prior work primarily focuses on learning object affordance with dense (e.g., at a per-pixel level) supervision. In stark contrast, we humans learn the object affordance without dense labels. As such, the fundamental question to devise a computational model is: What is the natural way to learn the object affordance from visual appearance and geometry with humanlike sparse supervision? In this work, we present a new task of part-level affordance discovery (PartAfford): Given only the affordance labels per object, the machine is tasked to (i) decompose 3D shapes into parts and (ii) discover how each part of the object corresponds to a certain affordance category. We propose a novel learning framework for PartAfford, which discovers part-level representations by leveraging only the affordance set supervision and geometric primitive regularization, without dense supervision. The proposed approach consists of two main components: (i) an abstraction encoder with slot attention for unsupervised clustering and abstraction, and (ii) an affordance decoder with branches for part reconstruction, affordance prediction, and cuboidal primitive regularization. To learn and evaluate PartAfford, we construct a part-level, cross-category 3D object affordance dataset, annotated with 24 affordance categories shared among >25, 000 objects. We demonstrate that our method enables both the abstraction of 3D objects and part-level affordance discovery, with generalizability to difficult and cross-category examples. Further ablations reveal the contribution of each component.

Motivation & Objective

• To address the gap in visual affordance learning by proposing a humanlike, sparse-supervision paradigm that mimics how humans learn affordances without dense annotations.
• To formulate a new task—PartAfford—where the model discovers part-level affordances from only object-level affordance labels and 3D geometry.
• To develop a self-supervised framework that jointly performs 3D part abstraction and affordance discovery using only set-level supervision and geometric priors.
• To construct a large-scale, part-level, cross-category 3D affordance dataset with 24 affordance categories across 25,000+ objects for benchmarking.
• To evaluate the model's ability to generalize to hard and cross-category examples, demonstrating robustness and interpretability in part-level affordance understanding.

Proposed method

• The method uses a slot-attention-based encoder to abstract 3D object features into a set of disentangled slot variables, enabling unsupervised clustering and part abstraction without ground-truth part annotations.
• A multi-branch decoder jointly predicts affordance labels, reconstructs 3D parts and the full object, and enforces geometric plausibility via cuboidal primitive regularization.
• The framework relies solely on affordance set supervision and geometric constraints, avoiding per-part or per-pixel dense annotations.
• The model learns correspondence between affordance labels and abstracted 3D parts through end-to-end training with contrastive supervision from diverse affordance compositions.
• The number of slots is set to the maximum number of affordance labels per object category to reduce ambiguity in slot-to-affordance matching.
• Data augmentation is applied to increase diversity in affordance compositions, improving contrastive learning and model generalization.

Experimental results

Research questions

• RQ1 Can a model discover meaningful 3D part-level affordances using only object-level affordance labels and geometric priors, without dense supervision?
• RQ2 How effective is the proposed method in generalizing to unseen, cross-category, or hard examples where parts are ambiguous or shape variations are large?
• RQ3 What is the contribution of geometric primitive regularization and slot-attention abstraction in improving part discovery and affordance prediction?
• RQ4 How does data augmentation influence the model’s ability to learn disentangled affordance-part correspondences?
• RQ5 What are the limitations of the method in discovering affordances linked to tiny, complex, or functionally ambiguous parts (e.g., handles, illumination)?

Key findings

• The proposed method successfully discovers part-level affordances and reconstructs 3D parts from only object-level affordance labels and geometric constraints, demonstrating strong zero-shot generalization to cross-category examples.
• The model generalizes well to challenging objects such as microwaves (openable) and objects with fine-grained parts, even when reconstruction is imperfect.
• Ablation studies confirm that both the slot-attention encoder and the multi-branch decoder with geometric regularization are essential for performance, with the latter significantly improving shape plausibility.
• Data augmentation greatly improves learning efficiency and model performance by increasing contrastive signal through diverse affordance compositions.
• Failure cases are primarily due to fine-grained shape reconstruction difficulties and violations of the cuboidal assumption, especially for small parts like handles.
• The model learns to assign affordances to parts even when the same affordance appears in multiple slots, though excessive slots can lead to 'null' slot assignments, indicating attention to ambiguity.