[Paper Review] R2-MLP: Round-Roll MLP for Multi-View 3D Object Recognition

Shuo Chen, Tan Yu+1 more|arXiv (Cornell University)|Nov 20, 2022

Brain Tumor Detection and Classification4 citations

TL;DR

This paper proposes R2-MLP, a novel MLP-based architecture for multi-view 3D object recognition that introduces a round-roll (R²) module to enable cross-view feature communication by shifting channels along the view dimension. By combining spatial-shift operations within views and round-roll shifts across views, R2-MLP achieves state-of-the-art performance on ModelNet10 (99.6% accuracy) and ModelNet40 (97.7% accuracy) with lower FLOPs and faster inference than CNN-based models.

ABSTRACT

Recently, vision architectures based exclusively on multi-layer perceptrons (MLPs) have gained much attention in the computer vision community. MLP-like models achieve competitive performance on a single 2D image classification with less inductive bias without hand-crafted convolution layers. In this work, we explore the effectiveness of MLP-based architecture for the view-based 3D object recognition task. We present an MLP-based architecture termed as Round-Roll MLP (R$^2$-MLP). It extends the spatial-shift MLP backbone by considering the communications between patches from different views. R$^2$-MLP rolls part of the channels along the view dimension and promotes information exchange between neighboring views. We benchmark MLP results on ModelNet10 and ModelNet40 datasets with ablations in various aspects. The experimental results show that, with a conceptually simple structure, our R$^2$-MLP achieves competitive performance compared with existing state-of-the-art methods.

Motivation & Objective

• To address the limitation of existing MLP-based models in capturing cross-view dependencies during multi-view 3D object recognition.
• To design an efficient, parameter-free mechanism for inter-view feature communication without relying on attention or convolution.
• To achieve state-of-the-art performance on standard benchmarks (ModelNet10 and ModelNet40) using a pure MLP architecture.

Proposed method

• The R2-MLP architecture employs a round-roll (R²) module that shifts a portion of the feature channels along the view dimension in a cyclic manner, enabling information exchange between adjacent views.
• Within each view, spatial-shift operations are applied across the width and height dimensions to facilitate intra-view feature communication.
• The R² module integrates both intra-view and inter-view feature interactions in a single, computation-free component, avoiding additional parameters or FLOPs.
• The model uses a standard MLP backbone with learnable linear projections and positional encoding to process multi-view features.
• The round-roll operation is implemented as a cyclic shift: for example, view 1 → view 2 → view 3 → view 1, allowing neighboring views to share features.
• The architecture is trained end-to-end for 3D classification and evaluated on both classification and retrieval tasks using standard benchmarks.

Experimental results

Research questions

• RQ1 Can a pure MLP architecture effectively model cross-view dependencies in multi-view 3D object recognition?
• RQ2 How does a round-roll feature shifting mechanism compare to attention or convolution in enabling inter-view communication?
• RQ3 Can a computation-free, parameter-free operation like round-rolling achieve competitive performance while reducing FLOPs and inference time?
• RQ4 Does the proposed R2-MLP outperform existing CNN-based and attention-based methods on standard 3D recognition benchmarks?
• RQ5 Can the R2-MLP generalize well to 3D shape retrieval tasks without direct retrieval loss optimization?

Key findings

• On ModelNet10 with 20 views, R2-MLP achieves 99.6% top-1 accuracy, surpassing all compared methods including CAR-Net.
• On ModelNet40 with 20 views, R2-MLP achieves 97.7% accuracy, matching the performance of the state-of-the-art CAR-Net while using fewer parameters and less computation.
• With only 12 views, R2-MLP achieves 97.2% accuracy on ModelNet40, ranking second among all methods, demonstrating strong generalization with fewer views.
• On the 3D shape retrieval task of ModelNet40, R2-MLP achieves 93.1% mAP with 12 views and 93.1% with 20 views, outperforming prior state-of-the-art methods like MLVCNN (92.2% mAP).
• R2-MLP achieves inference speed of 0.07s per sample on ModelNet40 using a V100 GPU, significantly faster than CAR-Net (0.20s) and other CNN-based models.
• The R2-MLP model has lower FLOPs and parameter count than CNN-based counterparts like CAR-Net, demonstrating superior efficiency.