[Paper Review] MotionMixer: MLP-based 3D Human Body Pose Forecasting

Arij Bouazizi, Adrian Holzbock+3 more|arXiv (Cornell University)|Jul 1, 2022

Human Pose and Action Recognition4 citations

TL;DR

MotionMixer proposes a novel 3D human body pose forecasting model based solely on multi-layer perceptrons (MLPs), leveraging spatial-temporal mixing to capture joint and temporal dependencies. It achieves state-of-the-art performance on Human3.6M, AMASS, and 3DPW with significantly fewer parameters than prior methods, outperforming SOTA models by up to 4% in error reduction while using only 0.5% of the parameters of the next best model.

ABSTRACT

In this work, we present MotionMixer, an efficient 3D human body pose forecasting model based solely on multi-layer perceptrons (MLPs). MotionMixer learns the spatial-temporal 3D body pose dependencies by sequentially mixing both modalities. Given a stacked sequence of 3D body poses, a spatial-MLP extracts fine grained spatial dependencies of the body joints. The interaction of the body joints over time is then modelled by a temporal MLP. The spatial-temporal mixed features are finally aggregated and decoded to obtain the future motion. To calibrate the influence of each time step in the pose sequence, we make use of squeeze-and-excitation (SE) blocks. We evaluate our approach on Human3.6M, AMASS, and 3DPW datasets using the standard evaluation protocols. For all evaluations, we demonstrate state-of-the-art performance, while having a model with a smaller number of parameters. Our code is available at: https://github.com/MotionMLP/MotionMixer

Motivation & Objective

• To develop a 3D human body pose forecasting model that eliminates reliance on RNNs, CNNs, or GCNs, using only MLPs.
• To model both spatial joint relationships and temporal motion dynamics efficiently through a unified MLP architecture.
• To reduce model complexity and computational cost while maintaining or improving prediction accuracy.
• To calibrate the importance of each time step in the input sequence using squeeze-and-excitation blocks for better long-term forecasting.
• To achieve state-of-the-art performance on standard benchmarks with minimal parameters.

Proposed method

• The model uses a two-stage MLP architecture: a spatial-MLP processes joint coordinates at each time step to extract fine-grained spatial dependencies.
• A temporal-MLP then models interactions across time steps to capture motion dynamics.
• Spatial and temporal features are mixed via a shared MLP-based mixing mechanism, enabling joint modeling of spatial and temporal dependencies.
• Squeeze-and-excitation (SE) blocks are applied to recalibrate the influence of each time step in the sequence, improving attention to critical poses.
• The model processes stacked 3D pose sequences and decodes future poses using a final MLP head.
• The architecture is trained end-to-end using mean squared error loss on 3D joint positions.

Experimental results

Research questions

• RQ1 Can a pure MLP-based architecture outperform existing RNN, CNN, and GCN-based models in 3D human pose forecasting?
• RQ2 How effective is spatial-temporal mixing via MLPs in capturing complex human motion dynamics?
• RQ3 To what extent do squeeze-and-excitation blocks improve long-term prediction accuracy by recalibrating temporal attention?
• RQ4 Can a lightweight MLP model achieve SOTA performance with significantly fewer parameters than existing methods?
• RQ5 How does the model generalize across diverse actions and datasets like Human3.6M, AMASS, and 3DPW?

Key findings

• MotionMixer achieves state-of-the-art performance on Human3.6M, AMASS, and 3DPW, with a 3.5mm average 3D error reduction compared to prior SOTA models.
• At 1000ms prediction horizon, MotionMixer achieves 71.6mm average 3D error on Human3.6M, outperforming the next best model by 4% in error reduction.
• The model uses only 30.2k parameters and 2.1M FLOPs to predict 25 frames, achieving 71.6mm error—outperforming a model with 57.5k parameters by 3.8mm.
• Removing either spatial or temporal mixing increases error by 4% and 6% respectively at 1000ms, confirming the necessity of joint modeling.
• Squeeze-and-excitation blocks reduce error by 2mm across all horizons, demonstrating their effectiveness in temporal recalibration.
• The model achieves a 5mm performance gain when using pose displacement representation, indicating improved generalization to unseen subjects and environments.