[Paper Review] AnthroNet: Conditional Generation of Humans via Anthropometrics

Francesco Picetti, Shrinath Deshpande+15 more|arXiv (Cornell University)|Sep 7, 2023

Human Pose and Action RecognitionComputer Science3 citations

TL;DR

AnthroNet is a deep generative model that creates high-resolution 3D human body meshes conditioned on an extensive set of anthropometric measurements, trained end-to-end on 100,000 synthetically generated posed human meshes. It enables precise, bi-directional conversion between SMPL-X and AnthroNet meshes and provides a publicly available synthetic data generator for academic research.

ABSTRACT

We present a novel human body model formulated by an extensive set of anthropocentric measurements, which is capable of generating a wide range of human body shapes and poses. The proposed model enables direct modeling of specific human identities through a deep generative architecture, which can produce humans in any arbitrary pose. It is the first of its kind to have been trained end-to-end using only synthetically generated data, which not only provides highly accurate human mesh representations but also allows for precise anthropometry of the body. Moreover, using a highly diverse animation library, we articulated our synthetic humans' body and hands to maximize the diversity of the learnable priors for model training. Our model was trained on a dataset of $100k$ procedurally-generated posed human meshes and their corresponding anthropometric measurements. Our synthetic data generator can be used to generate millions of unique human identities and poses for non-commercial academic research purposes.

Motivation & Objective

• To develop a high-resolution, expressive human body model conditioned on a comprehensive set of objective anthropometric measurements.
• To eliminate reliance on real human scans and motion capture data by training entirely on synthetically generated data.
• To enable accurate, bidirectional registration between SMPL-X and AnthroNet meshes for interoperability with existing models.
• To provide a publicly accessible synthetic data generator for non-commercial academic use, promoting diversity and reproducibility.
• To support precise anthropometric measurement extraction from any 3D scan or body model via a learned regressor.

Proposed method

• The model is trained end-to-end on 100,000 procedurally generated, multi-subject, multi-pose 3D human meshes with associated anthropometric measurements.
• A deep generative architecture decodes a latent vector Z and encoded anthropometric measurements A into a bind pose mesh X̃b^A.
• A mesh skinner and poser block applies pose-specific corrective offsets ΔX̃θ^A to animate the mesh into any desired pose θ.
• The model uses a highly diverse animation library to maximize the diversity of learned priors during training.
• Two registration pipelines enable bi-directional conversion between SMPL-X and AnthroNet meshes, ensuring compatibility with existing models.
• An anthropometric measurement regressor is trained to extract measurements from real scans or other models, enabling universal measurement extraction.

Experimental results

Research questions

• RQ1 Can a deep generative model produce high-resolution 3D human body meshes conditioned solely on anthropometric measurements?
• RQ2 To what extent can synthetic data alone enable accurate and diverse human body modeling without real-world scans or motion capture?
• RQ3 How effective is the bidirectional registration between AnthroNet and SMPL-X in preserving shape and pose fidelity?
• RQ4 Can the model generalize to extract anthropometric measurements from arbitrary 3D scans or body models with high accuracy?
• RQ5 What impact does using a comprehensive set of anthropometric descriptors have on modeling human body shape diversity and fidelity?

Key findings

• AnthroNet generates 3D human body meshes with over three times the resolution of SMPL-X, achieving high fidelity to anthropometric inputs.
• The model is trained end-to-end on 100,000 synthetically generated, diverse, and pose-annotated human meshes, eliminating the need for real human data.
• The bidirectional registration pipeline enables accurate transformation between SMPL-X and AnthroNet meshes, facilitating integration with existing tools.
• The anthropometric measurement regressor enables precise extraction of 30+ measurements from any 3D scan or body model without mesh-specific calibration.
• The synthetic data generator is publicly available for non-commercial academic use, supporting reproducibility and diversity in future research.
• Performance evaluation on the HBW dataset shows that AnthroNet, when combined with a pretrained image regressor, achieves competitive reconstruction accuracy in millimeter-level error.