[Paper Review] Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylization

Makito Kobayashi, Chen-Chieh Liao+3 more|arXiv (Cornell University)|Jun 15, 2023

Human Motion and Animation4 citations

TL;DR

This paper introduces a large-scale, industry-ready motion capture dataset featuring diverse motion styles—such as active, exhausted, and emotive expressions—using professional actors and high-fidelity optical motion capture. The dataset, built on an industrial-standard skeletal hierarchy, enables effective motion style transfer and interpolation via state-of-the-art neural networks, with validated results across multiple 3D characters and motion editing tasks in Unreal Engine 5.

ABSTRACT

In this work, we proposed a new style-diverse dataset for the domain of motion style transfer. The motion dataset uses an industrial-standard human bone structure and thus is industry-ready to be plugged into 3D characters for many projects. We claim the challenges in motion style transfer and encourage future work in this domain by releasing the proposed motion dataset both to the public and the market. We conduct a comprehensive study on motion style transfer in the experiment using the state-of-the-art method, and the results show the proposed dataset's validity for the motion style transfer task.

Motivation & Objective

• To address the scarcity of diverse, high-quality motion datasets with varied emotional and personality-based styles for AI-based motion editing.
• To enable practical, plug-and-play integration of motion data into 3D animation and game development pipelines.
• To validate the suitability of the dataset for motion style transfer and interpolation using state-of-the-art neural network methods.
• To release a public dataset for research and a commercial-grade emote-focused dataset for industry use.
• To support the development of universal motion style transfer models through a standardized, high-fidelity dataset.

Proposed method

• Captured motion data using a high-precision optical motion capture system with professional actors performing diverse movements.
• Structured motion data using an industry-standard human skeletal hierarchy (e.g., BVH-like structure) for direct compatibility with 3D animation tools.
• Collected content motions (e.g., walking, running) and style motions (e.g., active, feminine, youthful) to enable style transfer between distinct motion types.
• Applied neural network-based motion-to-motion translation models to perform style transfer, interpolation, and puzzle-style motion blending.
• Used Unreal Engine 5 to demonstrate motion retargeting and stylization across multiple 3D character models, including Mirai Komachi, Unreal Mannequin, and Mixamo.
• Cleaned and preprocessed raw motion data to ensure high fidelity and usability in real-world applications.

Figure 1 . Motions including in our dataset applied to Mirai Komachi character model.

Experimental results

Research questions

• RQ1 Can a large-scale, style-diverse motion dataset improve the performance and generalization of neural network-based motion style transfer?
• RQ2 To what extent can motion style transfer preserve content while transferring stylistic attributes like emotion or energy level?
• RQ3 How well does the proposed dataset support motion interpolation and novel style blending across different body parts?
• RQ4 Can the dataset be effectively used across diverse 3D character models without retraining or major retargeting?
• RQ5 What is the feasibility of creating a universal motion style transfer model using this dataset as a foundation?

Key findings

• The proposed motion dataset successfully enables high-quality motion style transfer, preserving content while accurately reflecting target style attributes such as energy level and emotional expression.
• Motion interpolation across multiple styles (e.g., active–normal–exhausted) produced smooth, realistic intermediate motions not present in the original dataset.
• Puzzle-style motion blending, where different styles are applied to different body parts, generated plausible and expressive hybrid motions.
• The dataset demonstrated strong compatibility with 3D animation pipelines, as shown by successful motion retargeting to Mirai Komachi, Unreal Mannequin, and Mixamo characters in Unreal Engine 5.
• The dataset’s industrial-standard skeletal structure allows direct use in commercial and research applications without additional preprocessing.
• Comprehensive experiments using state-of-the-art style transfer models confirmed the dataset’s validity and suitability for motion stylization tasks.

(a) walk