[Paper Review] MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control

Vittorio Caggiano, Huawei Wang+3 more|arXiv (Cornell University)|May 26, 2022

Muscle activation and electromyography studies38 citations

TL;DR

MyoSuite presents physiologically accurate musculoskeletal models in MuJoCo with 204 contact-rich tasks, enabling non-stationary motor learning and baselines for dexterous control.

ABSTRACT

Embodied agents in continuous control domains have had limited exposure to tasks allowing to explore musculoskeletal properties that enable agile and nimble behaviors in biological beings. The sophistication behind neuro-musculoskeletal control can pose new challenges for the motor learning community. At the same time, agents solving complex neural control problems allow impact in fields such as neuro-rehabilitation, as well as collaborative-robotics. Human biomechanics underlies complex multi-joint-multi-actuator musculoskeletal systems. The sensory-motor system relies on a range of sensory-contact rich and proprioceptive inputs that define and condition muscle actuation required to exhibit intelligent behaviors in the physical world. Current frameworks for musculoskeletal control do not support physiological sophistication of the musculoskeletal systems along with physical world interaction capabilities. In addition, they are neither embedded in complex and skillful motor tasks nor are computationally effective and scalable to study large-scale learning paradigms. Here, we present MyoSuite -- a suite of physiologically accurate biomechanical models of elbow, wrist, and hand, with physical contact capabilities, which allow learning of complex and skillful contact-rich real-world tasks. We provide diverse motor-control challenges: from simple postural control to skilled hand-object interactions such as turning a key, twirling a pen, rotating two balls in one hand, etc. By supporting physiological alterations in musculoskeletal geometry (tendon transfer), assistive devices (exoskeleton assistance), and muscle contraction dynamics (muscle fatigue, sarcopenia), we present real-life tasks with temporal changes, thereby exposing realistic non-stationary conditions in our tasks which most continuous control benchmarks lack.

Motivation & Objective

• Motivate embodied AI to study musculoskeletal motor control with realistic physiology and real-world-like tasks.
• Provide a fast, automated pipeline to convert OpenSim models into MuJoCo-compatible models.
• Develop a family of dexterous hand, elbow, and finger tasks with varying difficulty and non-stationarity.
• Introduce non-stationary factors (sarcopenia, fatigue, tendon transfer, exoskeleton) to study adaptation.
• Offer baselines and insights into learning control policies for complex musculoskeletal systems.

Proposed method

• Introduce a model-agnostic pipeline (MyoSim) to convert OpenSim musculoskeletal models into equivalent MuJoCo models with geometry transfer, moment arm optimization, and muscle force optimization.
• Create a library of physiologically accurate MuJoCo models: MyoFinger (4 DoF, 5 muscles), MyoElbow (1 DoF, 6 muscles), and MyoHand (29 bones, 23 joints, 39 muscles).
• Design 9 task families spanning simple to complex dexterous manipulation (e.g., finger pose, finger tip reach, key turn, pen twirl, baoding balls) with two difficulty levels and various resets to yield 204 tasks.
• Incorporate realistic non-stationarities (sarcopenia, fatigue, tendon transfer, exoskeleton assistance) to create non-stationary task conditions.
• Provide baseline RL results using Natural Policy Gradient and analyze policy behavior, sample efficiency, and qualitative movements.

Experimental results

Research questions

• RQ1 How can OpenSim musculoskeletal models be efficiently converted into fast, contact-rich MuJoCo models while preserving anatomical fidelity?
• RQ2 What is the performance of learning-based controllers on a suite of physiologically realistic dexterous tasks under varying difficulty and non-stationary conditions?
• RQ3 How do non-stationary effects like sarcopenia, fatigue, tendon transfer, and exoskeleton assistance impact control strategies and muscle activations?
• RQ4 Can policies adapt to intrinsic non-stationarities and recover control after surgical or injury-like perturbations?
• RQ5 What insights emerge about motor control and muscle coordination from learning in high-dimensional musculoskeletal systems?

Key findings

• MuJoCo models achieved close anatomical and dynamic alignment with OpenSim, with muscle moment-arm RMS differences around 0.04–0.38% and force RMS differences around 2.2–4.1% of Fmax.
• Forward simulations were orders of magnitude faster in MuJoCo (60x to 4000x faster) than OpenSim, enabling scalable experimentation.
• Baseline RL with Natural Policy Gradient solved several tasks, including finger and hand coordination tasks, though more complex tasks like Baoding balls required substantially more samples (e.g., >70M for some seeds).
• Non-stationary perturbations reveal compensatory muscle activations and cooperative strategies, such as auxiliary muscles supporting fatigued or weakened primary muscles.
• Tendon transfer experiments show re-learning is required after action-space remapping, highlighting adaptation needs after structural changes.
• Exoskeleton assistance reduces required muscle activation to reach target angles, illustrating potential efficiency gains in human-machine collaboration.