[论文解读] DeepTransition: Viability Leads to the Emergence of Gait Transitions in Learning Anticipatory Quadrupedal Locomotion Skills

Milad Shafiee, Guillaume Bellegarda+1 more|arXiv (Cornell University)|Jun 12, 2023

Robotic Locomotion and Control被引用 6

一句话总结

该论文提出，生存性——定义为防止跌倒——是四足运动中步态转换的主要驱动力，通过深度强化学习训练机器人控制器，使其在平坦地形和缝隙地形上自主实现如走-小跑和小跑-跳跃的步态转换。关键结果表明，步态转换后生存性始终得到改善，优于能量效率和峰值受力降低，且控制器使Unitree A1机器人以超过1.3 m/s的速度成功跨越30 cm缝隙（相当于体长的83.3%），展现出最先进的敏捷性。

ABSTRACT

Quadruped animals seamlessly transition between gaits as they change locomotion speeds. While the most widely accepted explanation for gait transitions is energy efficiency, there is no clear consensus on the determining factor, nor on the potential effects from terrain properties. In this article, we propose that viability, i.e. the avoidance of falls, represents an important criterion for gait transitions. We investigate the emergence of gait transitions through the interaction between supraspinal drive (brain), the central pattern generator in the spinal cord, the body, and exteroceptive sensing by leveraging deep reinforcement learning and robotics tools. Consistent with quadruped animal data, we show that the walk-trot gait transition for quadruped robots on flat terrain improves both viability and energy efficiency. Furthermore, we investigate the effects of discrete terrain (i.e. crossing successive gaps) on imposing gait transitions, and find the emergence of trot-pronk transitions to avoid non-viable states. Compared with other potential criteria such as peak forces and energy efficiency, viability is the only improved factor after gait transitions on both flat and discrete gap terrains, suggesting that viability could be a primary and universal objective of gait transitions, while other criteria are secondary objectives and/or a consequence of viability. Moreover, we deploy our learned controller in sim-to-real hardware experiments and demonstrate state-of-the-art quadruped agility in challenging scenarios, where the Unitree A1 quadruped autonomously transitions gaits between trot and pronk to cross consecutive gaps of up to 30 cm (83.3 % of the body-length) at over 1.3 m/s.

研究动机与目标

• 探究生存性——定义为防止跌倒——是否为四足运动中步态转换的主要驱动力。
• 探讨脑干控制、脊髓中枢模式生成器（CPGs）与本体感觉感知之间的相互作用，以实现自适应步态转换。
• 评估在平坦地形和离散地形上，步态转换是否主要改善生存性、能量效率或峰值受力。
• 展示在具有挑战性的缝隙跨越任务中，所学运动策略在仿真到现实之间的鲁棒迁移能力。

提出的方法

• 采用分层控制架构，其中多层感知器代表脑干驱动，非线性振幅调制相位振荡器模拟脊髓CPGs，逆运动学将输出映射为电机指令。
• 在Isaac Gym和PyBullet仿真器中通过深度强化学习训练策略，利用外部视觉和地形高度感知信息来指导步态决策。
• 步态转换通过学习自然涌现，无需对转换点进行显式奖励设计，而是依赖于前进距离和防跌倒的奖励。
• 在平坦地形和离散缝隙地形上进行评估，缝隙宽度随机分布在14–20 cm之间，接触面宽度为14 cm。
• 成功度量标准为缝隙跨越率和速度，跌倒定义为身体高度<15 cm。
• 通过在Unitree A1机器人上直接部署策略，实现仿真到现实的迁移，无需领域随机化或微调。

Figure 1: To model locomotion control, we consider three main interacting layers: the brain (higher centers), the spinal cord, and the body and sensory feedback modules Higher neural centers (such as the brainstem, basal ganglia, cerebellum, and motor cortex) send descending drive signals to modulat

实验结果

研究问题

• RQ1 在平坦地形和离散地形上，步态转换后生存性是否得到改善？其与能量效率和峰值受力相比是否为最主要因素？
• RQ2 脑干驱动、脊髓CPGs与外部感觉反馈如何协同作用，以实现自适应步态转换？
• RQ3 学习到的控制器能否在无显式转换奖励设计的情况下，自主实现如小跑到跳跃等步态的转换，以跨越连续缝隙？
• RQ4 在复杂地形中，步态转换的出现在多大程度上与稳定性提升和防跌倒能力增强相关？

主要发现

• 在平坦地形和缝隙地形上，步态转换后生存性始终得到改善，而能量效率和峰值受力降低并未表现出一致改善。
• 在平坦地形上，走-小跑转换同时提升了生存性和能量效率，但在缝隙地形上，只有生存性是持续提升的指标。
• 在缝隙跨越任务中，机器人自主采用小跑-跳跃步态以避免非生存状态，表明步态转换的出现是为了维持稳定性。
• Unitree A1机器人在仿真到现实的部署中，成功以超过1.3 m/s的速度跨越长达30 cm（体长的83.3%）的连续缝隙。
• 该策略在不同机器人（Unitree A1和Go1）和多种地形类型（如踏脚石、网格地形和混合地形）上均表现出良好泛化能力。
• 由于未使用显式转换奖励或振荡器耦合，步态转换通过生存性学习自然涌现。

Figure 2: Qualitative comparison data for robot and animal locomotion. A: The CoT and CV of stride duration of the animals and the robot are plotted against the locomotion speed of walk-trot gaits in: a) the quadruped robot, b) the domestic dog, c) the Australian water rat, d) the Virginia opossum a