[论文解读] TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback

Zihao Li, Yanan Zhou+4 more|arXiv (Cornell University)|Feb 10, 2026

Teleoperation and Haptic Systems被引用 0

一句话总结

TriPilot-FF 引入带踏板的脚控与 LiDAR 驱动的力反馈，结合领导-跟随的双手遥操作，提升移动机器人对全身的控制，在可达性、避免碰撞和长时任务方面表现更好，并采用力矩增强的策略学习。

ABSTRACT

Mobile manipulators broaden the operational envelope for robot manipulation. However, the whole-body teleoperation of such robots remains a problem: operators must coordinate a wheeled base and two arms while reasoning about obstacles and contact. Existing interfaces are predominantly hand-centric (e.g., VR controllers and joysticks), leaving foot-operated channels underexplored for continuous base control. We present TriPilot-FF, an open-source whole-body teleoperation system for a custom bimanual mobile manipulator that introduces a foot-operated pedal with lidar-driven pedal haptics, coupled with upper-body bimanual leader-follower teleoperation. Using only a low-cost base-mounted lidar, TriPilot-FF renders a resistive pedal cue from proximity-to-obstacle signals in the commanded direction, shaping operator commands toward collision-averse behaviour without an explicit collision-avoidance controller. The system also supports arm-side force reflection for contact awareness and provides real-time force and visual guidance of bimanual manipulability to prompt mobile base repositioning, thereby improving reach. We demonstrate the capability of TriPilot-FF to effectively ``co-pilot'' the human operator over long time-horizons and tasks requiring precise mobile base movement and coordination. Finally, we incorporate teleoperation feedback signals into an Action Chunking with Transformers (ACT) policy and demonstrate improved performance when the additional information is available. We release the pedal device design, full software stack, and conduct extensive real-world evaluations on a bimanual wheeled platform. The project page of TriPilot-FF is http://bit.ly/46H3ZJT.

研究动机与目标

• 为双手移动机械臂提供协同的全身遥操作的动机与实现。
• reclaim 脚部控制，使基座持续迁移与手部操作并行。
• 提供与任务相关的反馈（基于 LiDAR 的踏板引导、臂部力反馈、可操作性提示），以降低认知负荷和碰撞。
• 通过将反馈信号整合到增强 ACT 策略中，展示数据收集与学习收益。

提出的方法

• 引入一个 3 自由度的脚踏板用于基座控制，配合 LiDAR 驱动的阻抗反馈。
• 使用领导-跟随的双臂配置并进行重力补偿的力反馈以提升接触感知。
• 提供实时的可操作性指导以促使基座重新定位并提升可达性。
• 校准并融合多传感器输入，构建共享的以机器人为中心的反馈框架以实现一致反馈。
• 在 ACT 策略中加入力矩信息以增强学习能力的对冲（Transformer 支持的动作分块，Augmented Action Chunking with Transformers）。
• 在真实任务与仿真中进行评估，以展示遥操作性能与学习收益。

Figure 2 : TriPilot-FF combines bimanual leader-follower arms with a pedal that commands the mobile base. The low-cost pedal has 3 DoF, and the design is shown on the left. An overview of the system is shown on the right. TriPilot-FF provides force feedback on both the operator-side leader arms and

实验结果

研究问题

• RQ1 臂侧力反馈与 LiDAR 驱动脚踏线索是否在狭窄或杂物密布的空间中共同提升协同能力？
• RQ2 可操作性引导是否降低在可达性受限的检索任务中的协同负担和完成时间？
• RQ3 臂侧力反馈是否在带基座移动与操作的约束接触任务中提升效率和接触调控？
• RQ4 TriPilot-FF 是否能够在长期真实任务序列中保持性能？
• RQ5 将触觉信号融入 ACT 是否为模仿学习提供有用的监督？

主要发现

• 踏板反馈结合 LiDAR 阻力显著提升 BlindCarry 的成功率从 55% 提升至 100%，且避免碰撞。
• 可操作性引导降低 GuidedReach 的完成时间以及在低灵活度配置下的停留时间。
• 臂侧力反馈改善接触稳定性并降低在受限接触任务（如 MobileSwipe）中的能耗。
• 带力矩的 ACT 策略在仿真和真实任务中优于基线，成功率提升（如 CubeTransfer 从 22% 提升到 50%，CubePickPlace 从 28% 提升到 36%）。
• 长时域遥操作显示系统能够在包含真实任务的扩展工作流中实现可行的整合。

Figure 3 : The potential $\phi$ produces an increasingly larger repulsive force as obstacles approach the robot.