[论文解读] GelSight Fin Ray: Incorporating Tactile Sensing into a Soft Compliant Robotic Gripper

Sandra Q. Liu, Edward H. Adelson|arXiv (Cornell University)|Apr 14, 2022

Advanced Sensor and Energy Harvesting Materials被引用 5

一句话总结

本文提出了一种新型软体机器人夹爪GelSight Fin Ray，通过在柔性Fin Ray手指结构中嵌入视觉触觉传感器，将高分辨率触觉传感集成到顺应性结构中。通过结合触觉重建、物体姿态估计和标记追踪，该夹爪仅依靠触觉反馈即成功将酒杯重新定向并直立放置，展示了在非结构化环境中增强的操控能力。

ABSTRACT

To adapt to constantly changing environments and be safe for human interaction, robots should have compliant and soft characteristics as well as the ability to sense the world around them. Even so, the incorporation of tactile sensing into a soft compliant robot, like the Fin Ray finger, is difficult due to its deformable structure. Not only does the frame need to be modified to allow room for a vision sensor, which enables intricate tactile sensing, the robot must also retain its original mechanically compliant properties. However, adding high-resolution tactile sensors to soft fingers is difficult since many sensorized fingers, such as GelSight-based ones, are rigid and function under the assumption that changes in the sensing region are only from tactile contact and not from finger compliance. A sensorized soft robotic finger needs to be able to separate its overall proprioceptive changes from its tactile information. To this end, this paper introduces the novel design of a GelSight Fin Ray, which embodies both the ability to passively adapt to any object it grasps and the ability to perform high-resolution tactile reconstruction, object orientation estimation, and marker tracking for shear and torsional forces. Having these capabilities allow soft and compliant robots to perform more manipulation tasks that require sensing. One such task the finger is able to perform successfully is a kitchen task: wine glass reorientation and placement, which is difficult to do with external vision sensors but is easy with tactile sensing. The development of this sensing technology could also potentially be applied to other soft compliant grippers, increasing their viability in many different fields.

研究动机与目标

• 使软体机器人夹爪能够执行需要触觉反馈的复杂操控任务，例如重新定向易碎物体。
• 解决将高分辨率触觉传感器集成到可变形、顺应性机器人手指中而不损害其机械顺应性的挑战。
• 在软体机器人系统中分离本体感觉形变与触觉接触信息，以实现精确的状态估计。
• 开发一种仅依赖内部传感器反馈即可实现物体姿态估计与滑移检测的触觉传感系统。
• 展示仅使用触觉反馈在真实世界任务中实现操控的可行性，例如不依赖外部视觉系统重新定位酒杯。

提出的方法

• 设计一种柔韧的长条形GelSight传感器，其基于视觉的触觉传感系统嵌入3D打印的Fin Ray手指结构中。
• 使用带有嵌入标记的透明硅胶垫捕捉接触下的表面形变，从而实现高分辨率的触觉重建。
• 采用标记追踪算法，通过分析硅胶表面标记的位移来估计物体姿态及剪切力/扭转力。
• 应用触觉重建算法，从硅胶垫形变的原始图像数据中生成未经校准的三维表面图。
• 将传感器化手指集成到机器人系统中，实现在仅依赖触觉反馈下的闭环操控。
• 通过一项厨房任务验证系统：仅使用触觉传感将透明酒杯重新定向并直立放置。

实验结果

研究问题

• RQ1 具有嵌入式基于视觉的触觉传感的软性、顺应性机器人手指能否准确重建物体形状和表面特征？
• RQ2 该系统能否仅依靠来自可变形传感器的触觉反馈，可靠地估计物体姿态并检测滑移或扭转力？
• RQ3 仅使用触觉反馈的系统能否成功执行复杂操控任务——例如重新定向并放置易碎的酒杯——而无需外部视觉系统？
• RQ4 在传感器噪声和形变存在的情况下，触觉重建与标记追踪的性能表现如何比较？
• RQ5 该触觉传感系统在不同接触力和物体几何形状下，其鲁棒性能够维持到何种程度？

主要发现

• GelSight Fin Ray成功仅依靠触觉反馈即实现了透明酒杯的重新定向与直立放置，证明了仅使用触觉反馈进行操控的可行性。
• 标记追踪算法比触觉重建算法更具鲁棒性，因其仅需少量接触且对噪声不敏感。
• 通过检测手指杆主要轴向上标记的位移，系统实现了准确的物体姿态估计。
• 触觉重建生成了未经校准的三维表面图，使物体特征（如广口玻璃罐上的字母）得以可视化，尽管高度图未经过校准。
• 在非常小或沉重的物体上，系统在重建和姿态估计方面表现不佳，原因是硅胶垫形变不足。
• 未来改进措施（如集成神经网络）有望提升重建精度并减少对参考图像匹配的依赖。