[论文解读] Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping

Gabriella Pizzuto, Hetong Wang+4 more|arXiv (Cornell University)|Sep 29, 2022

Innovative Microfluidic and Catalytic Techniques Innovation被引用 5

一句话总结

本文提出了一种无模型强化学习（RL）方法，用于在化学实验室中自主学习机器人的样品刮擦操作，通过仿真和真实世界的机器人操作臂实现对瓶中粉末的精确、力感知刮擦。该方法在不同尺寸的瓶和不同刮刀工具下均实现了成功刮擦，证明了在结晶流程中自动化这一关键且高超操作的可行性。

ABSTRACT

The use of laboratory robotics for autonomous experiments offers an attractive route to alleviate scientists from tedious tasks while accelerating material discovery for topical issues such as climate change and pharmaceuticals. While some experimental workflows can already benefit from automation, sample preparation is still carried out manually due to the high level of motor function and dexterity required when dealing with different tools, chemicals, and glassware. A fundamental workflow in chemical fields is crystallisation, where one application is polymorph screening, i.e., obtaining a three dimensional molecular structure from a crystal. For this process, it is of utmost importance to recover as much of the sample as possible since synthesising molecules is both costly in time and money. To this aim, chemists scrape vials to retrieve sample contents prior to imaging plate transfer. Automating this process is challenging as it goes beyond robotic insertion tasks due to a fundamental requirement of having to execute fine-granular movements within a constrained environment (sample vial). Motivated by how human chemists carry out this process of scraping powder from vials, our work proposes a model-free reinforcement learning method for learning a scraping policy, leading to a fully autonomous sample scraping procedure. We first create a scenario-specific simulation environment with a Panda Franka Emika robot using a laboratory scraper that is inserted into a simulated vial, to demonstrate how a scraping policy can be learned successfully in simulation. We then train and evaluate our method on a real robotic manipulator in laboratory settings, and show that our method can autonomously scrape powder across various setups.

研究动机与目标

• 为解决在实验室工作流程中自动化手动、高精度操作（如样品刮擦）的挑战，这些任务目前因复杂性而由化学家手动执行。
• 开发一种数据驱动的机器人策略，使其能够在受限的瓶体环境中学习精细、力敏感的刮擦动作。
• 证明在仿真环境中训练的强化学习策略可成功部署到真实机器人操作臂上，实现瓶内刮擦。
• 通过实现自主、可重复的瓶壁粉末刮擦，提高结晶流程中的样品回收率。
• 探索课程学习作为提升复杂操作任务训练中样本效率和成功率的方法。

提出的方法

• 使用Panda Franka Emika机器人、实验室刮刀和瓶体制作了一个仿真环境，采用基于物理的动力学模型来模拟插入和刮擦过程。
• 采用无模型深度强化学习（DRL）算法，训练一个将本体感觉和力觉观测映射到末端执行器动作的策略。
• 通过在训练过程中逐步增加工作空间尺寸，应用课程学习以提升样本效率和成功率。
• 策略被训练以最大化在瓶体预定义区域内的粉末回收量，机器人在旋转瓶体后，对24个区域（每个区域π/12弧度）重复刮擦循环。
• 最终策略通过最小化领域随机化，成功迁移到真实机器人平台上，依赖于精心设计的仿真到现实的迁移策略。
• 实施了安全措施，包括设置一个力阈值（Fz > 20 N），当超过该阈值时触发返回起点动作，以防止瓶体倾倒或破裂。

Figure 1: Overview of laboratory sample scraping, where (left) a human scientist scrapes crystals that generally form on vial walls by transcending along the walls vertically and (right) our robot learning to scrape via RL.

实验结果

研究问题

• RQ1 无模型强化学习能否在受限瓶体环境中学习到一种高超、力感知的刮擦策略，用于机器人样品回收？
• RQ2 课程学习在提升复杂瓶内刮擦任务训练的样本效率和成功率方面有何作用？
• RQ3 在仿真环境中训练的策略能否成功部署到真实机器人操作臂上，实现对不同瓶体和工具配置的自主刮擦？
• RQ4 真实部署中的关键故障模式是什么？如何通过设计和安全约束加以缓解？
• RQ5 视觉输入或替代工具在多大程度上能提升刮擦策略的效率和鲁棒性？

主要发现

• 所提出的方法在仿真中成功学习了刮擦策略，该策略在不同瓶体尺寸和刮刀长度下均具有泛化能力，并在真实实验中实现了完全粉末清除。
• 课程学习的使用显著提升了训练性能，使智能体能够学习到标准强化学习失败的复杂插入和刮擦任务。
• 该策略成功部署于真实机器人平台，展示了在瓶体24个区域中的一致性自主刮擦性能。
• 该方法在从瓶壁刮除粉末方面实现了高成功率，仅出现一次不可恢复的故障（刮刀卡住），需人工干预。
• 安全措施（包括z轴力阈值Fz > 20 N）在真实操作中有效防止了瓶体倾倒和破裂。
• 结果表明，DRL能够实现传统实验室机器人难以实现的任务自动化，特别是那些需要精细运动控制和力反馈的任务。

Figure 2: The overall block diagram of autonomous robotic scraping. Our method consists of an RL policy and a robotic controller. We map the action (cartesian pose) to joint positions using inverse kinematics to achieve the target goal. The controller uses force/torque feedback at the end effector a