[Paper Review] LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

Paul Maria Scheikl, Balázs Gyenes+5 more|arXiv (Cornell University)|Feb 19, 2023

Surgical Simulation and Training19 citations

TL;DR

LapGym presents an open-source RL framework and a 12-environment suite for robot-assisted laparoscopic surgery, enabling standardized, parameterizable tasks and PPO baselines built on SOFA.

ABSTRACT

Recent advances in reinforcement learning (RL) have increased the promise of introducing cognitive assistance and automation to robot-assisted laparoscopic surgery (RALS). However, progress in algorithms and methods depends on the availability of standardized learning environments that represent skills relevant to RALS. We present LapGym, a framework for building RL environments for RALS that models the challenges posed by surgical tasks, and sofa_env, a diverse suite of 12 environments. Motivated by surgical training, these environments are organized into 4 tracks: Spatial Reasoning, Deformable Object Manipulation & Grasping, Dissection, and Thread Manipulation. Each environment is highly parametrizable for increasing difficulty, resulting in a high performance ceiling for new algorithms. We use Proximal Policy Optimization (PPO) to establish a baseline for model-free RL algorithms, investigating the effect of several environment parameters on task difficulty. Finally, we show that many environments and parameter configurations reflect well-known, open problems in RL research, allowing researchers to continue exploring these fundamental problems in a surgical context. We aim to provide a challenging, standard environment suite for further development of RL for RALS, ultimately helping to realize the full potential of cognitive surgical robotics. LapGym is publicly accessible through GitHub (https://github.com/ScheiklP/lap_gym).

Motivation & Objective

• Provide a standardized, open-source RL environment suite for robot-assisted laparoscopic surgery (RALS).
• Enable configurable, progressively difficult tasks across multiple surgical skill tracks.
• Facilitate integration of image-based observations and deformable object simulations using SOFA.
• Offer tools for easy environment creation, human control, and imitation learning data collection.

Proposed method

• Introduce LapGym as an open-source framework for RL in RALS built on the SOFA physics engine.
• Provide sofa env with 12 tasks organized into four tracks to cover Spatial Reasoning, Deformable Object Manipulation & Grasping, Dissection, and Thread Manipulation.
• Utilize PPO to establish baseline performance across environment configurations.
• Offer sofa templates and sofa godot to simplify creating new scenes and RL environments.
• Support multi-modal observations including RGB, RGBD, point clouds, and semantic images with Gym-compliant interfaces.
• Incorporate human control and path planning to generate expert trajectories and demonstrations.

Experimental results

Research questions

• RQ1 How can a standardized, configurable RL environment suite for RALS be designed to cover a broad range of surgical skills?
• RQ2 What is the impact of environment parameters and observation types on learning difficulty and PPO performance in RALS tasks?
• RQ3 Can deformable object simulation and image-based observations be effectively leveraged within an RL framework for RALS?
• RQ4 Does an open-source tooling chain (templates, Godot integration) reduce the barrier to creating new RL tasks in SOFA-based scenes?

Key findings

• LapGym provides a diverse, highly parameterizable suite of 12 environments across four surgical tracks for progressive skill learning.
• The environments support RGB, RGBD, point clouds, and semantic observations and are compatible with Gym interfaces for standard RL libraries.
• Baseline PPO experiments demonstrate how varying environment configurations affect task difficulty and learning performance.
• SOFA-based simulation enables realistic deformable object dynamics, topological changes, and RCM-constrained instrument motion suitable for RL experimentation.
• Godot-based interactive scene creation and sofa templates simplify building and adapting SOFA scenes for RL tasks.
• The framework supports human demonstrations and planner-generated trajectories to support imitation learning.