[Paper Review] Optical Tactile Sim-to-Real Policy Transfer via Real-to-Sim Tactile Image Translation.

Alex Church, John W. Lloyd+2 more|arXiv (Cornell University)|Jun 16, 2021

Robot Manipulation and Learning33 references4 citations

TL;DR

This paper proposes a sim-to-real policy transfer framework for tactile robotics using real-to-sim tactile image translation, enabling zero-shot deployment of policies trained in simulation onto real robots. By simulating high-resolution optical tactile sensors as depth images and translating real tactile sensor data into simulated counterparts via a data-driven method, the approach achieves successful task execution in physical manipulation tasks using PPO-trained policies.

ABSTRACT

Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A simple and fast method of simulating optical tactile sensors is provided, where high-resolution contact geometry is represented as depth images. Proximal Policy Optimisation (PPO) is used to learn successful policies across all considered tasks. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zero-shot sim-to-real policy transfer on several physically-interactive tasks requiring a sense of touch.

Motivation & Objective

• To bridge the sim-to-real gap in tactile reinforcement learning by enabling policy transfer from simulation to real-world robotic control.
• To develop a fast and accurate method for simulating optical tactile sensors using high-resolution depth images of contact geometry.
• To enable real-time policy deployment on physical robots by translating real tactile sensor observations into corresponding simulated sensor inputs.
• To demonstrate the effectiveness of the approach on physically interactive tasks requiring tactile perception.

Proposed method

• Simulate optical tactile sensors by rendering high-resolution depth images from detailed contact geometry in simulation.
• Train a data-driven translation model to convert real tactile sensor images into corresponding simulated depth images.
• Use Proximal Policy Optimization (PPO) to train control policies in simulation using visual and proprioceptive observations.
• Integrate the tactile translation model into a real-time control loop to enable zero-shot policy transfer to the real robot.
• Design a suite of simulation environments tailored for tactile robotics and complex manipulation tasks.
• Enable end-to-end policy deployment by aligning real sensor observations with simulated observations via image translation.

Experimental results

Research questions

• RQ1 Can real-to-sim tactile image translation enable zero-shot sim-to-real policy transfer in tactile robotics?
• RQ2 How effective is the proposed simulation method for optical tactile sensors in capturing high-fidelity contact geometry?
• RQ3 What is the performance of PPO-trained policies when deployed on a real robot using translated tactile observations?
• RQ4 How robust is the policy transfer across diverse physically-interactive tasks requiring tactile sensing?

Key findings

• The proposed method enables successful zero-shot sim-to-real policy transfer for multiple physically-interactive tasks requiring tactile feedback.
• The data-driven tactile image translation model effectively aligns real tactile sensor data with simulated depth images, enabling accurate policy deployment.
• PPO-trained policies generalize well from simulation to the real world when using the translated tactile observations.
• The simulation framework supports high-fidelity optical tactile sensing through depth image representation of contact geometry.
• Real-time control loop integration demonstrates the feasibility of deploying learned policies on physical robots without fine-tuning.
• The approach achieves reliable task execution across multiple tactile manipulation tasks without sim-to-real domain randomization or additional adaptation.