[Paper Review] ReSkin: versatile, replaceable, lasting tactile skins

Raunaq Bhirangi, Tess Hellebrekers+2 more|arXiv (Cornell University)|Oct 29, 2021

Advanced Sensor and Energy Harvesting Materials35 references4 citations

TL;DR

ReSkin is a low-cost, replaceable, and durable tactile sensing skin that uses magnetic field distortion and machine learning to enable high-precision force and contact localization across diverse form factors. It achieves robust performance across new and worn skins through self-supervised learning, enabling immediate usability after replacement with minimal calibration.

ABSTRACT

Soft sensors have continued growing interest in robotics, due to their ability to enable both passive conformal contact from the material properties and active contact data from the sensor properties. However, the same properties of conformal contact result in faster deterioration of soft sensors and larger variations in their response characteristics over time and across samples, inhibiting their ability to be long-lasting and replaceable. ReSkin is a tactile soft sensor that leverages machine learning and magnetic sensing to offer a low-cost, diverse and compact solution for long-term use. Magnetic sensing separates the electronic circuitry from the passive interface, making it easier to replace interfaces as they wear out while allowing for a wide variety of form factors. Machine learning allows us to learn sensor response models that are robust to variations across fabrication and time, and our self-supervised learning algorithm enables finer performance enhancement with small, inexpensive data collection procedures. We believe that ReSkin opens the door to more versatile, scalable and inexpensive tactile sensation modules than existing alternatives.

Motivation & Objective

• Address the lack of long-lasting, replaceable, and scalable tactile sensing solutions for robotics and human-robot interaction.
• Overcome the trade-off between soft conformal contact and sensor degradation over time and across samples.
• Enable accurate tactile perception across new sensor skins without requiring full retraining or recalibration.
• Develop a compact, low-cost, and versatile tactile skin suitable for diverse applications including robot grippers, wearable sensors, and large-area skins.
• Achieve high spatial (1mm) and temporal (up to 400Hz) resolution with force sensitivity below 0.1N

Proposed method

• Use a soft, magnetized skin layer that deforms under contact, altering local magnetic fields detected by embedded flexible magnetometers.
• Leverage machine learning to map magnetic field distortions to contact force and location, enabling robust inference across fabrication variations and time.
• Implement a self-supervised learning (SSL) adaptation procedure using triplet loss to refine models on new skins with minimal data collection.
• Decouple the electronic sensing circuitry from the passive skin interface, enabling easy replacement of worn skins like a band-aid.
• Scale ReSkin to large contiguous surfaces (e.g., 2in x 4in) by connecting multiple flexible sensor boards to a single microcontroller.
• Use on-board data logging and real-time sampling (up to 400Hz) for deployment in mobile and wearable applications.

Experimental results

Research questions

• RQ1 Can a tactile sensing system maintain high accuracy and spatial resolution across multiple sensor replacements without retraining?
• RQ2 How well can a self-supervised learning approach adapt a pre-trained model to a new, uncalibrated ReSkin sensor with minimal data?
• RQ3 To what extent can magnetic sensing with machine learning generalize across fabrication variations and long-term use?
• RQ4 Can ReSkin achieve sufficient force sensitivity (<0.1 N) and temporal resolution (>100 Hz) for dexterous manipulation tasks?
• RQ5 How effectively can ReSkin be scaled to large, contiguous tactile skins for full-surface contact localization?

Key findings

• ReSkin achieves 90% accuracy in contact localization at 1mm spatial resolution using a self-supervised adaptation procedure on new sensor skins.
• The system maintains accurate force and contact predictions across more than 50,000 interactions, demonstrating long-term durability.
• ReSkin sensors can detect forces as low as 0.2 N (equivalent to less than 20 mL of water), enabling sensitive detection of light contacts.
• The self-supervised learning method enables high-precision performance on new skins with minimal data collection, reducing calibration time and cost.
• ReSkin enables successful grasping of delicate objects like blueberries and grapes using force feedback, outperforming built-in force sensors in the Robotiq Hand-E gripper.
• ReSkin was successfully deployed in diverse applications: dog shoes (measuring gait forces), human finger gloves (dough sealing), and large 2in x 4in contiguous skins for full-surface sensing.