[Paper Review] Experimental Force-Torque Dataset for Robot Learning of Multi-Shape Insertion

Giovanni De Magistris, Asim Munawar+4 more|arXiv (Cornell University)|Jul 18, 2018

Robot Manipulation and Learning7 references6 citations

TL;DR

This paper presents a publicly available experimental force-torque dataset for multi-shape peg-in-hole insertion tasks, collected using a UR5 robot with a 6-axis F/T sensor, enabling learning-based control and shape recognition. The dataset achieves 81.01% accuracy in shape recognition and enables 100% success rate in robot insertion tasks using only F/T sensor feedback, demonstrating the importance of transient contact dynamics for alignment.

ABSTRACT

The accurate modeling of real-world systems and physical interactions is a common challenge towards the resolution of robotics tasks. Machine learning approaches have demonstrated significant results in the modeling of complex systems (e.g., articulated robot structures, cable stretch, fluid dynamics), or to learn robotics tasks (e.g., grasping, reaching) from raw sensor measurements without explicit programming, using reinforcement learning. However, a common bottleneck in machine learning techniques resides in the availability of suitable data. While many vision-based datasets have been released in the recent years, ones involving physical interactions, of particular interest for the robotic community, have been scarcer. In this paper, we present a public dataset on peg-in-hole insertion tasks containing force-torque and pose information for multiple variations of convex-shaped pegs. We demonstrate how this dataset can be used to train a robot to insert polyhedral pegs into holes using only 6-axis force/torque sensor measurements as inputs, as well as other tasks involving contact such as shape recognition.

Motivation & Objective

• To address the scarcity of real-world physical interaction datasets in robotics, particularly for contact-intensive tasks like peg-in-hole insertion.
• To enable machine learning approaches to learn insertion policies and shape recognition using only force-torque sensor data.
• To provide a benchmark dataset with controlled variations in peg geometry (3- to 200-sided polygons) and precise pose/force measurements.
• To demonstrate the feasibility of training neural networks on real F/T data for real robot deployment without explicit programming.
• To support future research in transfer learning, simulation-to-real transfer, and robust insertion under uncertainty.

Proposed method

• Collected force-torque and pose data at 100 Hz using a UR5 robot with a Robotiq FT-150 sensor during controlled peg-in-hole operations.
• Used polyhedral pegs with regular n-gonal faces (n = 3, 4, 5, 6, 200) and holes of matching shape, with precise manual alignment to eliminate grasp-induced errors.
• Recorded 11-dimensional state vectors: Fx, Fy, Fz, Mx, My, Mz, Px, Py, Pz, Az, t, and a counter (cont), with position and orientation relative to the hole.
• Applied a multi-layer perceptron (MLP) to classify insertion actions based on F/T and pose sequences, using sliding window inputs with parameters d=50ms, l=200ms, u=1000ms.
• Trained models on transient and steady-state phases separately to analyze which dynamics contain critical information for task success.
• Validated learned policies on the real UR5 robot using force/motion commands based on predicted actions, achieving 100% success rate.

Experimental results

Research questions

• RQ1 Can force-torque sensor data alone enable successful learning of multi-shape peg-in-hole insertion without vision or explicit programming?
• RQ2 Which phase of the contact interaction—transient or steady-state—contains the most informative signals for alignment and insertion?
• RQ3 How does the geometric shape of the peg (number of sides) affect the detectability of contact dynamics and task performance?
• RQ4 To what extent can a learned policy generalize across different peg shapes using only F/T feedback?
• RQ5 Can the same dataset be used to train models for shape recognition based on contact dynamics?

Key findings

• The dataset achieves 81.01% accuracy in shape recognition using MLP on F/T data, with performance increasing with the number of peg sides.
• For the T1 task (alignment), the best accuracy of 72.47% was achieved using data from the steady-state contact phase with a 1000ms window.
• For the T2 task (alignment), the best accuracy of 61.39% was achieved using only transient-phase data with a 50ms step size and 200ms window.
• The transient phase contains critical information for T2, while the steady-state phase is more informative for T1, indicating task-dependent dynamics importance.
• Using only Fx, Fy, Mx, My as inputs yielded the highest accuracy (70.57%) in T1, showing that Fz and Mz contribute less to decision-making.
• The learned policy achieved 100% success rate in real robot insertion experiments after an average of 7.5 actions, starting from a 2mm offset, using only F/T feedback.