[Paper Review] Multi-Sensor Surface Analysis for Robotic Ironing

Yinxiao Li, Xiuhan Hu+4 more|arXiv (Cornell University)|Feb 16, 2016

Robotics and Sensor-Based Localization20 references4 citations

TL;DR

This paper presents a probabilistic multi-sensor framework that fuses depth-based curvature scans and illumination-based discontinuity scans to detect and classify permanent wrinkles on deformable cloth surfaces for robotic ironing. By combining Kinect-based 3D scanning with RGB imaging under controlled lighting, the system identifies high-curvature wrinkles and generates an optimized ironing path, enabling a Baxter robot to effectively remove wrinkles in under 35 seconds with iterative ironing.

ABSTRACT

Robotic manipulation of deformable objects remains a challenging task. One such task is to iron a piece of cloth autonomously. Given a roughly flattened cloth, the goal is to have an ironing plan that can iteratively apply a regular iron to remove all the major wrinkles by a robot. We present a novel solution to analyze the cloth surface by fusing two surface scan techniques: a curvature scan and a discontinuity scan. The curvature scan can estimate the height deviation of the cloth surface, while the discontinuity scan can effectively detect sharp surface features, such as wrinkles. We use this information to detect the regions that need to be pulled and extended before ironing, and the other regions where we want to detect wrinkles and apply ironing to remove the wrinkles. We demonstrate that our hybrid scan technique is able to capture and classify wrinkles over the surface robustly. Given detected wrinkles, we enable a robot to iron them using shape features. Experimental results show that using our wrinkle analysis algorithm, our robot is able to iron the cloth surface and effectively remove the wrinkles.

Motivation & Objective

• To address the challenge of autonomous robotic ironing of deformable fabrics, a key bottleneck in household robotics.
• To detect and classify permanent wrinkles—distinct from smooth, developable deformations—using surface analysis.
• To enable a robot to plan and execute ironing motions based on detected wrinkles with minimal movement and optimal alignment.
• To integrate multi-sensor data (depth and illumination) into a probabilistic framework for robust wrinkle detection.
• To demonstrate a complete ironing pipeline from detection to execution on a Baxter robot, improving upon prior work in garment manipulation.

Proposed method

• A Kinect depth sensor performs curvature scans to detect height deviations (bumps) on the cloth surface, using Hessian matrix computation and volume estimation to identify significant deformations.
• Two light sources are used to create controlled illumination, enhancing surface gradient changes around wrinkles to highlight discontinuities.
• A support vector machine (SVM) classifier is trained on manually labeled discontinuity regions to detect sharp surface features such as permanent wrinkles.
• A probabilistic multi-sensor fusion model combines curvature and discontinuity results to classify and rank deformations, distinguishing between removable bumps and ironable wrinkles.
• Detected wrinkles are represented as line segments and sorted into an optimized ironing path to minimize robot movement and alignment overhead.
• The Baxter robot executes ironing using position control, aligning the iron with each detected wrinkle and applying iterative ironing based on the path plan.

Experimental results

Research questions

• RQ1 How can robotic ironing be enabled through robust detection of permanent wrinkles on deformable cloth surfaces?
• RQ2 What multi-sensor fusion strategy enables accurate distinction between smooth height bumps and sharp discontinuities (wrinkles) without requiring precise calibration?
• RQ3 How can detected wrinkles be represented and ordered to minimize robot motion and improve ironing efficiency?
• RQ4 To what extent can a position-controlled robot effectively remove wrinkles using only surface analysis and path planning?
• RQ5 Can a hybrid scanning approach combining depth and illumination-based sensing outperform single-sensor methods in detecting subtle, high-curvature wrinkles?

Key findings

• The hybrid curvature and discontinuity scan method successfully detected permanent wrinkles with high robustness, even on low-resolution depth data from a standard Kinect sensor.
• The integration of depth-based bump detection and illumination-based discontinuity scanning enabled accurate classification of wrinkles, reducing false positives from smooth deformations.
• The use of a probabilistic multi-sensor model improved detection reliability, allowing the system to distinguish between removable height bumps and ironable wrinkles.
• The optimized ironing path reduced total ironing time to approximately 35 seconds for garments with multiple wrinkles, outperforming non-optimized or randomized paths.
• Experimental results demonstrated that the Baxter robot could iteratively iron detected wrinkles and produce a visibly wrinkle-free surface, validating the effectiveness of the surface analysis pipeline.