[Paper Review] Automatic Defect Segmentation on Leather with Deep Learning

Sze‐Teng Liong, Y.S. Gan+3 more|arXiv (Cornell University)|Mar 28, 2019

Industrial Vision Systems and Defect Detection19 references37 citations

TL;DR

This paper presents an end-to-end automatic defect segmentation system for calf leather using Mask R-CNN, with robotic data collection, ground-truth annotation, and automated defect marking. It achieves 91.5% training accuracy and 70.35% test accuracy for tick-bite defect segmentation.

ABSTRACT

Leather is a natural and durable material created through a process of tanning of hides and skins of animals. The price of the leather is subjective as it is highly sensitive to its quality and surface defects condition. In the literature, there are very few works investigating on the defects detection for leather using automatic image processing techniques. The manual defect inspection process is essential in an leather production industry to control the quality of the finished products. However, it is tedious, as it is labour intensive, time consuming, causes eye fatigue and often prone to human error. In this paper, a fully automatic defect detection and marking system on a calf leather is proposed. The proposed system consists of a piece of leather, LED light, high resolution camera and a robot arm. Succinctly, a machine vision method is presented to identify the position of the defects on the leather using a deep learning architecture. Then, a series of processes are conducted to predict the defect instances, including elicitation of the leather images with a robot arm, train and test the images using a deep learning architecture and determination of the boundary of the defects using mathematical derivation of the geometry. Note that, all the processes do not involve human intervention, except for the defect ground truths construction stage. The proposed algorithm is capable to exhibit 91.5% segmentation accuracy on the train data and 70.35% on the test data. We also report confusion matrix, F1-score, precision and specificity, sensitivity performance metrics to further verify the effectiveness of the proposed approach.

Motivation & Objective

• Motivate automatic, objective defect detection in leather to replace labor-intensive manual inspection.
• Develop an end-to-end system that localizes and segments defects on leather surfaces.
• Leverage deep learning for segmentation and a robot arm for data collection and defect marking.
• Provide ground-truth annotations and evaluate using standard segmentation metrics to validate effectiveness.

Proposed method

• Use a six-axis robotic arm with a high-resolution camera and continuous LED lighting to capture top-down leather images.
• Partition each high-resolution image into 400x400 patches for local feature learning without resizing.
• Annotate defect boundaries with an open-source tool to create pixel-level ground-truth masks.
• Train a Mask R-CNN (FPN with ResNet-101 backbone) pre-trained on MS COCO, fine-tuned on leather defects.
• Adjust the Region Proposal Network (RPN) anchor scales to better detect small defect regions.
• Post-process Mask R-CNN outputs to extract salient boundary coordinates and map them to physical coordinates for robotic marking.

Experimental results

Research questions

• RQ1 Can Mask R-CNN accurately detect and segment tick-bite defects on calf leather in a practical setting?
• RQ2 What is the segmentation performance on training versus unseen test leather images for this defect type?
• RQ3 How can boundary coordinates be efficiently transformed from image space to real-world coordinates for automated marking?
• RQ4 Does a robotic system with automated chalk marking improve defect visualization and traceability compared to manual inspection?

Key findings

• Training accuracy on the leather defect dataset reached 91.50%.
• Test accuracy on the unseen leather images reached 70.35%.
• Sensitivity on the test set was 53.57% and specificity 75.81%, indicating more correct identification of non-defective areas than defective ones in testing.
• The system produced 97 true positives and 104 false negatives for the defective class in the test set, with 326 true negatives and 65 false positives across non-defective cases.
• Ground-truth ground assertions indicate the model correctly identified multiple defects per image, but some defects were missed or produced false alarms in testing.
• A boundary optimization technique was used to select salient points (vacuum values 4 or 5) for polygonal defect outlining, processed via Graham Scan to produce a convex boundary.