[论文解读] Automatic Defect Segmentation on Leather with Deep Learning

Sze‐Teng Liong, Y.S. Gan+3 more|arXiv (Cornell University)|Mar 28, 2019

Industrial Vision Systems and Defect Detection参考文献 19被引用 37

一句话总结

本文提出了一个用于小牛皮全自动缺陷分割系统，使用 Mask R-CNN、机器人数据采集、真实标注和自动缺陷标记。它在牛皮缺陷分割的训练准确率达到 91.5%，在测试集的准确率达到 70.35%。

ABSTRACT

Leather is a natural and durable material created through a process of tanning of hides and skins of animals. The price of the leather is subjective as it is highly sensitive to its quality and surface defects condition. In the literature, there are very few works investigating on the defects detection for leather using automatic image processing techniques. The manual defect inspection process is essential in an leather production industry to control the quality of the finished products. However, it is tedious, as it is labour intensive, time consuming, causes eye fatigue and often prone to human error. In this paper, a fully automatic defect detection and marking system on a calf leather is proposed. The proposed system consists of a piece of leather, LED light, high resolution camera and a robot arm. Succinctly, a machine vision method is presented to identify the position of the defects on the leather using a deep learning architecture. Then, a series of processes are conducted to predict the defect instances, including elicitation of the leather images with a robot arm, train and test the images using a deep learning architecture and determination of the boundary of the defects using mathematical derivation of the geometry. Note that, all the processes do not involve human intervention, except for the defect ground truths construction stage. The proposed algorithm is capable to exhibit 91.5% segmentation accuracy on the train data and 70.35% on the test data. We also report confusion matrix, F1-score, precision and specificity, sensitivity performance metrics to further verify the effectiveness of the proposed approach.

研究动机与目标

• 推动皮革自动、客观的缺陷检测，以替代劳动密集型的人工检验。
• 开发一个端到端的系统，在皮革表面定位并分割缺陷。
• 利用深度学习进行分割，机器人手臂用于数据采集和缺陷标记。
• 提供真实标注并使用标准分割指标进行评估，以验证有效性。

提出的方法

• 使用六轴机器人手臂、高分辨率相机和连续LED照明来获取皮革的顶视图影像。
• 将每张高分辨率图像分割为 400x400 的小块，以便在不进行尺寸缩放的情况下进行局部特征学习。
• 使用开源工具对缺陷边界进行标注，创建像素级真实掩膜。
• 训练一个基于 MS COCO 预训练、FPN + ResNet-101 主干的 Mask R-CNN，在皮革缺陷上进行微调。
• 调整 Region Proposal Network (RPN) 的锚框尺度，以更好地检测小缺陷区域。
• 对 Mask R-CNN 的输出进行后处理，提取显著边界坐标并映射到物理坐标以实现机器人标记。

实验结果

研究问题

• RQ1 Mask R-CNN 是否能够在实际场景下准确检测并分割小牛皮上的刺破缺陷？
• RQ2 对该缺陷类型，训练集与未见测试皮革影像的分割性能如何？
• RQ3 如何高效地将边界坐标从图像空间转换为实际世界坐标，以实现自动标记？
• RQ4 具备自动粉笔标记的机器人系统是否比人工检验在缺陷可视化与可追溯性方面有所提升？

主要发现

• 在皮革缺陷数据集上的训练准确率达到 91.50%。
• 在未见皮革影像的测试集上，准确率达到 70.35%。
• 测试集的灵敏度为 53.57%，特异性为 75.81%，表明在测试中对非缺陷区域的正确识别高于缺陷区域。
• 系统在测试集中缺陷类别的真阳性为 97、假阴性为 104；在非缺陷病例中，真阴性为 326、假阳性为 65。
• 真实标注表明模型在多张图像中正确识别了多个缺陷，但在测试中仍有部分缺陷被漏检或产生误报。
• 采用一种边界优化技术来选择显著点（真空值为 4 或 5）用于多边形缺陷轮廓描绘，并通过 Graham Scan 生成凸边界。