[论文解读] SAM-based instance segmentation models for the automation of structural damage detection

Zehao Ye, Lucy Lovell+2 more|arXiv (Cornell University)|Jan 27, 2024

Infrastructure Maintenance and Monitoring被引用 4

一句话总结

本文提出了一种基于SAM的新型实例分割方法，用于自动化砌体裂缝检测，引入了可学习的自生成提示器和无提示解码器集成。通过LoRA微调，模型在整体性能上超越了最先进水平约3%，在裂缝检测上提升约6%，同时利用单目相机与基于霍夫线变换的正交投影，实现了与激光扫描结果偏差小于10%的精确裂缝尺寸估计。

ABSTRACT

Automating visual inspection for capturing defects based on civil structures appearance is crucial due to its currently labour-intensive and time-consuming nature. An important aspect of automated inspection is image acquisition, which is rapid and cost-effective considering the pervasive developments in both software and hardware computing in recent years. Previous studies largely focused on concrete and asphalt, with less attention to masonry cracks. The latter also lacks publicly available datasets. In this paper, we first present a corresponding data set for instance segmentation with 1,300 annotated images (640 pixels x 640 pixels), named as MCrack1300, covering bricks, broken bricks, and cracks. We then test several leading algorithms for benchmarking, including the latest large-scale model, the prompt-based Segment Anything Model (SAM). We fine-tune the encoder using Low-Rank Adaptation (LoRA) and proposed two novel methods for automation of SAM execution. The first method involves abandoning the prompt encoder and connecting the SAM encoder to other decoders, while the second method introduces a learnable self-generating prompter. In order to ensure the seamless integration of the two proposed methods with SAM encoder section, we redesign the feature extractor. Both proposed methods exceed state-of-the-art performance, surpassing the best benchmark by approximately 3% for all classes and around 6% for cracks specifically. Based on successful detection, we propose a method based on a monocular camera and the Hough Line Transform to automatically transform images into orthographic projection maps. By incorporating known real sizes of brick units, we accurately estimate crack dimensions, with the results differing by less than 10% from those obtained by laser scanning. Overall, we address important research gaps in automated masonry crack detection and size estimation.

研究动机与目标

• 解决砌体裂缝实例分割领域缺乏公开可用数据集的问题。
• 利用深度学习实现砌体结构损伤的自动化检测，重点关注裂缝和破损砖块。
• 克服分割一切模型（SAM）在实际结构检测中对提示依赖的局限性。
• 通过几何变换，实现仅从单目图像中准确估计裂缝尺寸。
• 开发一种可扩展、低成本的解决方案，用于土木基础设施的自动化视觉检测。

提出的方法

• 提出了一项新数据集MCrack1300，包含1,300张标注图像（640×640像素），涵盖砖块、破损砖块和裂缝。
• 使用低秩适应（LoRA）对SAM编码器进行微调，实现高效参数适配。
• 提出一种无提示方法，通过将替代解码器直接连接至SAM编码器，替代提示编码器。
• 开发了一种可学习的自生成提示器，可自主生成有效的分割提示。
• 重新设计了特征提取器，以确保两种新方法与SAM编码器的无缝集成。
• 应用单目相机成像，结合霍夫线变换，从真实世界图像生成正交投影图。

实验结果

研究问题

• RQ1 与标准提示方法推理相比，微调后的无提示SAM适配方法是否能在砌体裂缝实例分割上实现更优的性能？
• RQ2 可学习的自生成提示器是否能有效替代人工提供的提示，用于SAM的自动化结构损伤检测？
• RQ3 在已知砖块尺寸的前提下，基于单目相机的正交投影方法在多大程度上可实现精确的裂缝长度估计？
• RQ4 在所有裂缝和砌体缺陷类别上，所提方法在mAP指标上与最先进模型相比如何？
• RQ5 该系统能否在裂缝尺寸估计上实现相对于激光扫描基准的误差小于10%？

主要发现

• 所提出的无提示方法在所有缺陷类别上，相较于最佳基线模型，mAP提升了3%。
• 在裂缝特异性检测中，该方法在mAP上比最先进水平高出约6%。
• 可学习的自生成提示器表现出稳健性能，无需人工提示工程。
• 利用单目相机与霍夫线变换方法进行裂缝长度估计，与激光扫描测量结果的偏差小于10%。
• MCrack1300数据集为砌体裂缝实例分割提供了新基准，包含1,300张高分辨率、完全标注的图像。
• 重新设计的特征提取器实现了两种新方法在推理过程中的稳定且高性能表现。