[Paper Review] From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring

Florent Forest, Hugo Porta+2 more|arXiv (Cornell University)|Sep 20, 2023

Infrastructure Maintenance and MonitoringEngineering3 citations

TL;DR

This paper proposes a weakly supervised method for crack segmentation and growth monitoring by leveraging explainable AI (XAI) to generate pixel-level masks from a binary classifier's explanations, requiring only image-level annotations. Results show that LRP-based XAI achieves a promising F1-score of ~0.4 and enables effective crack severity monitoring despite lower segmentation quality than fully supervised methods.

ABSTRACT

Monitoring surface cracks in infrastructure is crucial for structural health monitoring. Automatic visual inspection offers an effective solution, especially in hard-to-reach areas. Machine learning approaches have proven their effectiveness but typically require large annotated datasets for supervised training. Once a crack is detected, monitoring its severity often demands precise segmentation of the damage. However, pixel-level annotation of images for segmentation is labor-intensive. To mitigate this cost, one can leverage explainable artificial intelligence (XAI) to derive segmentations from the explanations of a classifier, requiring only weak image-level supervision. This paper proposes applying this methodology to segment and monitor surface cracks. We evaluate the performance of various XAI methods and examine how this approach facilitates severity quantification and growth monitoring. Results reveal that while the resulting segmentation masks may exhibit lower quality than those produced by supervised methods, they remain meaningful and enable severity monitoring, thus reducing substantial labeling costs.

Motivation & Objective

• Address the high cost of pixel-level annotation in crack segmentation by leveraging weak supervision via explainable AI.
• Enable severity quantification and long-term growth monitoring of surface cracks using only image-level labels.
• Evaluate the performance of multiple XAI methods in generating meaningful segmentation masks from classifier explanations.
• Propose modifications to XAI methods (e.g., using damage-free images as baselines) to improve explanation quality for crack detection.
• Assess the feasibility of using XAI-derived masks for practical structural health monitoring tasks, including crack area and width estimation.

Proposed method

• Train a binary classifier to distinguish between damage-free and cracked wall images using image-level labels.
• Apply various XAI methods—such as LRP, Integrated Gradients, DeepLift, GradientShap, and Input×Gradient—to generate attribution maps from the trained classifier.
• Use damage-free images as baselines in Integrated Gradients and DeepLift to improve explanation fidelity for crack detection.
• Transform attribution maps into binary segmentation masks via thresholding, including simple thresholding and Gaussian Mixture Model (GMM) post-processing.
• Apply morphological operations to refine the final masks and reduce noise.
• Use the resulting masks to compute severity metrics such as crack area, maximum width, and number of cracks for growth monitoring.

Experimental results

Research questions

• RQ1 Can XAI-derived attribution maps generate segmentation masks of sufficient quality for crack severity quantification and growth monitoring?
• RQ2 How do different XAI methods compare in terms of segmentation quality, computational efficiency, and ability to monitor crack evolution?
• RQ3 What impact do methodological modifications—such as using damage-free images as baselines—have on the quality of XAI explanations for crack detection?
• RQ4 To what extent can XAI-based segmentation replace or complement fully supervised segmentation in real-world structural health monitoring?
• RQ5 Can XAI-derived masks effectively track changes in crack severity over time despite potential overestimation of crack extent?

Key findings

• The Layer-wise Relevance Propagation (LRP) method achieved the best balance between segmentation quality, growth monitoring capability, and computational efficiency.
• XAI-derived masks achieved an F1-score of approximately 0.4, which is lower than fully supervised methods like U-Net but significantly outperforms unsupervised approaches such as autoencoders.
• The computational runtime of XAI-based segmentation was in the same order of magnitude as a U-Net inference, with Input×Gradient being the fastest (0.251 s/image) and Integrated Gradients the slowest (due to 50 interpolation steps).
• Post-processing with GMM thresholding increased runtime by an order of magnitude (0.372 s/image) compared to simple thresholding (0.018 s/image), but improved mask quality.
• Despite overestimation of crack extent, the method enabled effective monitoring of crack severity evolution over time, particularly for metrics like crack area and number of cracks.
• The modified use of damage-free images as baselines in Integrated Gradients and DeepLift improved explanation quality, especially in distinguishing true crack patterns from background noise.