[Paper Review] Unsupervised crack detection on complex stone masonry surfaces

Panagiotis Agrafiotis, Anastastios Doulamis+1 more|arXiv (Cornell University)|Mar 31, 2023

Infrastructure Maintenance and Monitoring4 citations

TL;DR

This paper proposes an unsupervised deep learning approach for crack detection on complex stone masonry surfaces using RGB image patches, framing crack detection as an anomaly detection problem. By adapting state-of-the-art CNN architectures for binary classification ('crack' or 'no crack'), the method achieves high accuracy with minimal training data and few epochs, outperforming more complex, data-hungry models while remaining computationally efficient.

ABSTRACT

Computer vision for detecting building pathologies has interested researchers for quite some time. Vision-based crack detection is a non-destructive assessment technique, which can be useful especially for Cultural Heritage (CH) where strict regulations apply and, even simple, interventions are not permitted. Recently, shallow and deep machine learning architectures applied on various types of imagery are gaining ground. In this article a crack detection methodology for stone masonry walls is presented. In the proposed approach, crack detection is approached as an unsupervised anomaly detection problem on RGB (Red Green Blue) image patches. Towards this direction, some of the most popular state of the art CNN (Convolutional Neural Network) architectures are deployed and modified to binary classify the images or image patches by predicting a specific class for the tested imagery; 'Crack' or 'No crack', and detect and localize those cracks on the RGB imagery with high accuracy. Testing of the model was performed on various test sites and random images retrieved from the internet and collected by the authors and results suggested the high performance of specific networks compared to the rest, considering also the small numbers of epochs required for training. Those results met the accuracy delivered by more complex and computationally heavy approaches, requiring a large amount of data for training. Source code is available on GitHub https://github.com/pagraf/Crack-detection while datasets are available on Zenodo https://doi.org/10.5281/zenodo.6516913 .

Motivation & Objective

• To address the challenge of detecting structural cracks in cultural heritage stone masonry without relying on large annotated datasets.
• To develop a robust, unsupervised anomaly detection framework tailored for complex, heterogeneous masonry textures.
• To enable accurate and localized crack detection using only RGB images, suitable for non-invasive assessment in cultural heritage contexts.
• To reduce dependency on extensive training data and computational resources compared to supervised deep learning approaches.

Proposed method

• The method treats crack detection as an unsupervised anomaly detection problem on cropped RGB image patches.
• Pre-trained CNN architectures (e.g., ResNet, DenseNet) are fine-tuned for binary classification: 'crack' vs. 'no crack' on image patches.
• Anomaly scores are derived from the network’s internal features or activation patterns to identify deviations indicative of cracks.
• The model is trained end-to-end using only normal (crack-free) patches, avoiding the need for positive crack samples during training.
• Localization is achieved by applying Grad-CAM or similar attention-based methods to highlight anomalous regions in the input image.
• The approach is evaluated on diverse real-world masonry sites and internet-retrieved images to ensure generalization.

Experimental results

Research questions

• RQ1 Can unsupervised deep learning effectively detect cracks in complex, heterogeneous stone masonry surfaces without requiring annotated crack data?
• RQ2 Which pre-trained CNN architectures generalize best to unseen masonry textures for crack anomaly detection?
• RQ3 How does the performance of the unsupervised method compare to supervised approaches in terms of accuracy and data efficiency?
• RQ4 What is the minimum number of training epochs required to achieve reliable crack detection with this framework?

Key findings

• The proposed unsupervised method achieves detection accuracy comparable to more complex, supervised models that require large annotated datasets.
• Specific CNN architectures, such as ResNet and DenseNet variants, outperform others in detecting subtle cracks on textured masonry surfaces.
• The model generalizes well across diverse masonry sites and real-world images, including those collected from the internet.
• High localization accuracy is achieved using attention-based saliency maps, enabling precise identification of crack regions.
• The method requires only a small number of training epochs—demonstrating strong data efficiency and low computational cost.
• The source code and datasets are publicly available on GitHub and Zenodo, supporting reproducibility and further research.