[Paper Review] Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network

Zhun Fan, Yuming Wu+2 more|arXiv (Cornell University)|Feb 1, 2018

Infrastructure Maintenance and MonitoringEngineering22 references227 citations

TL;DR

A CNN-based structured-prediction method detects pavement cracks at pixel level, handling imbalanced data, and outperforms several baselines on CFD and AigleRN datasets, with cross-database generalization explored.

ABSTRACT

Automated pavement crack detection is a challenging task that has been researched for decades due to the complicated pavement conditions in real world. In this paper, a supervised method based on deep learning is proposed, which has the capability of dealing with different pavement conditions. Specifically, a convolutional neural network (CNN) is used to learn the structure of the cracks from raw images, without any preprocessing. Small patches are extracted from crack images as inputs to generate a large training database, a CNN is trained and crack detection is modeled as a multi-label classification problem. Typically, crack pixels are much fewer than non-crack pixels. To deal with the problem with severely imbalanced data, a strategy with modifying the ratio of positive to negative samples is proposed. The method is tested on two public databases and compared with five existing methods. Experimental results show that it outperforms the other methods.

Motivation & Objective

• Motivate automated, robust crack detection across diverse pavement conditions.
• Learn crack structure directly from raw images without preprocessing.
• Model crack detection as a multi-label structured prediction problem per image patch.
• Address severe class imbalance to improve crack pixel recall without sacrificing precision.

Proposed method

• Use patches centered at each pixel (27x27) as CNN inputs to predict a 5x5 centered structure (s=5).
• Model multi-label output with sigmoid activations and cross-entropy loss, plus L2 regularization and dropout for regularization.
• Train separate CNNs for 3-channel (CFD) and 1-channel (AigleRN) inputs; apply data augmentation via patch sampling and mean-variance normalization.
• Adjust positive/negative sample ratio during training (typical 1:3) to mitigate severe class imbalance.
• Aggregate per-pixel outputs into an image-wide probability map by summing overlapping predictions and normalizing.

Experimental results

Research questions

• RQ1 Can structured prediction via CNN outperform pixel-wise CNN classifiers for crack detection?
• RQ2 How does output structure size (s) affect detection performance?
• RQ3 What is the impact of imbalanced training data on crack detection, and how can ratio adjustments improve results?
• RQ4 Do the learned models generalize across different pavement datasets (cross-database evaluation)?

Key findings

• The proposed CNN with structured prediction achieves higher F1 than Canny, local thresholding, and CrackForest on CFD (F1=0.9244).
• On CFD, the method attains Pr=0.9119, Re=0.9481.
• On AigleRN, it achieves Pr=0.9178, Re=0.8812, F1=0.8954, outperforming Canny, local thresholding, FFA, and MPS.
• Structured prediction (s=5) outperforms pixel-level classification (s=1) in robustness to noise and in producing coherent crack maps.
• Adjusting the positive-to-negative sampling ratio during training (R between 2 and 5 for CFD, 2–3 for AigleRN) improves F1 balance between precision and recall.
• Cross-database tests show trade-offs: models trained on AigleRN produce thinner cracks with high precision and lower recall, while CFD-trained models yield thicker cracks with higher recall; hybrid training improves generalization.