[Paper Review] Roof fall hazard detection with convolutional neural networks using transfer learning

E. Isleyen, Şebnem Düzgün+1 more|arXiv (Cornell University)|Nov 12, 2020

Infrastructure Maintenance and Monitoring4 citations

TL;DR

This paper proposes a transfer learning-based convolutional neural network (CNN) for automated detection of roof fall hazards in mining and tunneling environments caused by high horizontal stresses. Using limited real-world images of roof conditions, the model achieves 86% accuracy in classifying hazardous vs. non-hazardous conditions, with integrated gradients revealing that the network learns geologically meaningful features consistent with expert judgment.

ABSTRACT

Roof falls due to geological conditions are major safety hazards in mining and tunneling industries, causing lost work times, injuries, and fatalities. Several large-opening limestone mines in the Eastern and Midwestern United States have roof fall problems caused by high horizontal stresses. The typical hazard management approach for this type of roof fall hazard relies heavily on visual inspections and expert knowledge. In this study, we propose an artificial intelligence (AI) based system for the detection roof fall hazards caused by high horizontal stresses. We use images depicting hazardous and non-hazardous roof conditions to develop a convolutional neural network for autonomous detection of hazardous roof conditions. To compensate for limited input data, we utilize a transfer learning approach. In transfer learning, an already-trained network is used as a starting point for classification in a similar domain. Results confirm that this approach works well for classifying roof conditions as hazardous or safe, achieving a statistical accuracy of 86%. However, accuracy alone is not enough to ensure a reliable hazard management system. System constraints and reliability are improved when the features being used by the network are understood. Therefore, we used a deep learning interpretation technique called integrated gradients to identify the important geologic features in each image for prediction. The analysis of integrated gradients shows that the system mimics expert judgment on roof fall hazard detection. The system developed in this paper demonstrates the potential of deep learning in geological hazard management to complement human experts, and likely to become an essential part of autonomous tunneling operations in those cases where hazard identification heavily depends on expert knowledge.

Motivation & Objective

• To develop an AI-based system for automated detection of roof fall hazards in mining and tunneling operations.
• To address the challenge of limited labeled training data for roof condition images through transfer learning.
• To improve system reliability by interpreting the features driving the model's predictions using integrated gradients.
• To validate that the deep learning model's decision-making aligns with expert geological knowledge in hazard assessment.

Proposed method

• Fine-tune a pre-trained convolutional neural network (CNN) on a small dataset of roof condition images collected from limestone mines.
• Apply transfer learning by initializing the CNN with ImageNet weights and adapting the final layers for binary classification (hazardous vs. safe roof conditions).
• Use data augmentation techniques to artificially expand the training set and improve generalization.
• Employ integrated gradients—a deep learning interpretation method—to identify and visualize the most influential image regions for each prediction.
• Train the model using cross-entropy loss and Adam optimization with a learning rate schedule.
• Evaluate model performance using standard metrics including accuracy, precision, recall, and F1-score on a held-out test set.

Experimental results

Research questions

• RQ1 Can a transfer learning-based CNN effectively classify roof fall hazards from limited real-world images in mining environments?
• RQ2 How does the model's performance compare to traditional visual inspection methods in terms of accuracy and reliability?
• RQ3 Which geologic features in roof images are most influential in the model's hazard predictions?
• RQ4 To what extent does the model's decision-making align with expert geological judgment?
• RQ5 Can explainability techniques like integrated gradients enhance trust and usability of AI in safety-critical mining applications?

Key findings

• The proposed transfer learning-based CNN achieved a classification accuracy of 86% in distinguishing hazardous from non-hazardous roof conditions.
• Integrated gradients analysis revealed that the model focuses on geologically relevant features such as joint patterns, fractures, and stress-induced deformation—consistent with expert visual assessments.
• The model's attention to specific structural features indicates it learns meaningful representations rather than spurious correlations.
• The system demonstrates potential for integration into autonomous tunneling operations to support human experts in real-time hazard detection.
• The use of explainable AI techniques enhances transparency and supports trust in the model’s predictions for safety-critical applications.
• Despite high accuracy, the model’s performance is constrained by dataset size and domain-specific variability, highlighting the need for further data collection and domain adaptation.