[Paper Review] Improving Train Track Safety using Drones, Computer Vision and Machine Learning

K. Anandha Kumar, Anuraag Kaashyap|arXiv (Cornell University)|Jun 4, 2020

Railway Engineering and Dynamics4 citations

TL;DR

This paper proposes a drone-based system integrating computer vision and machine learning to automate train track defect detection, using supervised and semi-supervised deep learning models—particularly a Convolutional Neural Network (CNN)—to classify track safety from aerial imagery. The approach achieves faster, more accurate defect detection than manual inspections, with a prototype system demonstrating improved consistency and scalability in identifying track defects using simulated data.

ABSTRACT

Millions of human casualties resulting from train accidents globally are caused by the inefficient, manual track inspections. Government agencies are seriously concerned about the safe operations of the rail industry after series of accidents reported across e USA and around the globe, mainly attributed to track defects. Casualties resulting from track defects result in billions of dollars loss in public and private investments and loss of revenue due to downtime, ultimately resulting in loss of the public's confidence. The manual, mundane, and expensive monitoring of rail track safety can be transform through the use of drones, computer vision, and machine learning. The primary goal of this study is to develop multiple algorithms that implement supervised and semi-supervised learning that accurately analyze whether a track is safe or unsafe based on simulated training data of train tracks. This includes being able to develop a Convolutional Neural Network that can identify track defects using supervised learning without having to specify a particular algorithm for detecting those defects, and that the new model would both speed up and improve the quality of the track defect detection process, accompanied with a computer vision image-processing algorithm. Our other goals included designing and building a prototype representation of train tracks to simulate track defects, to precisely and consistently conduct the visual inspection using drones. Ultimately, the goal demonstrates that the state of good repairs in railway tracks can be attained through the use of drones, computer vision and machine learning.

Motivation & Objective

• To address the high incidence of train accidents caused by undetected track defects through automated, scalable inspection.
• To reduce reliance on manual, time-consuming, and error-prone track inspections.
• To develop a robust, data-driven system using drones and machine learning for real-time track safety assessment.
• To design and validate a prototype simulation of track defects for consistent training and evaluation of vision models.
• To demonstrate that supervised and semi-supervised learning can effectively detect track defects without prior specification of defect types.

Proposed method

• Utilized drones to capture high-resolution aerial imagery of simulated railway tracks with embedded defects.
• Developed a Convolutional Neural Network (CNN) for supervised learning to classify track conditions as safe or unsafe based on visual features.
• Implemented a computer vision image-processing pipeline to preprocess and extract relevant features from drone-captured images.
• Designed a semi-supervised learning approach to improve model generalization with limited labeled data.
• Constructed a physical prototype of train tracks with controlled defect placements to simulate realistic inspection conditions.
• Trained and evaluated models on synthetic data representing various defect types, including misaligned rails and broken sleepers.

Experimental results

Research questions

• RQ1 Can a drone-based system with computer vision and machine learning effectively detect track defects with higher accuracy than manual inspection?
• RQ2 How well can a CNN model trained on simulated data generalize to real-world track defect detection?
• RQ3 To what extent does semi-supervised learning improve defect detection performance when labeled data is scarce?
• RQ4 Can a consistent and repeatable simulation of track defects be created to support reliable model training and evaluation?
• RQ5 Does the integration of drones, computer vision, and machine learning significantly reduce inspection time and cost while improving safety outcomes?

Key findings

• The proposed CNN model achieved high accuracy in classifying track safety from aerial images, demonstrating the feasibility of automated defect detection.
• The system significantly reduced inspection time compared to traditional manual methods, enabling faster response to potential hazards.
• Semi-supervised learning improved model performance with limited labeled data, enhancing scalability for large-scale deployment.
• The prototype simulation of track defects provided consistent and repeatable conditions for training and testing the vision models.
• The integration of drones, computer vision, and machine learning enabled a scalable, cost-effective, and reliable solution for improving rail track safety.
• The study confirms that deep learning models can detect track defects without requiring predefined defect-specific algorithms, increasing adaptability.