[Paper Review] Automatic Counting and Identification of Train Wagons Based on Computer Vision and Deep Learning

Rayson Laroca, Alessander Cidral Boslooper+1 more|arXiv (Cornell University)|Oct 30, 2020

Vehicle License Plate Recognition4 citations

TL;DR

This paper proposes a low-cost, deep learning-based computer vision system for automatic counting and identification of train wagons using Full HD video, achieving 100% counting accuracy and 99.7% recognition rate with automatic rejection of 11.6% damaged codes—outperforming costly RFID systems while running efficiently on low-end hardware.

ABSTRACT

In this work, we present a robust and efficient solution for counting and identifying train wagons using computer vision and deep learning. The proposed solution is cost-effective and can easily replace solutions based on radiofrequency identification (RFID), which are known to have high installation and maintenance costs. According to our experiments, our two-stage methodology achieves impressive results on real-world scenarios, i.e., 100% accuracy in the counting stage and 99.7% recognition rate in the identification one. Moreover, the system is able to automatically reject some of the train wagons successfully counted, as they have damaged identification codes. The results achieved were surprising considering that the proposed system requires low processing power (i.e., it can run in low-end setups) and that we used a relatively small number of images to train our Convolutional Neural Network (CNN) for character recognition. The proposed method is registered, under number BR512020000808-9, with the National Institute of Industrial Property (Brazil).

Motivation & Objective

• To develop a cost-effective alternative to RFID-based wagon tracking systems that reduces installation and maintenance costs.
• To address challenges in automatic wagon counting and identification due to variable lighting, font styles, damaged codes, and camera angles.
• To create a system robust to real-world conditions such as dirt, corrosion, and low-resolution text projections on corrugated surfaces.
• To enable deployment on low-end hardware by minimizing computational requirements without sacrificing accuracy.
• To automatically detect and reject wagons with illegible or damaged identification codes for improved system reliability.

Proposed method

• A two-stage methodology: first, object detection via YOLOv4-tiny to localize wagon regions in Full HD video frames.
• Second, a custom Convolutional Neural Network (CNN) for end-to-end scene text recognition of alphanumeric wagon codes.
• The system uses data augmentation and transfer learning to train effectively on a relatively small dataset of 14,935 images.
• A rejection mechanism identifies and excludes wagons with damaged or illegible codes based on OCR output length and coding standards.
• Results from multiple cameras are combined using heuristic rules to improve recognition accuracy in multi-view scenarios.
• The system processes 16 FPS on a high-end GPU, enabling real-time deployment on embedded devices.

Experimental results

Research questions

• RQ1 Can a computer vision system achieve 100% counting accuracy for train wagons in real-world outdoor conditions?
• RQ2 What recognition accuracy can be achieved for wagon identification under variable lighting, font, and code damage conditions?
• RQ3 To what extent can a deep learning model trained on a limited dataset generalize to real-world wagon code variations?
• RQ4 How effective is the automatic rejection of wagons with damaged or illegible identification codes?
• RQ5 Can multi-camera fusion improve recognition accuracy in complex scenarios?

Key findings

• The system achieved 100% counting accuracy across all evaluated videos, correctly identifying and tracking every wagon in the train composition.
• The recognition rate for legible wagon codes reached 99.7%, even with variable font styles, colors, and partial occlusions.
• The system automatically rejected 11.6% of wagons due to damaged or illegible identification codes, improving data quality.
• The system processed 16 frames per second on a high-end GPU, indicating feasibility for real-time deployment on low-cost embedded hardware.
• Multi-camera setups reduced recognition errors from 8 to 2 by combining results from both sides of the track using heuristic fusion.
• The system remains robust even with low-resolution, noisy, or slanted text projections due to corrugated wagon surfaces.