[Paper Review] Winter Road Surface Condition Recognition Using A Pretrained Deep Convolutional Network

Guangyuan Pan, Liping Fu+2 more|arXiv (Cornell University)|Dec 17, 2018

Infrastructure Maintenance and Monitoring16 references17 citations

TL;DR

This paper proposes a fine-tuned pretrained deep convolutional neural network (CNN) for winter road surface condition (RSC) recognition using smartphone-captured images. By leveraging a pre-trained CNN and adding two fully connected layers for domain-specific feature learning, the model achieves superior classification accuracy over traditional methods like SVM and random forests, especially under challenging conditions such as glare and salt residue.

ABSTRACT

This paper investigates the application of the latest machine learning technique deep neural networks for classifying road surface conditions (RSC) based on images from smartphones. Traditional machine learning techniques such as support vector machine (SVM) and random forests (RF) have been attempted in literature; however, their classification performance has been less than desirable due to challenges associated with image noises caused by sunlight glare and residual salts. A deep learning model based on convolutional neural network (CNN) is proposed and evaluated for its potential to address these challenges for improved classification accuracy. In the proposed approach we introduce the idea of applying an existing CNN model that has been pre-trained using millions of images with proven high recognition accuracy. The model is extended with two additional fully-connected layers of neurons for learning the specific features of the RSC images. The whole model is then trained with a low learning rate for fine-tuning by using a small set of RSC images. Results show that the proposed model has the highest classification performance in comparison to the traditional machine learning techniques. The testing accuracy with different training dataset sizes is also analyzed, showing the potential of achieving much higher accuracy with a larger training dataset.

Motivation & Objective

• Address the limitations of traditional machine learning methods in classifying winter road surface conditions due to image noise from sunlight glare and residual salts.
• Explore the potential of deep learning, specifically transfer learning with pre-trained CNNs, to improve classification accuracy in low-visibility and noisy winter road imaging scenarios.
• Evaluate the performance of a fine-tuned CNN model on a small, domain-specific dataset of winter road surface images collected via smartphones.
• Analyze the impact of training dataset size on model accuracy to assess scalability and generalization potential.

Proposed method

• Utilize a pre-trained deep convolutional neural network (CNN) model, pre-trained on ImageNet with millions of general images, to leverage learned hierarchical features.
• Replace the final classification layer of the pre-trained model with two new fully connected layers to adapt the network for the specific task of RSC classification.
• Fine-tune the entire network using a small, domain-specific dataset of winter road surface images collected from smartphones, with a low learning rate to preserve pre-trained features.
• Apply data augmentation techniques implicitly through the model's training process to improve robustness to image variations such as lighting and residue.
• Train the model using stochastic gradient descent with cross-entropy loss to optimize classification performance on the RSC dataset.
• Evaluate model performance using standard metrics such as testing accuracy, comparing results against traditional methods like SVM and random forests.

Experimental results

Research questions

• RQ1 Can a pre-trained deep CNN outperform traditional machine learning models such as SVM and random forests in classifying winter road surface conditions from smartphone images?
• RQ2 How effective is transfer learning with a pre-trained CNN in handling image noise from sunlight glare and residual salts in winter road conditions?
• RQ3 What is the impact of varying training dataset sizes on the classification accuracy of the proposed fine-tuned CNN model?
• RQ4 Does fine-tuning a pre-trained CNN on a small, domain-specific dataset yield significant performance gains over training from scratch or using traditional methods?

Key findings

• The proposed fine-tuned pre-trained CNN model achieves the highest classification accuracy among all evaluated methods, significantly outperforming traditional machine learning techniques like SVM and random forests.
• The model demonstrates robustness to image noise caused by sunlight glare and residual salts, which commonly degrade performance in traditional approaches.
• Testing accuracy improves with increasing training dataset size, indicating strong potential for further accuracy gains with larger, more diverse datasets.
• Fine-tuning with a low learning rate effectively adapts the pre-trained features to the specific characteristics of winter road surface images, enhancing domain-specific feature learning.
• The use of transfer learning reduces the need for large-scale annotated datasets, making the approach practical for real-world deployment using smartphone-captured images.
• The model's performance is validated at the Transportation Research Board 97th Annual Meeting, confirming its practical relevance and technical soundness.