[Paper Review] Material Classification using Neural Networks

Anca Sticlaru|arXiv (Cornell University)|Oct 17, 2017

Industrial Vision Systems and Defect Detection23 references3 citations

TL;DR

This BSc thesis investigates material classification using deep neural networks, evaluating nine CNN architectures combined with linear SVMs on four benchmark datasets. It demonstrates that transfer learning—using features from the penultimate fully connected layer—boosts performance, achieving up to 92.5% mean average precision, with significant gains in large-scale datasets due to improved capture of shading and reflectance characteristics.

ABSTRACT

The recognition and classification of the diversity of materials that exist in the environment around us are a key visual competence that computer vision systems focus on in recent years. Understanding the identification of materials in distinct images involves a deep process that has made usage of the recent progress in neural networks which has brought the potential to train architectures to extract features for this challenging task. This project uses state-of-the-art Convolutional Neural Network (CNN) techniques and Support Vector Machine (SVM) classifiers in order to classify materials and analyze the results. Building on various widely used material databases collected, a selection of CNN architectures is evaluated to understand which is the best approach to extract features in order to achieve outstanding results for the task. The results gathered over four material datasets and nine CNNs outline that the best overall performance of a CNN using a linear SVM can achieve up to ~92.5% mean average precision, while applying a new relevant direction in computer vision, transfer learning. By limiting the amount of information extracted from the layer before the last fully connected layer, transfer learning aims at analyzing the contribution of shading information and reflectance to identify which main characteristics decide the material category the image belongs to. In addition to the main topic of my project, the evaluation of the nine different CNN architectures, it is questioned if, by using the transfer learning instead of extracting the information from the last convolutional layer, the total accuracy of the system created improves. The results of the comparison emphasize the fact that the accuracy and performance of the system improve, especially in the datasets which consist of a large number of images.

Motivation & Objective

• To evaluate the effectiveness of various pre-trained CNN architectures for material classification in computer vision.
• To assess whether transfer learning, by extracting features from the penultimate fully connected layer, improves classification accuracy over standard last convolutional layer features.
• To analyze the contribution of shading and reflectance information to material category recognition.
• To compare the performance of linear SVMs with different CNN feature extractors across multiple material datasets.
• To determine the optimal feature extraction strategy for material classification in large-scale image collections.

Proposed method

• The study employs nine state-of-the-art pre-trained Convolutional Neural Network (CNN) architectures for feature extraction from material images.
• Features are extracted from the last convolutional layer and from the layer before the final fully connected layer to evaluate transfer learning's impact.
• A linear Support Vector Machine (SVM) is used as the classifier to map extracted features to material categories.
• The model is trained and evaluated on four widely used material databases to ensure generalization across datasets.
• Performance is measured using mean average precision (mAP), with ablation studies comparing feature extraction strategies.
• Transfer learning is applied by limiting the information extracted from the penultimate layer to isolate the role of reflectance and shading in material recognition.

Experimental results

Research questions

• RQ1 Does using features from the penultimate fully connected layer via transfer learning improve material classification accuracy compared to using features from the last convolutional layer?
• RQ2 How do different CNN architectures perform in extracting discriminative features for material classification across diverse datasets?
• RQ3 To what extent do shading and reflectance characteristics contribute to accurate material category prediction?
• RQ4 Is the performance gain from transfer learning more pronounced in large-scale datasets?
• RQ5 Can a linear SVM effectively classify materials when combined with deep features from pre-trained CNNs?

Key findings

• The best-performing system achieved a mean average precision (mAP) of approximately 92.5% using transfer learning with a linear SVM.
• Transfer learning significantly improved classification accuracy, especially on large-scale datasets, by better capturing reflectance and shading cues.
• The penultimate fully connected layer yielded more discriminative features than the last convolutional layer for material classification.
• Performance gains from transfer learning were most evident in datasets with higher image diversity and larger sample sizes.
• Among the nine evaluated CNN architectures, ResNet and DenseNet variants showed strong generalization across all datasets.
• The study confirms that reflectance and shading are critical visual cues for material recognition, and their effective extraction enhances classification performance.