[Paper Review] Go-CaRD - Generic, Optical Car Part Recognition and Detection: Collection, Insights, and Applications.

Lukas Stappen, Xinchen Du+3 more|arXiv (Cornell University)|Jun 15, 2020

Advanced Neural Network Applications19 references6 citations

TL;DR

This paper introduces Go-CaRD, a generic, optical system for recognizing and detecting 29 interior and exterior vehicle parts using deep learning. It presents three new datasets and demonstrates state-of-the-art performance, achieving 93.67% F1 score for recognition (ResNet50) and 58.20% mAP for detection (best Darknet model), with insights into joint and transfer learning applications.

ABSTRACT

Systems for the automatic recognition and detection of automotive parts are crucial in several emerging research areas in the development of intelligent vehicles. They enable, for example, the detection and modelling of interactions between human and the vehicle. In this paper, we present three suitable datasets as well as quantitatively and qualitatively explore the efficacy of state-of-the-art deep learning architectures for the localisation of 29 interior and exterior vehicle regions, independent of brand, model, and environment. A ResNet50 model achieved an F1 score of 93.67 % for recognition, while our best Darknet model achieved an mAP of 58.20 % for detection. We also experiment with joint and transfer learning approaches and point out potential applications of our systems.

Motivation & Objective

• To develop a generic, brand- and model-agnostic system for recognizing and detecting vehicle parts using optical data.
• To address the lack of standardized datasets for interior and exterior vehicle part recognition across diverse environments.
• To evaluate state-of-the-art deep learning architectures for localizing 29 specific vehicle regions.
• To explore the effectiveness of joint and transfer learning in improving recognition and detection performance.
• To demonstrate practical applications of the system in intelligent vehicle research.

Proposed method

• Collection of three new, diverse datasets containing images of 29 interior and exterior vehicle parts across various brands, models, and environmental conditions.
• Application of ResNet50 for end-to-end vehicle part recognition with softmax classification and F1 score evaluation.
• Implementation of Darknet-based YOLO-style models for object detection, evaluated using mean Average Precision (mAP).
• Use of transfer learning by fine-tuning pre-trained ImageNet weights on the collected datasets to improve model generalization.
• Application of joint learning strategies to simultaneously optimize recognition and detection tasks.
• Evaluation of model performance using standard metrics: F1 score for recognition and mAP for detection.

Experimental results

Research questions

• RQ1 How effective are state-of-the-art deep learning models in recognizing 29 generic vehicle parts across diverse makes, models, and environments?
• RQ2 What is the performance gap between recognition and detection models in the context of vehicle part localization?
• RQ3 To what extent can transfer learning and joint learning improve recognition and detection accuracy on the proposed datasets?
• RQ4 How do different architectures (ResNet50 vs. Darknet) compare in terms of F1 score and mAP on this specific task?
• RQ5 What are the practical applications of a generic, optical vehicle part recognition and detection system in intelligent vehicle research?

Key findings

• The ResNet50 model achieved an F1 score of 93.67% for vehicle part recognition, demonstrating strong classification performance.
• The best-performing Darknet-based model achieved an mAP of 58.20% for object detection, indicating moderate detection capability on the dataset.
• Transfer learning significantly improved model convergence and performance, especially on limited or diverse data.
• Joint learning strategies showed promise in improving model generalization across recognition and detection tasks.
• The proposed datasets enable benchmarking of generic vehicle part recognition and detection systems independent of make, model, or environment.
• The system supports practical applications such as human-vehicle interaction modeling and intelligent vehicle behavior analysis.