[Paper Review] Real-time Distracted Driver Posture Classification

Yehya Abouelnaga, Hesham M. Eraqi+1 more|arXiv (Cornell University)|Jun 28, 2017

Hand Gesture Recognition SystemsComputer Science20 references81 citations

TL;DR

The paper introduces a new distracted-driver posture dataset and a genetically weighted ensemble of CNNs (AlexNet and InceptionV3) to achieve state-of-the-art posture classification accuracy, including a realtime, lighter variant.

ABSTRACT

In this paper, we present a new dataset for "distracted driver" posture estimation. In addition, we propose a novel system that achieves 95.98% driving posture estimation classification accuracy. The system consists of a genetically-weighted ensemble of Convolutional Neural Networks (CNNs). We show that a weighted ensemble of classifiers using a genetic algorithm yields in better classification confidence. We also study the effect of different visual elements (i.e. hands and face) in distraction detection and classification by means of face and hand localizations. Finally, we present a thinned version of our ensemble that could achieve a 94.29% classification accuracy and operate in a realtime environment.

Motivation & Objective

• Motivate the study of distracted driving and its safety impact.
• Create a challenging dataset of driver postures captured in real-world driving conditions.
• Develop a robust vision-based posture classifier using multiple CNNs and modalities (raw, face, hands, face+hands).
• Investigate the benefits of face and hand localization for posture attribution.
• Enable real-time deployment with a lighter ensemble while preserving high accuracy.

Proposed method

• Assemble a new AUC Distracted Driver dataset with 10 postures collected from 31 participants in 4 vehicles.
• Detect faces and hands in each frame to form multiple input sources (raw, face, hands, face+hands).
• Train AlexNet (from scratch) and InceptionV3 (via transfer learning) on each input source, yielding 8 models in total.
• Combine model outputs via a genetically weighted ensemble to optimize class probabilities, with weights learned by a genetic algorithm.
• Evaluate models using negative log-likelihood loss and accuracy on a 75/25 train/test split, and compare against a realtime, reduced ensemble variant.

Experimental results

Research questions

• RQ1 How accurately can distracted driving postures be classified from driver images using CNN ensembles?
• RQ2 What is the impact of using face, hands, or both as input modalities on posture classification accuracy?
• RQ3 Can a realtime, lighter model maintain high accuracy for practical deployment?
• RQ4 Does a genetic algorithm-based weighting of ensemble members improve classification performance over unweighted or majority voting ensembles?
• RQ5 How does the model handle confusions among similar postures (e.g., texting vs. talking) and what temporal cues might further improve performance?

Key findings

• GA-weighted ensemble achieves the highest reported accuracy of 95.98%.
• InceptionV3-based models attain higher accuracy overall than AlexNet, with Original inputs yielding the best per-model performance (95.17% for InceptionV3 Original).
• Hands-based models outperform face-based models, and combining face+hands provides marginal gains over hands alone due to detector failures.
• Realtime AlexNet-based ensemble (two AlexNets) achieves 94.29% accuracy with real-time CPU/GPU throughput.
• A simple majority vote ensemble reaches 95.77% accuracy, slightly below the GA-weighted ensemble.
• The best realtime system remains practical on CPU, and face/hands detectors improve accuracy though with higher overhead.