[Paper Review] Driver Drowsiness Detection Model Using Convolutional Neural Networks Techniques for Android Application

Rateb Jabbar, Mohammed Shinoy+4 more|arXiv (Cornell University)|Jan 17, 2020

Sleep and Work-Related FatiguePsychology20 references195 citations

TL;DR

The paper proposes a lightweight CNN-based driver drowsiness detection model using facial landmarks for Android, achieving average accuracy above 83% with a model size under 75 KB.

ABSTRACT

A sleepy driver is arguably much more dangerous on the road than the one who is speeding as he is a victim of microsleeps. Automotive researchers and manufacturers are trying to curb this problem with several technological solutions that will avert such a crisis. This article focuses on the detection of such micro sleep and drowsiness using neural network based methodologies. Our previous work in this field involved using machine learning with multi-layer perceptron to detect the same. In this paper, accuracy was increased by utilizing facial landmarks which are detected by the camera and that is passed to a Convolutional Neural Network (CNN) to classify drowsiness. The achievement with this work is the capability to provide a lightweight alternative to heavier classification models with more than 88% for the category without glasses, more than 85% for the category night without glasses. On average, more than 83% of accuracy was achieved in all categories. Moreover, as for model size, complexity and storage, there is a marked reduction in the new proposed model in comparison to the benchmark model where the maximum size is 75 KB. The proposed CNN based model can be used to build a real-time driver drowsiness detection system for embedded systems and Android devices with high accuracy and ease of use.

Motivation & Objective

• Motivate the need for real-time, low-resource drowsiness detection to prevent microsleeps.
• Explore a CNN-based approach that leverages facial landmarks for robustness to variations.
• Develop a compact model suitable for embedded/Android devices without heavy hardware.
• Evaluate accuracy, model size, and runtime performance against existing benchmarks.

Proposed method

• Preprocess NTHU Driver Drowsiness Dataset to extract 600,000 images via frame sampling and data augmentation.
• Detect 68 facial landmarks per image using Dlib/OpenCV to represent salient facial regions.
• Propose D2CNN-FLD: a five-layer CNN with Leaky ReLU activations, dropout (≈25%), MaxPooling, and a Softmax output for 2-class classification.
• Train the model on normalized landmark coordinates with Min-Max scaling and a 2-class softmax output.
• Implement Android pipeline where JNI transfers landmark data from a Java-based app to the CNN for real-time inference.

Experimental results

Research questions

• RQ1 Can a lightweight CNN model using facial landmark coordinates achieve competitive drowsiness detection accuracy on the NTHU dataset?
• RQ2 How does the proposed D2CNN-FLD compare to prior MLP-based and heavier CNN/RCNN baselines in accuracy and resource usage?
• RQ3 What is the impact of lighting and eyewear (glasses/sunglasses) on detection performance?
• RQ4 Is the model compact enough to run in real-time on mobile/embedded devices without GPU acceleration?

Key findings

• D2CNN-FLD achieves an overall accuracy of 83.3% across driving scenarios.
• The proposed model size is 0.075 MB (~75 KB), substantially smaller than benchmark CNN/RCNN models.
• D2CNN-FLD shows improved accuracy over the prior D2MLP-FLD by about 2.5 percentage points in aggregate results.
• Detection time varies by device; e.g., 38.45 ms on NVIDIA Quadro P4000 and 142 ms on Samsung Galaxy S8 Plus for D2CNN-FLD.