[Paper Review] A Machine Learning Smartphone-based Sensing for Driver Behavior Classification

Sarra Ben Brahim, Hakim Ghazzai+2 more|arXiv (Cornell University)|Feb 1, 2022

Autonomous Vehicle Technology and Safety4 citations

TL;DR

This paper proposes a low-cost, smartphone-only driver behavior classification system using sensor fusion from accelerometers, gyroscopes, and GPS in the Carla simulator. It evaluates gradient boosting (LightGBM, XGBoost, CatBoost) and LSTM-based models on simulated time-series data, achieving up to 88% F1-score with LightGBM, demonstrating high accuracy in detecting normal, intermediate, aggressive, and dangerous driving behaviors under varying road and weather conditions.

ABSTRACT

Driver behavior profiling is one of the main issues in the insurance industries and fleet management, thus being able to classify the driver behavior with low-cost mobile applications remains in the spotlight of autonomous driving. However, using mobile sensors may face the challenge of security, privacy, and trust issues. To overcome those challenges, we propose to collect data sensors using Carla Simulator available in smartphones (Accelerometer, Gyroscope, GPS) in order to classify the driver behavior using speed, acceleration, direction, the 3-axis rotation angles (Yaw, Pitch, Roll) taking into account the speed limit of the current road and weather conditions to better identify the risky behavior. Secondly, after fusing inter-axial data from multiple sensors into a single file, we explore different machine learning algorithms for time series classification to evaluate which algorithm results in the highest performance.

Motivation & Objective

• To develop a low-cost, smartphone-only solution for real-time driver behavior classification using only built-in sensors.
• To address privacy and trust issues by simulating data instead of using real-world data collection.
• To evaluate and compare multiple machine learning models—specifically GBDT and LSTM architectures—for classifying driving behavior into four risk levels.
• To incorporate contextual factors such as speed limits, weather conditions, and traffic signs into the classification framework.
• To provide a generalized, scalable framework for fleet management and insurance applications using simulated, realistic driving scenarios.

Proposed method

• Data is collected from the Carla simulator using smartphone sensors (accelerometer, gyroscope, GPS) under varied road types, speed limits, and weather conditions.
• Sensor fusion combines multi-axis accelerometer and gyroscope data with GPS speed and direction into a unified time-series dataset.
• Time-series data is preprocessed via resampling to a uniform frequency and normalized for model input.
• A sliding window approach segments each trip into one-minute intervals for classification, enabling sub-trip behavior analysis.
• Three Gradient Boosting Decision Trees (LightGBM, XGBoost, CatBoost) and three LSTM-based architectures (LSTM, CNN-LSTM, FCN-LSTM) are trained and evaluated.
• Hyperparameter tuning is performed via random search, optimizing learning rate, tree depth, number of estimators, and L2 regularization.

Figure 1 : Proposed workflow of the AI-based driver behavior classification.

Experimental results

Research questions

• RQ1 Can smartphone-based sensing using only accelerometers, gyroscopes, and GPS effectively classify driver behavior into four risk levels (normal, intermediate, aggressive, dangerous)?
• RQ2 How do different machine learning models—particularly GBDT and LSTM-based architectures—perform in classifying driving behavior from simulated smartphone sensor data?
• RQ3 What is the impact of contextual factors such as speed limits and weather conditions on the accuracy of driver behavior classification?
• RQ4 Which hyperparameter configuration yields the highest performance in terms of F1-score and generalization across imbalanced data?
• RQ5 Does sensor fusion of multi-axis accelerometer and gyroscope data improve classification accuracy compared to single-sensor approaches?

Key findings

• LightGBM achieved the highest F1-score of 88% on the test set, outperforming XGBoost (83%) and CatBoost (81%) across all metrics.
• XGBoost was the fastest model to train, completing hyperparameter tuning and classification in just a few seconds.
• Among LSTM-based models, FCN-LSTM achieved the best F1-score of 74%, surpassing standard LSTM (58%) and CNN-LSTM (68%).
• The use of a sliding window approach enabled real-time classification of one-minute driving segments, with the model detecting dangerous behavior based on abnormal acceleration (e.g., over-speeding in AccelX).
• The framework demonstrated robust performance across imbalanced data using macro-averaging, with LightGBM achieving 88% macro F1-score and 88% accuracy.
• The simulation-based data collection method successfully mimicked real-world driving conditions, including variable speed limits and weather effects, enabling reliable behavior profiling without privacy risks.

Figure 2 : Smartphone sensors collected using the Carla simulator.