[Paper Review] Learning Accurate, Comfortable and Human-like Driving

Simon Hecker, Dengxin Dai+1 more|arXiv (Cornell University)|Mar 26, 2019

Autonomous Vehicle Technology and SafetyEngineering49 references20 citations

TL;DR

This paper proposes an end-to-end deep learning framework that improves autonomous driving by jointly optimizing for accuracy, passenger comfort, and human-likeness using high-fidelity map data, sequence-based learning, and adversarial training. The model achieves superior performance on the Drive360 dataset, outperforming prior methods in accuracy, ride comfort, and human-like driving behavior.

ABSTRACT

Autonomous vehicles are more likely to be accepted if they drive accurately, comfortably, but also similar to how human drivers would. This is especially true when autonomous and human-driven vehicles need to share the same road. The main research focus thus far, however, is still on improving driving accuracy only. This paper formalizes the three concerns with the aim of accurate, comfortable and human-like driving. Three contributions are made in this paper. First, numerical map data from HERE Technologies are employed for more accurate driving; a set of map features which are believed to be relevant to driving are engineered to navigate better. Second, the learning procedure is improved from a pointwise prediction to a sequence-based prediction and passengers' comfort measures are embedded into the learning algorithm. Finally, we take advantage of the advances in adversary learning to learn human-like driving; specifically, the standard L1 or L2 loss is augmented by an adversary loss which is based on a discriminator trained to distinguish between human driving and machine driving. Our model is trained and evaluated on the Drive360 dataset, which features 60 hours and 3000 km of real-world driving data. Extensive experiments show that our driving model is more accurate, more comfortable and behaves more like a human driver than previous methods. The resources of this work will be released on the project page.

Motivation & Objective

• To formalize and integrate three key objectives in autonomous driving: accuracy, passenger comfort, and human-likeness.
• To improve driving accuracy by leveraging high-fidelity numerical map data from HERE Technologies and engineering map features relevant to driving decisions.
• To enhance ride comfort by modeling driving as a sequence-based prediction task and incorporating longitudinal and lateral oscillation minimization into the loss function.
• To promote human-like driving behavior by introducing an adversarial loss that trains a discriminator to distinguish human from machine driving, encouraging the model to mimic human patterns.
• To evaluate the integrated framework on a real-world driving dataset and demonstrate its superiority over prior methods in all three dimensions.

Proposed method

• The model uses high-resolution numerical map data from HERE Technologies to extract driving-relevant features such as road type, speed limits, traffic lights, crosswalks, and intersection geometry.
• A sequence-based learning approach replaces pointwise regression, modeling temporal dependencies in steering and speed predictions to improve smoothness and comfort.
• Passenger comfort is embedded via a loss function that minimizes both longitudinal and lateral acceleration oscillations, reducing motion sickness risk.
• An adversarial training component introduces a discriminator network trained to detect whether driving trajectories are from human drivers or the learned model, with the generator trained to fool it.
• The overall training objective combines three losses: a standard regression loss for accuracy, a comfort loss for ride smoothness, and an adversarial loss for human-likeness.
• The framework is trained and evaluated on the Drive360 dataset, comprising 60 hours and 3,000 km of real-world driving data with rich sensor and map annotations.

Experimental results

Research questions

• RQ1 Can high-fidelity map data significantly improve the accuracy and robustness of end-to-end driving models?
• RQ2 How does sequence-based learning with comfort-aware loss functions affect ride quality and passenger motion sickness compared to pointwise regression?
• RQ3 To what extent can adversarial training be used to learn human-like driving behaviors without explicit imitation of human demonstrations?
• RQ4 Can a unified learning framework simultaneously optimize for driving accuracy, passenger comfort, and human-likeness in a single end-to-end model?
• RQ5 What are the failure modes of current driving models in challenging road scenarios, and how can map-based feature engineering help diagnose and improve performance?

Key findings

• The integration of HERE-provided map features significantly improved driving accuracy, particularly in complex maneuvers such as lane changes and turns, as shown in qualitative comparisons.
• The sequence-based learning approach reduced longitudinal and lateral oscillations, resulting in smoother driving trajectories that correlate with reduced passenger discomfort.
• The adversarial loss component successfully encouraged the model to produce driving behaviors that were more difficult for a discriminator to distinguish from human driving, improving human-likeness.
• Error diagnosis revealed that the model performs worse in high-complexity scenarios such as intersections and winding roads, highlighting areas for future improvement.
• Quantitative evaluation showed that the proposed model outperformed previous methods in all three metrics: lower steering and speed prediction errors, reduced oscillation amplitudes, and higher human-likeness scores.
• The model demonstrated superior generalization on unseen road attributes, with normalized error rates showing consistent performance across speed limits, road types, and intersection types.