[Paper Review] Behavioral Research and Practical Models of Drivers' Attention

Iuliia Kotseruba, John K. Tsotsos|arXiv (Cornell University)|Apr 12, 2021

Human-Automation Interaction and Safety446 references4 citations

TL;DR

This paper synthesizes over 175 behavioral studies and 100 practical papers to analyze how internal (e.g., age, experience) and external (e.g., automation, distractions) factors influence drivers' visual attention. It identifies key limitations in current attention modeling—such as lack of task representation, poor safety validation, and limited cognitive modeling—and proposes a cross-disciplinary framework to improve driver monitoring, ADAS, and highly automated driving systems by aligning with human attention patterns.

ABSTRACT

Driving is a routine activity for many, but it is far from simple. Drivers deal with multiple concurrent tasks, such as keeping the vehicle in the lane, observing and anticipating the actions of other road users, reacting to hazards, and dealing with distractions inside and outside the vehicle. Failure to notice and respond to the surrounding objects and events can cause accidents. The ongoing improvements of the road infrastructure and vehicle mechanical design have made driving safer overall. Nevertheless, the problem of driver inattention has remained one of the primary causes of accidents. Therefore, understanding where the drivers look and why they do so can help eliminate sources of distractions and identify unsafe attention patterns. Research on driver attention has implications for many practical applications such as policy-making, improving driver education, enhancing road infrastructure and in-vehicle infotainment systems, as well as designing systems for driver monitoring, driver assistance, and automated driving. This report covers the literature on changes in drivers' visual attention distribution due to factors, internal and external to the driver. Aspects of attention during driving have been explored across multiple disciplines, including psychology, human factors, human-computer interaction, intelligent transportation, and computer vision, each offering different perspectives, goals, and explanations for the observed phenomena. We link cross-disciplinary theoretical and behavioral research on driver's attention to practical solutions. Furthermore, limitations and directions for future research are discussed. This report is based on over 175 behavioral studies, nearly 100 practical papers, 20 datasets, and over 70 surveys published since 2010. A curated list of papers used for this report is available at \url{https://github.com/ykotseruba/attention_and_driving}.

Motivation & Objective

• To identify and synthesize the key factors—both internal (e.g., age, experience) and external (e.g., automation, distractions)—that influence drivers’ visual attention allocation.
• To bridge gaps between behavioral research in psychology and human factors and practical applications in ADAS, DMS, and autonomous driving systems.
• To address critical limitations in current attention modeling, including lack of task representation, insufficient safety validation, and inadequate cognitive modeling.
• To provide a cross-disciplinary foundation for developing more reliable, transparent, and safety-critical attention models in intelligent transportation systems.
• To guide future research by identifying open problems, data inconsistencies, and methodological shortcomings in gaze-based attention modeling.

Proposed method

• Systematic literature review of 175 behavioral studies, 100 practical papers, 20 datasets, and 70 surveys from the last decade.
• Integration of findings across disciplines: psychology, human factors, human-computer interaction, computer vision, and intelligent transportation systems.
• Use of eye-tracking data as a proxy for attention, with critical evaluation of its limitations and reliability in real-world driving contexts.
• Analysis of attention modeling approaches in AI and computer vision, focusing on their failure to incorporate top-down task context and cognitive constraints.
• Evaluation of current ADAS and DMS systems through the lens of attention alignment, highlighting gaps in warning design and context-aware adaptation.
• Proposed framework for future models emphasizing task-based attention, cognitive resource allocation, and sequential processing, with emphasis on transparency and safety validation.

Experimental results

Research questions

• RQ1 How do internal factors such as driving experience, age, and fatigue affect drivers’ visual attention distribution?
• RQ2 To what extent do external factors like in-vehicle infotainment, roadside billboards, and vehicle automation influence attention allocation?
• RQ3 Why do current AI-based attention models in autonomous driving fail to replicate human-like attention patterns despite high accuracy in gaze prediction?
• RQ4 What are the key methodological shortcomings in data collection, processing, and validation of gaze datasets used in attention modeling?
• RQ5 How can attention models be improved to support cooperative driver assistance and safe automation by aligning with human cognitive processes and decision-making?

Key findings

• Driver inattention remains a leading cause of crashes, with distractions from in-vehicle systems and dynamic roadside displays significantly impairing attention allocation.
• Gaze is a strong but incomplete proxy for attention; many studies fail to validate gaze data with behavioral or control data, leading to ambiguous conclusions.
• Most current AI-based attention models focus on saliency and implicit dependencies rather than explicit task representation, limiting their interpretability and safety utility.
• Only a small fraction of attention models consider driving experience or environmental context, despite strong empirical evidence linking these to attention patterns.
• Existing ADAS systems rely on fixed, non-adaptive warnings (e.g., audio-tactile) that do not account for driver attention state or scene context, reducing effectiveness.
• Simulation-based validation of attention models lacks real-world validation, and accident datasets often lack driver gaze data, making it impossible to assess whether better attention could have prevented crashes.