[Paper Review] Exploring the Impacts of Background Noise on Auditory Stimuli of Audio-Visual eHMIs for Hearing, Deaf, and Hard-of-Hearing People

Wenge Xu, Foroogh Hajiseyedjavadi+3 more|arXiv (Cornell University)|Jan 19, 2026

Human-Automation Interaction and Safety0 citations

TL;DR

The study uses a VR crossing task to examine how quiet vs loud background noise and auditory eHMI types (baseline, bell, speech) affect crossing experience and behavior for hearing and DHH pedestrians.

ABSTRACT

External Human-Machine Interfaces (eHMIs) have been proposed to enhance communication between automated vehicles (AVs) and pedestrians, with growing interest in multi-modal designs such as audio-visual eHMIs. Just as poor lighting can impair visual cues, a loud background noise may mask the auditory stimuli. However, its effects within these systems have not been examined, and little is known about how pedestrians -- particularly Deaf and Hard-of-Hearing (DHH) people -- perceive different types of auditory stimuli. We conducted a virtual reality study (Hearing N=25, DHH N=11) to examine the effects of background noise (quiet and loud) on auditory stimuli (baseline, bell, speech) within an audio-visual eHMI. Results revealed that: (1) Crossing experiences of DHH pedestrians significantly differ from Hearing pedestrians. (2) Loud background noise adversely affects pedestrians' crossing experiences. (3) Providing an additional auditory eHMI (bell/speech) improves crossing experiences. We outlined four practical implications for future eHMI design and research.

Motivation & Objective

• Motivate inclusive eHMI design by assessing accessibility for Deaf and Hard-of-Hearing pedestrians.
• Investigate how background noise interacts with auditory stimuli in audio-visual eHMIs during street crossing.
• Compare crossing experiences and behaviors between Hearing and DHH participants under different noise and auditory conditions.
• Provide design implications for multi-modal eHMI to improve safety and inclusivity.

Proposed method

• Use a VR street-crossing scenario in Unity to simulate urban road crossing with autonomous vehicles.
• Employ a 2x3 within-subjects design: Background Noise (Quiet vs. Loud) and Auditory Stimuli (Baseline, Bell, Speech).
• Implement an audio-visual eHMI on vehicles with a fixed visual cue plus auditory cues (Speech or Bell) and a Baseline natural vehicle sound.
• Manipulate two ambient urban sound conditions (Quiet ~58-64 dB; Loud ~70-79 dB) and randomize playback onset to avoid predictability.
• Measure subjective crossing experience (NASA-TLX, Trust in Automation, Perceived Safety, Acceptance) and objective behaviors (eye gaze, Step Into Road Time, Early Step Into Road Count).
• Analyze data with nonparametric LD-type ANOVAs (nparLD) due to non-normality and small samples; apply Bonferroni corrections for post-hoc tests.

Figure 1. Setup of the virtual simulation environment: The left figure shows a top-down view, while the right figure shows a leftward view. a) The starting position of the participant, which is 4.7 m from the road. b) The 2 m wide grey pavement area, which has a trigger to inform one of the AVs to y

Experimental results

Research questions

• RQ1 RQ1: How do hearing versus DHH participants differ in crossing experience and behavior across conditions?
• RQ2 RQ2: How does background noise level affect pedestrians' crossing experience and behavior?
• RQ3 RQ3: How do auditory stimuli (Baseline, Bell, Speech) of the audio-visual eHMI affect experience and behavior?
• RQ4 RQ4: Do multi-modal eHMI cues provide practical improvements for accessibility and safety in AV crossings?

Key findings

• Loud background noise significantly affects trust in the eHMI across participants.
• Auditory stimuli influence trust and perceived safety, with significant main effects for Auditory Stimuli.
• Crossing experiences differ between Hearing and DHH groups, indicating differential accessibility considerations.
• Providing an additional auditory eHMI (bell or speech) improves crossing experiences under certain conditions.
• Eye-gaze and behavioral measures varied across conditions, supporting the benefit of multi-modal eHMIs for safety and usability.
• The study outlines four practical implications for future eHMI design and research.

Figure 2. The appearance of the AV used in our study, where a human avatar sits in the driver’s seat but does not engage with the vehicle. Active mode of i) the Light Strip and ii) the Display when the AV was fully stopped.