[Paper Review] Comparison of a Head-Mounted Display and a Curved Screen in a Multi-Talker Audiovisual Listening Task

Gerard Llorach, Maartje M. E. Hendrikse+2 more|arXiv (Cornell University)|Apr 3, 2020

Hearing Loss and Rehabilitation4 citations

TL;DR

This study compares a head-mounted display (HMD) and a curved screen (CS) in a multi-talker audiovisual listening task across three visual conditions: audio-only, virtual characters, and video recordings. Results show that older normal-hearing participants preferred the CS over the HMD, and video recordings were the most preferred visual condition, suggesting CS with video recordings is optimal for hearing research settings despite HMDs being viable when necessary.

ABSTRACT

Introduction: Virtual audiovisual technology and its methodology has yet to be established for psychoacoustic research. This study examined the effects of different audiovisual conditions on preference when listening to multi-talker conversations. The study's goal is to explore and assess audiovisual technologies in the context of hearing research. Methods: The participants listened to audiovisual conversations between four talkers. Two displays were tested and compared: a curved screen (CS) and a head-mounted display (HMD). Using three visual conditions (audio-only, virtual characters and video recordings), three groups of participants were tested: seventeen young normal-hearing, ten older normal-hearing, and ten older hearing-impaired listeners. Results: Open interviews showed that the CS was preferred over the HMD for older normal-hearing participants and that video recordings were the preferred visual condition. Young and older hearing-impaired participants did not show a preference between the CS and the HMD. Conclusions: CSs and video recordings should be the preferred audiovisual setup of laboratories and clinics, although HMDs and virtual characters can be used for hearing research when necessary and suitable.

Motivation & Objective

• To evaluate the impact of different audiovisual display technologies—head-mounted display (HMD) and curved screen (CS)—on listener preference in multi-talker listening tasks.
• To assess the influence of visual condition (audio-only, virtual characters, video recordings) on user preference and listening performance.
• To determine whether hearing status (young normal-hearing, older normal-hearing, older hearing-impaired) moderates preferences for HMD vs. CS.
• To provide evidence-based recommendations for selecting audiovisual setups in hearing research laboratories and clinical settings.

Proposed method

• Participants were exposed to audiovisual conversations involving four talkers under three visual conditions: audio-only, virtual characters, and video recordings.
• Two display types were tested: a head-mounted display (HMD) and a curved screen (CS), each delivering the same audiovisual stimuli in controlled conditions.
• A within-subjects design was used, with participants completing listening tasks across all combinations of display and visual condition.
• Preference was assessed via open-ended interviews to gather qualitative feedback on perceived comfort, realism, and usability.
• Data were collected from 17 young normal-hearing, 10 older normal-hearing, and 10 older hearing-impaired listeners.
• Statistical analysis focused on identifying patterns in preference across hearing groups and display types.

Experimental results

Research questions

• RQ1 Does the choice of display (HMD vs. CS) significantly affect listener preference in a multi-talker audiovisual listening task?
• RQ2 Which visual condition—audio-only, virtual characters, or video recordings—is most preferred across different hearing groups?
• RQ3 Do older normal-hearing listeners show a stronger preference for one display type over another compared to younger or hearing-impaired listeners?
• RQ4 To what extent does hearing status moderate the preference for HMDs versus curved screens in audiovisual listening tasks?

Key findings

• Older normal-hearing participants expressed a clear preference for the curved screen (CS) over the head-mounted display (HMD).
• Among all visual conditions, video recordings were the most preferred by older normal-hearing participants.
• Young and older hearing-impaired participants did not show a significant preference between the HMD and the CS.
• The HMD was perceived as less comfortable or less realistic by older normal-hearing listeners, though no performance differences were reported.
• The study concludes that curved screens with video recordings should be the preferred audiovisual setup in hearing research laboratories and clinics.
• HMDs and virtual characters remain viable options when necessary, but are not preferred over CS and video recordings for most user groups.