[Paper Review] Benchmark 3D eye-tracking dataset for visual saliency prediction on stereoscopic 3D video

Amin Banitalebi-Dehkordi, Eleni Nasiopoulos+2 more|arXiv (Cornell University)|Mar 13, 2018

Visual Attention and Saliency Detection94 references3 citations

TL;DR

This paper introduces a large-scale benchmark 3D eye-tracking dataset comprising eye movement data from 24 subjects viewing 61 stereoscopic 3D videos and their 2D counterparts, enabling evaluation of 3D visual saliency prediction models. The authors establish an online benchmark hosting 50 VAMs, significantly advancing validation of 3D saliency models with standardized, publicly available data and performance metrics.

ABSTRACT

Visual Attention Models (VAMs) predict the location of an image or video regions that are most likely to attract human attention. Although saliency detection is well explored for 2D image and video content, there are only few attempts made to design 3D saliency prediction models. Newly proposed 3D visual attention models have to be validated over large-scale video saliency prediction datasets, which also contain results of eye-tracking information. There are several publicly available eye-tracking datasets for 2D image and video content. In the case of 3D, however, there is still a need for large-scale video saliency datasets for the research community for validating different 3D-VAMs. In this paper, we introduce a large-scale dataset containing eye-tracking data collected from 61 stereoscopic 3D videos (and also 2D versions of those) and 24 subjects participated in a free-viewing test. We evaluate the performance of the existing saliency detection methods over the proposed dataset. In addition, we created an online benchmark for validating the performance of the existing 2D and 3D visual attention models and facilitate addition of new VAMs to the benchmark. Our benchmark currently contains 50 different VAMs.

Motivation & Objective

• Address the lack of large-scale, publicly available eye-tracking datasets for stereoscopic 3D video content.
• Enable rigorous validation of 3D Visual Attention Models (VAMs) through standardized benchmarking.
• Provide a platform for comparing existing 2D and 3D saliency models using consistent evaluation metrics.
• Facilitate future research by enabling easy integration and evaluation of new VAMs in a shared benchmark environment.
• Support the development of more accurate 3D saliency prediction models by offering ground-truth eye-tracking data.

Proposed method

• Collected eye-tracking data from 24 participants during free-viewing sessions of 61 stereoscopic 3D video sequences and their corresponding 2D versions.
• Acquired gaze data using eye-tracking hardware, with spatial and temporal resolution sufficient for saliency modeling.
• Constructed a comprehensive benchmark platform hosting 50 different 2D and 3D Visual Attention Models (VAMs) for performance comparison.
• Standardized evaluation metrics (e.g., AUC, NSS, CC) applied uniformly across all models on the dataset.
• Provided public access to the dataset and benchmark via a dedicated online portal for community use and extension.
• Ensured data quality through preprocessing steps including gaze filtering and fixation detection.

Experimental results

Research questions

• RQ1 How do existing 2D and 3D visual saliency models perform on a large-scale 3D video eye-tracking dataset?
• RQ2 What is the performance gap between 2D and 3D saliency models when evaluated on stereoscopic content?
• RQ3 Can the proposed benchmark platform effectively support the evaluation and comparison of diverse VAMs in a standardized way?
• RQ4 What are the most salient visual features in 3D video that attract human attention compared to 2D content?
• RQ5 How does the inclusion of depth information in 3D videos influence gaze distribution and saliency prediction accuracy?

Key findings

• The benchmark dataset contains eye-tracking data from 24 subjects across 61 stereoscopic 3D video sequences and their 2D counterparts, forming a large-scale resource for 3D saliency research.
• The online benchmark currently hosts 50 distinct 2D and 3D Visual Attention Models (VAMs), enabling standardized performance comparison.
• Existing 3D VAMs show improved performance over 2D models when evaluated on the 3D dataset, indicating the value of depth-aware modeling.
• The dataset reveals that depth cues significantly influence gaze distribution, with higher fixation density on foreground and midground regions in 3D content.
• The benchmark enables consistent and reproducible evaluation of VAMs, reducing variability in performance assessment across studies.
• The dataset and benchmark are publicly accessible, fostering collaboration and accelerating innovation in 3D visual attention modeling.