[Paper Review] Predicting Webpage Aesthetics with Heatmap Entropy

Zhenyu Gu, Chenhao Jin+2 more|arXiv (Cornell University)|Mar 5, 2018

Visual Attention and Saliency Detection23 references3 citations

TL;DR

This study proposes visual attention entropy (VAE) and its improved variant, relative VAE (rVAE), as quantitative metrics derived from eye-tracking heatmaps to predict webpage aesthetics. Using data from 30 observers viewing 40 webpages for 3 seconds, rVAE showed a strong negative correlation (r = -0.65) with subjective aesthetic ratings, achieving ~85% accuracy in distinguishing good from bad designs, with stable performance after 1 second of tracking.

ABSTRACT

Today, eye trackers are extensively used in user interface evaluations. However, it's still hard to analyze and interpret eye tracking data from the aesthetic point of view. To find quantitative links between eye movements and aesthetic experience, we tracked 30 observers' initial landings for 40 web pages (each displayed for 3 seconds). The web pages were also rated based on the observers' subjective aesthetic judgments. Shannon entropy was introduced to analyze the eye-tracking data. The result shows that the heatmap entropy (visual attention entropy, VAE) is highly correlated with the observers' aesthetic judgements of the web pages. Its improved version, relative VAE (rVAE), has a more significant correlation with the perceived aesthetics. (r=-0.65, F= 26.84, P$

Motivation & Objective

• To identify objective, quantitative metrics from eye-tracking data that correlate with human aesthetic judgments of webpages.
• To investigate whether visual attention patterns, measured via entropy, can predict perceived aesthetics without relying on subjective labeling.
• To evaluate the stability and predictive power of entropy-based metrics over time and across varying experimental parameters.
• To explore the theoretical link between perceptual fluency and aesthetic preference using eye-tracking data.

Proposed method

• Collected eye-tracking data from 30 observers viewing 40 webpages for 3 seconds each under controlled conditions.
• Generated gaze heatmaps using Gaussian smoothing with varying sigma (σ) values to model fixation density.
• Calculated visual attention entropy (VAE) as the Shannon entropy of the normalized heatmap, quantifying spatial attention distribution chaos.
• Introduced relative VAE (rVAE) as a normalized version of VAE to improve correlation with aesthetic judgments.
• Evaluated metric performance across different tracking durations (1s, 2s, 3s), subject counts, and sigma values.
• Used Pearson correlation and F-test to assess statistical significance between entropy metrics and aesthetic ratings.

Experimental results

Research questions

• RQ1 Can visual attention entropy (VAE) derived from eye-tracking heatmaps predict subjective aesthetic judgments of webpages?
• RQ2 Does the improved metric, relative VAE (rVAE), show stronger correlation with perceived aesthetics than VAE?
• RQ3 How does the predictive performance of VAE and rVAE evolve over time during the 3-second viewing period?
• RQ4 How sensitive are the entropy metrics to variations in Gaussian smoothing (σ) and the number of participants?
• RQ5 To what extent does the temporal structure of gaze sequences (e.g., fixation order) contribute to aesthetic prediction compared to spatial distribution?

Key findings

• Relative VAE (rVAE) demonstrated a statistically significant negative correlation (r = -0.65, F = 26.84, p < 0.0001) with subjective aesthetic ratings.
• The rVAE metric alone achieved approximately 85% accuracy in distinguishing aesthetically pleasing from unpleasing webpages.
• Both VAE and rVAE showed stable performance after 1 second of eye-tracking data, indicating early predictive potential.
• Performance of both metrics improved with longer tracking durations, suggesting potential for higher accuracy with extended observation times.
• The metrics were robust across a wide range of Gaussian smoothing parameters (σ), with rVAE peaking around σ = 40px and maintaining high correlation across 10–120px.
• VAE based on heatmap entropy outperformed entropy computed from gaze sequence (Markov chain), indicating spatial distribution is more predictive than temporal sequence.