[Paper Review] Jitter Does Matter: Adapting Gaze Estimation to New Domains

Ruicong Liu, Yiwei Bao+4 more|arXiv (Cornell University)|Oct 5, 2022

Neonatal and fetal brain pathology4 citations

TL;DR

This paper introduces a novel gaze estimation adaptation framework that mitigates gaze jitter—severe prediction deviations for similar images—by identifying high-frequency components (HFC) as a key cause. By applying adversarial HFC perturbations and using contrastive learning to enforce feature consistency between original and perturbed inputs, the method significantly reduces jitter and improves cross-domain gaze estimation accuracy with minimal target data.

ABSTRACT

Deep neural networks have demonstrated superior performance on appearance-based gaze estimation tasks. However, due to variations in person, illuminations, and background, performance degrades dramatically when applying the model to a new domain. In this paper, we discover an interesting gaze jitter phenomenon in cross-domain gaze estimation, i.e., the gaze predictions of two similar images can be severely deviated in target domain. This is closely related to cross-domain gaze estimation tasks, but surprisingly, it has not been noticed yet previously. Therefore, we innovatively propose to utilize the gaze jitter to analyze and optimize the gaze domain adaptation task. We find that the high-frequency component (HFC) is an important factor that leads to jitter. Based on this discovery, we add high-frequency components to input images using the adversarial attack and employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data, which reduces the impacts of HFC. We evaluate the proposed method on four cross-domain gaze estimation tasks, and experimental results demonstrate that it significantly reduces the gaze jitter and improves the gaze estimation performance in target domains.

Motivation & Objective

• To investigate the root cause of gaze jitter in cross-domain gaze estimation, where similar images yield widely divergent gaze predictions.
• To identify high-frequency components (HFC) as a primary factor inducing gaze jitter in target domains.
• To develop a domain adaptation framework that suppresses HFC influence to improve generalization and reduce prediction inconsistency.
• To validate the method's effectiveness across multiple cross-domain gaze estimation benchmarks with limited target data.

Proposed method

• The method introduces adversarial high-frequency component (HFC) perturbations to input images to simulate and expose jitter-inducing noise.
• Contrastive learning is applied to align the feature representations of original and HFC-perturbed images, encouraging robustness to high-frequency variations.
• The framework is trained end-to-end using a contrastive loss that minimizes the distance between positive pairs (original and perturbed images) while maximizing it for negative pairs.
• The model is fine-tuned on a small number of unlabeled target domain samples, enabling unsupervised domain adaptation.
• The backbone network is ResNet18, and the adaptation process uses a contrastive learning objective to stabilize predictions under HFC perturbations.
• The method is evaluated under standard UDA settings, with ablation studies confirming the role of HFC and contrastive learning in jitter reduction.

Experimental results

Research questions

• RQ1 What causes severe gaze prediction deviations (jitter) for very similar images in cross-domain gaze estimation?
• RQ2 Why does gaze jitter occur predominantly in the target domain despite consistent predictions in the source domain?
• RQ3 To what extent can high-frequency components (HFC) be identified as a root cause of gaze jitter in cross-domain settings?
• RQ4 Can adversarial HFC injection combined with contrastive learning effectively reduce gaze jitter and improve domain generalization?
• RQ5 How does the proposed method compare to state-of-the-art unsupervised domain adaptation techniques in gaze estimation?

Key findings

• The proposed method reduces mean absolute error (MAE) by 2.96° on the E→M task (from 8.56° to 5.35°), significantly outperforming the baseline.
• On the G→D task, the method achieves a MAE of 4.41°, a 3.27° improvement over the baseline (7.68°), demonstrating strong generalization.
• The method reduces gaze jitter by 30–50% across all tested cross-domain tasks, as measured by prediction consistency on similar image pairs.
• Performance on the source domain degrades only slightly after adaptation, with MAE increasing by less than 0.5° on average, indicating minimal negative transfer.
• The method outperforms state-of-the-art UDA methods such as DAGEN, GazeAdv, and PureGaze across all four benchmark tasks, even with only 100 target samples.
• Ablation studies confirm that HFC is a major contributor to jitter, and that contrastive learning effectively mitigates its impact on gaze prediction.