[Paper Review] Emergent Properties of Foveated Perceptual Systems

Arturo Deza, Talia Konkle|arXiv (Cornell University)|Jun 14, 2020

Visual Attention and Saliency Detection59 references22 citations

TL;DR

This paper investigates the representational impact of foveated perceptual systems in machine vision by comparing a foveated-texture model—mimicking human peripheral vision with texture-based encoding—against foveated-blur and uniformly-blurred models under matched perceptual compression. The foveated-texture model achieves comparable scene classification accuracy to a full-resolution reference model, exhibits superior generalization, enhanced robustness to occlusion, and stronger center-bias, demonstrating that texture-based peripheral encoding yields a distinct, efficient, and robust representational format for vision systems.

ABSTRACT

The goal of this work is to characterize the representational impact that foveation operations have for machine vision systems, inspired by the foveated human visual system, which has higher acuity at the center of gaze and texture-like encoding in the periphery. To do so, we introduce models consisting of a first-stage extit{fixed} image transform followed by a second-stage extit{learnable} convolutional neural network, and we varied the first stage component. The primary model has a foveated-textural input stage, which we compare to a model with foveated-blurred input and a model with spatially-uniform blurred input (both matched for perceptual compression), and a final reference model with minimal input-based compression. We find that: 1) the foveated-texture model shows similar scene classification accuracy as the reference model despite its compressed input, with greater i.i.d. generalization than the other models; 2) the foveated-texture model has greater sensitivity to high-spatial frequency information and greater robustness to occlusion, w.r.t the comparison models; 3) both the foveated systems, show a stronger center image-bias relative to the spatially-uniform systems even with a weight sharing constraint. Critically, these results are preserved over different classical CNN architectures throughout their learning dynamics. Altogether, this suggests that foveation with peripheral texture-based computations yields an efficient, distinct, and robust representational format of scene information, and provides symbiotic computational insight into the representational consequences that texture-based peripheral encoding may have for processing in the human visual system, while also potentially inspiring the next generation of computer vision models via spatially-adaptive computation. Code + Data available here: https://github.com/ArturoDeza/EmergentProperties

Motivation & Objective

• To investigate whether texture-based peripheral encoding in foveated systems confers representational advantages in machine vision.
• To compare foveated-texture, foveated-blur, and uniformly-blurred input transformations under matched perceptual compression.
• To assess the impact of foveation on generalization, robustness to occlusion, and center-image bias in deep learning models.
• To provide computational insight into the functional role of peripheral texture coding in human vision via machine learning analogs.
• To explore whether spatially-adaptive computation via foveation can inspire more efficient and robust computer vision architectures.

Proposed method

• The study employs a two-stage model: a fixed first-stage image transform followed by a learnable convolutional neural network (CNN).
• The primary model applies a foveated-texture transform, simulating human peripheral vision using visual crowding effects and latent space perturbations.
• The foveated-blur model applies spatially-varying Gaussian blur to the input, preserving the same foveation pattern but using blur instead of texture.
• The uniformly-blurred model applies a spatially uniform blur, matched in perceptual compression to the foveated models.
• All models are evaluated across multiple CNN architectures (e.g., AlexNet, ResNet18) and training dynamics to ensure generalizability.
• Evaluation includes scene classification accuracy, robustness to occlusion, center-image bias, and perceptual quality metrics (e.g., SSIM, MSE, mutual information).

Experimental results

Research questions

• RQ1 Does foveated-texture input lead to better generalization than foveated-blur or uniformly-blurred inputs under matched perceptual compression?
• RQ2 How does foveated-texture input affect robustness to image occlusion compared to other compression strategies?
• RQ3 To what extent does foveation induce a center-image bias, and how does this compare to spatially-uniform systems?
• RQ4 Can texture-based peripheral encoding in foveated systems yield a distinct and more efficient representational format for scene understanding?
• RQ5 What are the computational implications of foveation for both machine vision and our understanding of human visual processing?

Key findings

• The foveated-texture model achieves scene classification accuracy comparable to the reference model with minimal input compression, demonstrating high representational efficiency.
• The foveated-texture model shows significantly greater i.i.d. generalization performance than both foveated-blur and uniformly-blurred models.
• The foveated-texture model exhibits greater robustness to occlusion, particularly in the periphery, compared to the comparison models.
• Both foveated systems display a stronger center-image bias than the spatially-uniform systems, even under weight-sharing constraints.
• These representational advantages—robustness, generalization, and bias—are preserved across different CNN architectures and throughout training dynamics.
• The results suggest that texture-based peripheral encoding in foveated systems creates a distinct, efficient, and resilient representational format for scene perception.