[论文解读] Emergent Properties of Foveated Perceptual Systems

Arturo Deza, Talia Konkle|arXiv (Cornell University)|Jun 14, 2020

Visual Attention and Saliency Detection参考文献 59被引用 22

一句话总结

本文通过在相同感知压缩条件下，对比模拟人类周边视觉的基于纹理编码的foveated-texture模型与foveated-blur及均匀模糊模型，研究了foveated感知系统在机器视觉中的表征影响。foveated-texture模型在场景分类准确率上与全分辨率参考模型相当，表现出更优的泛化能力、更强的遮挡鲁棒性以及更显著的中心偏好，证明了基于纹理的周边编码可为视觉系统提供一种独特、高效且鲁棒的表征格式。

ABSTRACT

The goal of this work is to characterize the representational impact that foveation operations have for machine vision systems, inspired by the foveated human visual system, which has higher acuity at the center of gaze and texture-like encoding in the periphery. To do so, we introduce models consisting of a first-stage extit{fixed} image transform followed by a second-stage extit{learnable} convolutional neural network, and we varied the first stage component. The primary model has a foveated-textural input stage, which we compare to a model with foveated-blurred input and a model with spatially-uniform blurred input (both matched for perceptual compression), and a final reference model with minimal input-based compression. We find that: 1) the foveated-texture model shows similar scene classification accuracy as the reference model despite its compressed input, with greater i.i.d. generalization than the other models; 2) the foveated-texture model has greater sensitivity to high-spatial frequency information and greater robustness to occlusion, w.r.t the comparison models; 3) both the foveated systems, show a stronger center image-bias relative to the spatially-uniform systems even with a weight sharing constraint. Critically, these results are preserved over different classical CNN architectures throughout their learning dynamics. Altogether, this suggests that foveation with peripheral texture-based computations yields an efficient, distinct, and robust representational format of scene information, and provides symbiotic computational insight into the representational consequences that texture-based peripheral encoding may have for processing in the human visual system, while also potentially inspiring the next generation of computer vision models via spatially-adaptive computation. Code + Data available here: https://github.com/ArturoDeza/EmergentProperties

研究动机与目标

• 探究在foveated系统中，基于纹理的周边编码是否在机器视觉中带来表征优势。
• 在相同感知压缩条件下，对比foveated-texture、foveated-blur与均匀模糊的输入变换。
• 评估foveation对深度学习模型泛化能力、遮挡鲁棒性以及图像中心偏置的影响。
• 通过机器学习类比，为人类视觉中周边纹理编码的功能作用提供计算洞察。
• 探究通过foveation实现的空间自适应计算是否能启发更高效且鲁棒的计算机视觉架构。

提出的方法

• 本研究采用两阶段模型：固定的第一阶段图像变换，后接可学习的卷积神经网络（CNN）。
• 主要模型应用foveated-texture变换，通过视觉拥挤效应与潜在空间扰动模拟人类周边视觉。
• foveated-blur模型对输入应用空间可变的高斯模糊，保留相同的foveation模式但使用模糊而非纹理。
• uniformly-blurred模型应用空间均匀的模糊，其感知压缩程度与foveated模型匹配。
• 所有模型在多个CNN架构（如AlexNet、ResNet18）及训练动态下进行评估，以确保泛化性。
• 评估内容包括场景分类准确率、对遮挡的鲁棒性、图像中心偏置，以及感知质量度量（如SSIM、MSE、互信息）。

实验结果

研究问题

• RQ1 在相同感知压缩条件下，foveated-texture输入是否比foveated-blur或均匀模糊输入带来更好的泛化性能？
• RQ2 与其它压缩策略相比，foveated-texture输入如何影响对图像遮挡的鲁棒性？
• RQ3 foveation在多大程度上诱导了图像中心偏置？与空间均匀系统相比如何？
• RQ4 foveated系统中基于纹理的周边编码是否能产生一种独特且更高效的场景理解表征格式？
• RQ5 foveation对机器视觉及人类视觉处理理解的计算影响是什么？

主要发现

• foveated-texture模型在极低输入压缩下实现了与参考模型相当的场景分类准确率，证明了其高度的表征效率。
• foveated-texture模型在i.i.d.泛化性能上显著优于foveated-blur和均匀模糊模型。
• 与对比模型相比，foveated-texture模型在周边区域表现出更强的遮挡鲁棒性。
• 两种foveated系统在权重共享约束下，其图像中心偏置均强于空间均匀系统。
• 这些表征优势——鲁棒性、泛化能力与偏置——在不同CNN架构及整个训练动态过程中均得以保持。
• 结果表明，foveated系统中基于纹理的周边编码可构建一种独特、高效且具有韧性的场景感知表征格式。