[Paper Review] Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure

Been Kim, Emily Reif+3 more|arXiv (Cornell University)|Mar 4, 2019

Visual perception and processing mechanisms48 references4 citations

TL;DR

This paper investigates whether deep neural networks trained on natural images spontaneously exhibit Gestalt closure—perceiving incomplete shapes as whole—without explicit supervision. Using a similarity measure on internal representations, the authors show that a state-of-the-art CNN treats aligned edge fragments as perceptually equivalent to complete triangles, demonstrating closure-like behavior through representational similarity, supporting the hypothesis that Gestalt principles may emerge from statistical learning in natural environments.

ABSTRACT

The Gestalt laws of perceptual organization, which describe how visual elements in an image are grouped and interpreted, have traditionally been thought of as innate despite their ecological validity. We use deep-learning methods to investigate whether natural scene statistics might be sufficient to derive the Gestalt laws. We examine the law of closure, which asserts that human visual perception tends to "close the gap" by assembling elements that can jointly be interpreted as a complete figure or object. We demonstrate that a state-of-the-art convolutional neural network, trained to classify natural images, exhibits closure on synthetic displays of edge fragments, as assessed by similarity of internal representations. This finding provides support for the hypothesis that the human perceptual system is even more elegant than the Gestaltists imagined: a single law---adaptation to the statistical structure of the environment---might suffice as fundamental.

Motivation & Objective

• To test whether Gestalt closure—perceiving incomplete shapes as whole—can emerge in deep neural networks trained on natural images without explicit supervision.
• To investigate whether the statistical structure of natural scenes is sufficient to induce closure-like perception, challenging the view that Gestalt principles are innate.
• To provide a computational alternative to traditional cognitive models that assume specialized, innate mechanisms for perceptual filling-in.
• To quantify closure using representational similarity in deep network activations, avoiding reliance on behavioral response models.

Proposed method

• A state-of-the-art convolutional neural network (CNN) is trained on ImageNet to classify natural images.
• Synthetic stimuli are created: complete triangles, aligned edge fragments (inducing closure), and disordered fragments (preventing closure), varying in orientation, size, and global position.
• Representational similarity is measured using cosine similarity between feature embeddings of matched image triples: complete, aligned, and disordered fragments.
• The closure measure 𝒞 is defined as the difference in similarity between complete-aligned and complete-disordered pairs: 𝒞 = s(f(𝑎𝑖), f(𝑐𝑖)) − s(f(𝑑𝑖), f(𝑐𝑖)).
• The mean closure 𝒞̄ is computed across 768 stimulus triples, with constraints ensuring equal pixel-level overlap between complete and fragment pairs.
• A cosine similarity function is used: s(x,y) = (x⋅y^T)/(|x||y|), with special handling for zero vectors.

Experimental results

Research questions

• RQ1 Can a deep neural network trained on natural images exhibit Gestalt closure without explicit supervision or innate mechanisms?
• RQ2 Is the statistical structure of natural scenes sufficient to give rise to closure-like perception in artificial neural networks?
• RQ3 Does the representational similarity in deep networks mirror human perceptual grouping, as seen in closure tasks?
• RQ4 Can closure be quantified via internal representations without relying on behavioral response models?

Key findings

• The mean closure measure 𝒞̄ was significantly positive, indicating that the network’s internal representations treat aligned fragments as more similar to complete triangles than disordered fragments.
• The network’s representations for complete triangles and aligned fragments were nearly identical in embedding space, suggesting perceptual equivalence.
• The closure measure reached values close to +1, indicating strong representational similarity between complete and aligned stimuli, consistent with Gestalt closure.
• The network showed no significant negative closure values, indicating no reversal of perceptual grouping, and the effect was robust across varying sizes, orientations, and global positions.
• The results suggest that Gestalt principles like closure may not be innate but could emerge from statistical learning in natural environments.
• The findings support the hypothesis that a single principle—adaptation to environmental statistics—could underlie multiple Gestalt laws, including closure.