[Paper Review] Graph-based Representation for Image based on Granular-ball

Xia Shuyin, Dawei Dai+5 more|arXiv (Cornell University)|Mar 4, 2023

Slime Mold and Myxomycetes Research4 citations

TL;DR

This paper proposes a graph-based image representation using adaptive granular-rectangle units derived from pixel-level data, where each node represents a structurally coherent region and edges encode spatial relationships. The method achieves superior classification accuracy with fewer parameters and enhanced model interpretability through attention visualization, demonstrating improved efficiency, robustness, and understandability over pixel-level deep learning models.

ABSTRACT

Current image processing methods usually operate on the finest-granularity unit; that is, the pixel, which leads to challenges in terms of efficiency, robustness, and understandability in deep learning models. We present an improved granular-ball computing method to represent the image as a graph, in which each node expresses a structural block in the image and each edge represents the association between two nodes. Specifically:(1) We design a gradient-based strategy for the adaptive reorganization of all pixels in the image into numerous rectangular regions, each of which can be regarded as one node. (2) Each node has a connection edge with the nodes with which it shares regions. (3) We design a low-dimensional vector as the attribute of each node. All nodes and their corresponding edges form a graphical representation of a digital image. In the experiments, our proposed graph representation is applied to benchmark datasets for image classification tasks, and the efficiency and good understandability demonstrate that our proposed method offers significant potential in artificial intelligence theory and application.

Motivation & Objective

• To address the limitations of pixel-level processing in deep learning, including poor robustness, low efficiency, and lack of interpretability.
• To develop a multi-granularity, structure-aware image representation that aligns better with human cognitive processes.
• To construct a graph-based image representation where nodes correspond to coherent image regions and edges encode spatial associations.
• To evaluate the proposed representation in image classification tasks using graph neural networks (GNNs).
• To enhance model interpretability by visualizing attention scores on graph nodes during decision-making.

Proposed method

• A gradient-based adaptive reorganization strategy groups similar pixels into rectangular regions, replacing the original ball descriptor in granular-ball computing.
• Each rectangular region is treated as a node in a graph, with edges established between adjacent or overlapping regions.
• A low-dimensional vector is assigned to each node based on the statistical properties of its constituent pixels, capturing local texture and intensity features.
• The resulting graph structure is fed into a Graph Attention Network (GAT) for image classification, enabling message passing over structural units.
• The model uses attention mechanisms to highlight key regions in the graph during inference, improving interpretability.
• Training employs Adam optimizer with a decreasing learning rate schedule and early stopping when validation error plateaus.

Experimental results

Research questions

• RQ1 Can a graph-based image representation using adaptive rectangular granular units improve model efficiency and robustness compared to pixel-level processing?
• RQ2 Does the proposed graph representation enhance the interpretability of deep learning models in image classification tasks?
• RQ3 How does the performance of GNNs based on this graph representation compare to traditional CNNs and GNNs on standard benchmarks?
• RQ4 To what extent do attention mechanisms in GATs highlight meaningful image regions during classification?
• RQ5 Can the proposed method reduce model parameters while maintaining or improving classification accuracy?

Key findings

• The GAT model using the proposed graph representation achieved higher test accuracy than all baseline GNN models on both MNIST and CIFAR-10 datasets.
• The number of model parameters in the GAT model based on the proposed representation was significantly smaller than that of traditional GNNs.
• The model converged to a lower training error despite a slightly slower initial convergence rate, indicating strong optimization capability.
• Attention visualization revealed that correct predictions focused on object surfaces, while incorrect predictions were driven by background regions, confirming interpretability.
• The graph representation effectively captured structural and semantic content, enabling GNNs to generalize better than standard deep neural networks on the tested datasets.
• The method demonstrated strong potential for building efficient, robust, and interpretable AI models in computer vision.