[Paper Review] A New Algorithm for Interactive Structural Image Segmentation

Alexandre Noma, Ana Beatriz Vicentim Graciano+3 more|ArXiv.org|May 13, 2008

Medical Image Segmentation Techniques17 references13 citations

TL;DR

This paper proposes a novel interactive image segmentation algorithm that uses attributed relational graphs (ARGs) to model objects and match them to input images via a deformation graph, enabling efficient, structure-aware segmentation even with topological differences. The method achieves faster performance (O(|Vi||Vm|)) and robust, reusable models compared to prior work, with high-quality results on natural and target-specific color images.

ABSTRACT

This paper proposes a novel algorithm for the problem of structural image segmentation through an interactive model-based approach. Interaction is expressed in the model creation, which is done according to user traces drawn over a given input image. Both model and input are then represented by means of attributed relational graphs derived on the fly. Appearance features are taken into account as object attributes and structural properties are expressed as relational attributes. To cope with possible topological differences between both graphs, a new structure called the deformation graph is introduced. The segmentation process corresponds to finding a labelling of the input graph that minimizes the deformations introduced in the model when it is updated with input information. This approach has shown to be faster than other segmentation methods, with competitive output quality. Therefore, the method solves the problem of multiple label segmentation in an efficient way. Encouraging results on both natural and target-specific color images, as well as examples showing the reusability of the model, are presented and discussed.

Motivation & Objective

• To address the challenge of efficient, interactive segmentation in images with complex structures and appearance variations.
• To overcome limitations in existing model-based segmentation methods that struggle with topological mismatches between model and input graphs.
• To develop a method that leverages both appearance and structural features for robust segmentation using user-defined traces.
• To enable model reusability across similar images without retraining or redefining the model from scratch.
• To improve computational efficiency over prior ARG-based segmentation methods while maintaining high segmentation quality.

Proposed method

• The input image and user-defined model are represented as attributed relational graphs (ARGs), where vertices represent image regions and edges represent spatial or structural relations.
• Appearance features (e.g. color) are encoded as vertex attributes, and structural relationships are modeled as edge attributes in the ARGs.
• A new structure called the deformation graph is introduced to represent how the model ARG is locally altered when matched to input vertices, enabling topological flexibility.
• The segmentation is formulated as a graph matching problem that minimizes deformation costs, with each input vertex matched independently to minimize structural distortion.
• An optimization technique is used to compute the labeling of the input graph that minimizes total deformation, with cost function balancing appearance and structural similarity.
• The model is made reusable via a 'stamp' mechanism—once defined on a reference image, it can be applied to other images by placing the stamp over the region of interest.

Experimental results

Research questions

• RQ1 How can topological differences between model and input image graphs be effectively handled in interactive segmentation?
• RQ2 Can a deformation-based graph matching approach improve segmentation speed and robustness compared to traditional ARG matching?
• RQ3 To what extent can a single user-defined model be reused across different images with varying appearances but similar structures?
• RQ4 How does the integration of appearance and structural features affect segmentation accuracy in complex scenes?
• RQ5 Can the segmentation process be made independent of vertex labeling order while maintaining high performance?

Key findings

• The proposed algorithm runs in O(|Vi||Vm|) time, significantly faster than the Ω(|Vi||Vm|²) complexity of the prior method in [4].
• The method achieves high segmentation accuracy on natural color images from the Berkeley Segmentation Database, even in textured or complex regions like mountains and faces.
• The model demonstrates strong reusability: the same user-defined stroke model successfully segments similar objects across different images from the XM2VTS database and web-retrieved images.
• The deformation graph approach effectively handles topological mismatches, such as when input edges connect oversegmented regions that map to distinct model parts.
• The segmentation quality remains robust even when input images vary in lighting, texture, or object positioning, due to structural constraints in the model.
• The method does not depend on the order of vertex labeling during matching, improving consistency and performance.