[Paper Review] Interactive Binary Image Segmentation with Edge Preservation

Jianfeng Zhang, Liezhuo Zhang+4 more|arXiv (Cornell University)|Sep 10, 2018

Medical Image Segmentation Techniques24 references3 citations

TL;DR

This paper proposes a novel interactive binary image segmentation method using Markov Random Fields (MRF) with geodesic distance-based unary terms on superpixels and bilateral affinity for pairwise regularization. By combining fast bilateral solver (FBS) and steepest gradient descent via alternating direction, the method achieves high-quality, edge-preserving segmentation with improved efficiency and accuracy over state-of-the-art approaches on the VGG dataset.

ABSTRACT

Binary image segmentation plays an important role in computer vision and has been widely used in many applications such as image and video editing, object extraction, and photo composition. In this paper, we propose a novel interactive binary image segmentation method based on the Markov Random Field (MRF) framework and the fast bilateral solver (FBS) technique. Specifically, we employ the geodesic distance component to build the unary term. To ensure both computation efficiency and effective responsiveness for interactive segmentation, superpixels are used in computing geodesic distances instead of pixels. Furthermore, we take a bilateral affinity approach for the pairwise term in order to preserve edge information and denoise. Through the alternating direction strategy, the MRF energy minimization problem is divided into two subproblems, which then can be easily solved by steepest gradient descent (SGD) and FBS respectively. Experimental results on the VGG interactive image segmentation dataset show that the proposed algorithm outperforms several state-of-the-art ones, and in particular, it can achieve satisfactory edge-smooth segmentation results even when the foreground and background color appearances are quite indistinctive.

Motivation & Objective

• To address the challenge of ambiguous color appearances in interactive binary image segmentation by incorporating user-provided scribbles.
• To improve segmentation accuracy and edge preservation in complex or low-contrast images where traditional methods fail.
• To reduce computational cost while maintaining high segmentation quality through superpixel-level geodesic distance computation.
• To enhance boundary sensitivity and smoothness using bilateral affinity as a regularizer in the MRF framework.
• To achieve real-time responsiveness in interactive segmentation by decoupling optimization via alternating direction strategy.

Proposed method

• The unary term is constructed using geodesic distances computed from superpixel centers to foreground and background seed sets, reducing computational load compared to pixel-level computation.
• The pairwise term employs bilateral affinity to preserve edges and denoise, leveraging the edge-aware smoothing properties of the fast bilateral solver (FBS).
• The MRF energy minimization problem is decomposed into two subproblems using the alternating direction method of multipliers (ADMM).
• The first subproblem is solved via steepest gradient descent (SGD) for label optimization, while the second is solved efficiently using FBS for smoothness enforcement.
• Superpixels are used to approximate image structure, enabling faster geodesic distance computation without sacrificing segmentation fidelity.
• The method integrates user clicks or scribbles as seed inputs to guide the segmentation process, ensuring alignment with user intent.

Experimental results

Research questions

• RQ1 Can superpixel-based geodesic distance computation significantly reduce computational cost while maintaining segmentation accuracy in interactive binary segmentation?
• RQ2 How does bilateral affinity regularization compare to total variation in preserving object boundaries under low-contrast or complex textures?
• RQ3 To what extent does the alternating direction strategy enable efficient and responsive interactive segmentation with high-quality results?
• RQ4 What is the optimal number of superpixels for balancing computational efficiency and segmentation performance?
• RQ5 Can the proposed method achieve superior edge-smoothness and accuracy when foreground and background colors are visually similar?

Key findings

• The proposed method achieved an average Intersection over Union (IoU) of 0.834 and an F2-score of 0.930 on the VGG dataset with original annotations, outperforming several state-of-the-art methods.
• With modified annotations, the method achieved an IoU of 0.841 and an F2-score of 0.934, demonstrating robustness and consistency across annotation variations.
• The use of the FBS module significantly improved boundary continuity and smoothness, as visually confirmed in segmentation results with 1600 superpixels.
• Increasing the number of superpixels beyond 1600 yielded minimal performance gains, indicating that 1600 provides a good trade-off between accuracy and efficiency.
• The method outperformed TV-based regularizers in edge preservation, as shown in comparative visualizations where FBS produced more continuous and accurate object boundaries.
• The ablation study confirmed that the FBS-based bilateral affinity term effectively reduced noise while maintaining sharp edges, especially in challenging low-contrast regions.