[Paper Review] Natural Image Matting via Guided Contextual Attention

Yaoyi Li, Hongtao Lu|arXiv (Cornell University)|Jan 13, 2020

Image Enhancement Techniques40 references6 citations

TL;DR

This paper proposes a novel end-to-end deep learning method for natural image matting called GCA Matting, which introduces a guided contextual attention (GCA) module to propagate high-level alpha features globally using low-level image affinity as guidance. The method achieves state-of-the-art performance on both Composition-1k and alphamatting.com benchmarks, particularly excelling in preserving fine textures and structures in semitransparent regions.

ABSTRACT

Over the last few years, deep learning based approaches have achieved outstanding improvements in natural image matting. Many of these methods can generate visually plausible alpha estimations, but typically yield blurry structures or textures in the semitransparent area. This is due to the local ambiguity of transparent objects. One possible solution is to leverage the far-surrounding information to estimate the local opacity. Traditional affinity-based methods often suffer from the high computational complexity, which are not suitable for high resolution alpha estimation. Inspired by affinity-based method and the successes of contextual attention in inpainting, we develop a novel end-to-end approach for natural image matting with a guided contextual attention module, which is specifically designed for image matting. Guided contextual attention module directly propagates high-level opacity information globally based on the learned low-level affinity. The proposed method can mimic information flow of affinity-based methods and utilize rich features learned by deep neural networks simultaneously. Experiment results on Composition-1k testing set and alphamatting.com benchmark dataset demonstrate that our method outperforms state-of-the-art approaches in natural image matting. Code and models are available at https://github.com/Yaoyi-Li/GCA-Matting.

Motivation & Objective

• To address the challenge of blurry textures and structures in semitransparent regions of alpha matte predictions.
• To enable global opacity information propagation in deep neural networks using local image affinity as guidance.
• To overcome the limitations of traditional affinity-based methods, which suffer from high computational cost in high-resolution settings.
• To combine the strengths of affinity-based matting and deep learning priors for improved matting accuracy.
• To develop a scalable, end-to-end framework that processes full-resolution images efficiently.

Proposed method

• The guided contextual attention (GCA) module performs global alpha feature transmission based on learned low-level image affinity, mimicking affinity-based matting in a fully convolutional network.
• The GCA module uses low-level image features as guidance to propagate high-level alpha features between patches with similar appearance.
• The method treats image matting as a guided inpainting task, where the unknown alpha region is filled using 0 or 1 values from known foreground/background areas under image guidance.
• The framework integrates GCA blocks in both encoder and decoder branches, with separate processing for image features and alpha features.
• A residual connection and spectral normalization are used to stabilize training and improve feature learning.
• The model is trained end-to-end on the Adobe Image Matting dataset and evaluated on Composition-1k and alphamatting.com benchmarks.

Experimental results

Research questions

• RQ1 Can global opacity propagation guided by local image appearance improve matting quality in semitransparent regions?
• RQ2 Can a deep neural network effectively mimic the information flow of affinity-based matting while maintaining computational efficiency?
• RQ3 How does the proposed guided contextual attention module compare to existing attention mechanisms in image matting and inpainting?
• RQ4 Does the method generalize well across varying trimap sizes, especially large unknown regions?
• RQ5 Can the model achieve high-resolution matting without image scaling, preserving fine texture details?

Key findings

• On the Composition-1k benchmark, GCA Matting achieves the best performance in terms of Gradient Error and MSE, outperforming previous state-of-the-art methods.
• On the alphamatting.com benchmark, GCA Matting ranks first in the Gradient Error metric, especially excelling on the 'large' and 'user' trimaps with larger unknown regions.
• The method achieves full-resolution inference on a single NVIDIA GTX 1080 (8GB), processing entire images without scaling, which contributes to better texture preservation.
• Visualization of attention maps confirms that the model learns to propagate opacity from structurally and appearance-similar patches, especially from background regions.
• The model correctly predicts complex structures such as iron wires in a plastic bag, where previous methods failed, due to effective contextual reasoning.
• The method maintains strong performance across different trimap types, demonstrating robustness to changes in unknown region size.