[Paper Review] Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution

Liangbin Xie, Xintao Wang+3 more|arXiv (Cornell University)|Aug 2, 2021

Image Processing Techniques and Applications19 citations

TL;DR

This paper proposes FAIG, a novel filter attribution method based on integral gradients in parameter space, to identify minimal sets of discriminative filters (as few as 1%) in one-branch blind super-resolution networks that enable automatic degradation prediction. The method reveals that filter weights, locations, and connections collectively enable implicit degradation classification without explicit supervision, offering interpretability and design insights for efficient blind SR architectures.

ABSTRACT

Recent blind super-resolution (SR) methods typically consist of two branches, one for degradation prediction and the other for conditional restoration. However, our experiments show that a one-branch network can achieve comparable performance to the two-branch scheme. Then we wonder: how can one-branch networks automatically learn to distinguish degradations? To find the answer, we propose a new diagnostic tool -- Filter Attribution method based on Integral Gradient (FAIG). Unlike previous integral gradient methods, our FAIG aims at finding the most discriminative filters instead of input pixels/features for degradation removal in blind SR networks. With the discovered filters, we further develop a simple yet effective method to predict the degradation of an input image. Based on FAIG, we show that, in one-branch blind SR networks, 1) we are able to find a very small number of (1%) discriminative filters for each specific degradation; 2) The weights, locations and connections of the discovered filters are all important to determine the specific network function. 3) The task of degradation prediction can be implicitly realized by these discriminative filters without explicit supervised learning. Our findings can not only help us better understand network behaviors inside one-branch blind SR networks, but also provide guidance on designing more efficient architectures and diagnosing networks for blind SR.

Motivation & Objective

• To investigate whether one-branch blind super-resolution networks can implicitly learn to distinguish specific degradations like blur and noise without explicit two-branch supervision.
• To develop a diagnostic tool capable of identifying the minimal set of filters responsible for specific degradation removal in unified networks.
• To understand the role of filter weights, spatial locations, and connectivity in enabling degradation-specific functionality within a single-branch architecture.
• To explore whether degradation prediction can be achieved implicitly through discovered filters without additional training or supervision.
• To provide interpretability insights that guide the design of more efficient and robust blind super-resolution models.

Proposed method

• Propose FAIG (Filter Attribution based on Integral Gradient), a method that traces functional changes in the network to changes in filter parameters along paths in the parameter space, rather than input pixels or features.
• Use gradient-based attribution along parameter paths to identify filters most responsible for removing specific degradations (e.g., blur, noise) in a one-branch SR network.
• Apply a gradient elimination strategy during attribution to isolate the contribution of filters for a target degradation while preserving the function for other degradations.
• Develop a degradation prediction mechanism using learned thresholds on filter activation magnitudes, enabling inference without fine-tuning or labeled degradation data.
• Visualize and analyze the spatial distribution and connectivity of discovered filters across network layers to study functional specialization.
• Validate findings on SRCNN-style and SRResNet-based blind SR models using standard benchmarks (Set14, DIV2K, BSD100).

Experimental results

Research questions

• RQ1 Can a unified one-branch blind super-resolution network implicitly learn to distinguish different types of image degradations without explicit supervision?
• RQ2 What is the minimal set of filters required to enable removal of a specific degradation in a one-branch network?
• RQ3 How do the weights, spatial locations, and connectivity of filters collectively determine their functional role in degradation-specific restoration?
• RQ4 Can degradation prediction be achieved implicitly through filter activation patterns without additional training or labels?
• RQ5 How do the distributions of discriminative filters differ across degradation types (e.g., blur vs. noise) in the network architecture?

Key findings

• The proposed FAIG method successfully identifies a minimal set of 1% filters in one-branch blind SR networks that are responsible for removing specific degradations such as blur and noise.
• Filter weights, spatial locations, and connectivity are all critical to the network’s ability to perform degradation-specific restoration, indicating a distributed yet specialized functional organization.
• Degradation prediction accuracy reaches 98% for noise and 96% for blur on the DIV2K dataset using only filter activation thresholds, without any supervised training on degradation labels.
• The deblurring and denoising filters are spatially distributed differently across the network, with deblurring filters concentrated toward the back and denoising filters more uniformly distributed, suggesting functional specialization and division of labor.
• The gradient elimination technique during attribution is essential—without it, filters for one degradation may inadvertently affect other degradation functions, reducing discriminative power.
• The findings demonstrate that one-branch networks implicitly learn a functional decomposition similar to two-branch designs, with dedicated sub-networks for each degradation emerging through self-supervised training.