[Paper Review] A Deep Tree-Structured Fusion Model for Single Image Deraining

Xueyang Fu, Qi Qi+4 more|arXiv (Cornell University)|Nov 21, 2018

Image Enhancement Techniques31 references4 citations

TL;DR

This paper proposes a deep tree-structured fusion model that enhances single image deraining by hierarchically fusing features within and across dilated convolutional blocks to reduce redundancy and improve representation learning. The method achieves state-of-the-art performance with only 35,427 parameters, outperforming deeper networks like ResNet and DenseNet on both synthetic and real-world datasets.

ABSTRACT

We propose a simple yet effective deep tree-structured fusion model based on feature aggregation for the deraining problem. We argue that by effectively aggregating features, a relatively simple network can still handle tough image deraining problems well. First, to capture the spatial structure of rain we use dilated convolutions as our basic network block. We then design a tree-structured fusion architecture which is deployed within each block (spatial information) and across all blocks (content information). Our method is based on the assumption that adjacent features contain redundant information. This redundancy obstructs generation of new representations and can be reduced by hierarchically fusing adjacent features. Thus, the proposed model is more compact and can effectively use spatial and content information. Experiments on synthetic and real-world datasets show that our network achieves better deraining results with fewer parameters.

Motivation & Objective

• To address the challenge of single image deraining, where rain streaks degrade image quality and impair downstream vision tasks.
• To reduce feature redundancy in deep networks, which hinders the generation of new, meaningful representations.
• To improve deraining performance using a compact network architecture with fewer parameters than existing methods.
• To explore the effectiveness of tree-structured feature fusion in capturing both spatial and content information in rainy images.
• To demonstrate that simple, well-designed feature fusion can outperform deeper, more complex architectures.

Proposed method

• Uses multi-scale dilated convolutions as the basic block to capture spatial structures of rain streaks.
• Introduces a tree-structured fusion mechanism applied both within each block (spatial fusion) and across blocks (content fusion).
• Applies residual learning by predicting the residual between the rainy image and the derained output.
• Employs a combined loss function of MSE and SSIM to balance edge preservation and structural similarity.
• Designs a lightweight network with 8 dilated convolution blocks and a feature extraction/reconstruction head, minimizing parameter count.
• Uses standard CNN operations with skip connections to enable effective training and feature propagation.

Experimental results

Research questions

• RQ1 Can a simple, lightweight network with hierarchical feature fusion outperform deeper, more complex architectures in single image deraining?
• RQ2 How effective is tree-structured feature fusion in reducing redundancy and improving representation learning for deraining?
• RQ3 To what extent does combining MSE and SSIM losses improve visual quality and structural fidelity in derained images?
• RQ4 Can a compact network with fewer than 36,000 parameters achieve state-of-the-art performance on both synthetic and real-world deraining benchmarks?
• RQ5 How does the proposed fusion mechanism compare to standard skip connections in ResNet and dense connections in DenseNet?

Key findings

• The proposed model achieves a PSNR of 27.46 on the Rain100H dataset, outperforming ResNet (shallow: 25.71, deep: 28.03) and DenseNet (shallow: 25.87, deep: 28.24) under similar parameter counts.
• On the Rain1400 dataset, the model achieves a PSNR of 31.32 and SSIM of 0.92, demonstrating strong performance on large-scale real-world rainy images.
• The model uses only 35,427 parameters, significantly fewer than ResNet (186,483) and DenseNet (232,883) in deep configurations, while achieving better results.
• Visual comparisons show that the combined MSE + SSIM loss produces cleaner results with better contrast and fewer artifacts than using either loss alone.
• Feature map visualizations confirm that the tree-structured model generates more distinguishable and semantically meaningful features than deeper ResNet and DenseNet variants.
• The model achieves state-of-the-art performance on both synthetic (Rain100H) and real-world (Rain1400, Rain1200) datasets, validating its robustness and generalization.