[Paper Review] EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining

Qing Guo, Jingyang Sun+5 more|arXiv (Cornell University)|Sep 19, 2020

Image Enhancement Techniques43 references17 citations

TL;DR

This paper proposes EfficientDeRain, a model-free, single-stage deraining method that uses pixel-wise dilation filtering with a learnable kernel prediction network to achieve real-time inference (under 10 ms per image), over 80× faster than SOTA methods like RCDNet, while maintaining high de-rain performance. It further introduces RainMix, a data augmentation strategy that bridges the sim-to-real generalization gap for real-world rainy images.

ABSTRACT

Single-image deraining is rather challenging due to the unknown rain model. Existing methods often make specific assumptions of the rain model, which can hardly cover many diverse circumstances in the real world, making them have to employ complex optimization or progressive refinement. This, however, significantly affects these methods' efficiency and effectiveness for many efficiency-critical applications. To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i.e., EfficientDeRain, which is able to process a rainy image within 10~ms (i.e., around 6~ms on average), over 80 times faster than the state-of-the-art method (i.e., RCDNet), while achieving similar de-rain effects. We first propose the novel pixel-wise dilation filtering. In particular, a rainy image is filtered with the pixel-wise kernels estimated from a kernel prediction network, by which suitable multi-scale kernels for each pixel can be efficiently predicted. Then, to eliminate the gap between synthetic and real data, we further propose an effective data augmentation method (i.e., RainMix) that helps to train network for real rainy image handling.We perform comprehensive evaluation on both synthetic and real-world rainy datasets to demonstrate the effectiveness and efficiency of our method. We release the model and code in https://github.com/tsingqguo/efficientderain.git.

Motivation & Objective

• To address the inefficiency of existing deraining methods that rely on complex optimization or progressive refinement.
• To eliminate dependency on specific rain model assumptions, which limits generalization to real-world scenarios.
• To develop a high-efficiency, model-free deraining solution suitable for real-time applications like autonomous driving.
• To bridge the domain gap between synthetic and real rainy images through effective data augmentation.

Proposed method

• Proposes pixel-wise dilation filtering, where each pixel is processed by a multi-scale kernel predicted by a dedicated kernel prediction network.
• Employs a lightweight, single-stage architecture without iterative optimization or progressive refinement, enabling high inference speed.
• Introduces RainMix, a data augmentation technique that mixes synthetic rainy images with real rainy patches to improve robustness on real-world data.
• Uses a hybrid loss function combining L1 and SSIM to balance structural preservation and pixel-level accuracy.
• Trains the kernel prediction network end-to-end to adaptively estimate spatially varying kernels per pixel based on local image content.
• Applies the learned kernels via dilated convolutions to effectively suppress multi-scale rain streaks while preserving object boundaries.

Experimental results

Research questions

• RQ1 Can a model-free, single-stage deraining method achieve both high performance and ultra-high efficiency without relying on explicit rain model assumptions?
• RQ2 How effective is pixel-wise dilation filtering with learnable, spatially adaptive kernels in suppressing diverse rain patterns?
• RQ3 To what extent does RainMix improve generalization to real-world rainy images compared to standard data augmentation?
• RQ4 What is the trade-off between PSNR/SSIM performance and inference speed across different deraining architectures?

Key findings

• EfficientDeRain achieves an average inference time of 6 ms per image, over 80× faster than the SOTA method RCDNet.
• The final model variant (v4) with RainMix, dilation filtering, and combined L1+SSIM loss achieves the highest PSNR and SSIM on Rain100H, demonstrating the effectiveness of all components.
• The addition of the dilation structure increases inference time by only ~1.5 ms, indicating minimal efficiency cost for improved performance.
• Visualizations show that RainMix significantly enhances the model's ability to remove heavy, diverse rain streaks in real-world images, which standard training fails to address.
• On the SPA real-world dataset, models trained without RainMix consistently leave residual rain traces (evident in red arrows), confirming its critical role in real-world robustness.
• The ablation study confirms that each component—dilation filtering, RainMix, and the combined loss—contributes incrementally to performance, with the full pipeline achieving optimal results.