[论文解读] Swift Parameter-free Attention Network for Efficient Super-Resolution

Cheng Wan, Hongyuan Yu+6 more|arXiv (Cornell University)|Nov 21, 2023

Advanced Image Processing Techniques被引用 7

一句话总结

SPAN 引入一种无参数注意力机制，由对称激活和残差构建，在不增加大量参数的前提下提升 SISR 的高频细节，同时保持较低的参数量和快速推理。

ABSTRACT

Single Image Super-Resolution (SISR) is a crucial task in low-level computer vision, aiming to reconstruct high-resolution images from low-resolution counterparts. Conventional attention mechanisms have significantly improved SISR performance but often result in complex network structures and large number of parameters, leading to slow inference speed and large model size. To address this issue, we propose the Swift Parameter-free Attention Network (SPAN), a highly efficient SISR model that balances parameter count, inference speed, and image quality. SPAN employs a novel parameter-free attention mechanism, which leverages symmetric activation functions and residual connections to enhance high-contribution information and suppress redundant information. Our theoretical analysis demonstrates the effectiveness of this design in achieving the attention mechanism's purpose. We evaluate SPAN on multiple benchmarks, showing that it outperforms existing efficient super-resolution models in terms of both image quality and inference speed, achieving a significant quality-speed trade-off. This makes SPAN highly suitable for real-world applications, particularly in resource-constrained scenarios. Notably, we won the first place both in the overall performance track and runtime track of the NTIRE 2024 efficient super-resolution challenge. Our code and models are made publicly available at https://github.com/hongyuanyu/SPAN.

研究动机与目标

• Motivate the need for fast, lightweight SR models that balance accuracy and inference speed.
• Propose a parameter-free attention mechanism to reduce model complexity.
• Develop SPAN with attention via symmetric activations and residuals to preserve information.
• Demonstrate SPAN's effectiveness against existing efficient SR models on standard benchmarks.

提出的方法

• Introduce SPAB (Swift Parameter-free Attention Block) that uses three 3x3 convolutions and a residual path.
• Compute attention map directly from convolution outputs using a symmetric origin-centered activation, without learnable parameters.
• Fuse features and attention via element-wise multiplication to produce SPAB outputs.
• Assemble six SPABs into SPAN with feature concatenation and PixelShuffle upsampling.
• Apply re-parameterization to optimize inference efficiency and report PSNR/SSIM on standard benchmarks.

实验结果

研究问题

• RQ1 Can a parameter-free attention mechanism improve SR performance without increasing model size?
• RQ2 How do symmetric origin-centered activations affect attention effectiveness in SR?
• RQ3 What is the trade-off between speed and accuracy for SPAN versus existing ESR models?
• RQ4 Do residual connections stabilize information flow and preserve details when using parameter-free attention?

主要发现

• SPAN achieves competitive PSNR/SSIM on Set5, Set14, BSD100, Urban100, and Manga109 while maintaining lower parameter counts.
• SPAN and SPAN-S show faster inference speeds than many ESR models at similar performance.
• In NTIRE 2023 efficient SR challenge, the approach achieved the best PSNR of 27.09 dB with a 7.08 ms runtime reduction.
• Ablation studies confirm residual connections and parameter-free attention improve quality and speed compared to variants without them.