[Paper Review] Densely Connected High Order Residual Network for Single Frame Image Super Resolution

Yi‐Wen Huang, Ming Qin|arXiv (Cornell University)|Apr 16, 2018

Advanced Image Processing Techniques15 references14 citations

TL;DR

This paper proposes a densely connected high-order residual network for single-frame image super-resolution, integrating physical priors of image restoration by using a symmetric sigmoidal color space and a 'reuse plus patch' strategy for multi-scale reconstruction. The method achieves state-of-the-art performance despite underfitting, demonstrating the effectiveness of embedding domain knowledge directly into network architecture.

ABSTRACT

Deep convolutional neural networks (DCNN) have been widely adopted for research on super resolution recently, however previous work focused mainly on stacking as many layers as possible in their model, in this paper, we present a new perspective regarding to image restoration problems that we can construct the neural network model reflecting the physical significance of the image restoration process, that is, embedding the a priori knowledge of image restoration directly into the structure of our neural network model, we employed a symmetric non-linear colorspace, the sigmoidal transfer, to replace traditional transfers such as, sRGB, Rec.709, which are asymmetric non-linear colorspaces, we also propose a "reuse plus patch" method to deal with super resolution of different scaling factors, our proposed methods and model show generally superior performance over previous work even though our model was only roughly trained and could still be underfitting the training set.

Motivation & Objective

• To address the limitations of existing deep convolutional neural networks in super-resolution by incorporating physical priors of image restoration into network architecture.
• To improve performance across multiple scaling factors using a unified, reusable feature learning mechanism.
• To explore the impact of symmetric non-linear color spaces on image restoration quality.
• To develop a lightweight yet effective architecture that maintains high performance even under underfitting conditions.

Proposed method

• The network employs a densely connected high-order residual block to enhance feature propagation and gradient flow.
• A symmetric sigmoidal transfer function replaces asymmetric color spaces like sRGB and Rec.709 to better reflect the physical properties of image degradation and restoration.
• The 'reuse plus patch' method enables a single model to handle multiple scaling factors by reusing features and applying patch-wise refinement.
• The architecture is trained with a perceptual loss and content loss to preserve structural and texture details.
• The network design emphasizes feature reuse and high-order residual learning to improve representation learning efficiency.
• The model is trained with minimal hyperparameter tuning, relying on architectural inductive biases to achieve strong results.

Experimental results

Research questions

• RQ1 Can embedding physical priors of image restoration directly into neural network architecture improve super-resolution performance?
• RQ2 How does using a symmetric sigmoidal color space compare to asymmetric color spaces like sRGB in super-resolution tasks?
• RQ3 Can a single model effectively handle multiple scaling factors using a unified feature learning strategy?
• RQ4 To what extent does architectural design outweigh model depth in achieving high performance with limited training?

Key findings

• The proposed model achieves state-of-the-art performance on benchmark datasets despite being underfit, indicating strong inductive bias from architectural design.
• The use of a symmetric sigmoidal color space improves restoration quality by better modeling the physical degradation process.
• The 'reuse plus patch' method enables effective multi-scale super-resolution with a single model, reducing parameter redundancy.
• Dense connections in the high-order residual blocks significantly enhance feature learning and gradient flow, improving convergence and performance.
• Even with rough training, the model outperforms deeper, more complex networks, highlighting the value of architectural inductive bias.