[논문 리뷰] Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections

Xiao-Jiao Mao, Chunhua Shen외 1명|arXiv (Cornell University)|2016. 06. 29.

Advanced Image Processing Techniques참고 문헌 20인용 수 343

한 줄 요약

논문은 대단히 깊은 완전 컨벌루셔널 오토인코더를 대칭 스킵 연결(RED-Net)과 함께 도입하여 이미지 복원 작업에 활용하며, 하나의 모델로 최첨단 잡음 제거, 초해상도, JPEG 디블로킹, 비블라인드 디블러링, 인페인팅을 가능하게 한다.

ABSTRACT

Image restoration, including image denoising, super resolution, inpainting, and so on, is a well-studied problem in computer vision and image processing, as well as a test bed for low-level image modeling algorithms. In this work, we propose a very deep fully convolutional auto-encoder network for image restoration, which is a encoding-decoding framework with symmetric convolutional-deconvolutional layers. In other words, the network is composed of multiple layers of convolution and de-convolution operators, learning end-to-end mappings from corrupted images to the original ones. The convolutional layers capture the abstraction of image contents while eliminating corruptions. Deconvolutional layers have the capability to upsample the feature maps and recover the image details. To deal with the problem that deeper networks tend to be more difficult to train, we propose to symmetrically link convolutional and deconvolutional layers with skip-layer connections, with which the training converges much faster and attains better results.

연구 동기 및 목표

• Develop a very deep, fully convolutional encoder-decoder network for image restoration tasks.
• Enhance trainability and restoration performance via symmetric skip connections between corresponding encoder and decoder layers.
• Demonstrate a single, scalable model that can handle multiple corruption types (denoising, super-resolution, deblocking, deblurring, inpainting).

제안 방법

• Propose RED-Net, a very deep symmetric convolutional auto-encoder with mirrored convolutional and deconvolutional layers.
• Incorporate skip connections between corresponding encoder and decoder layers to improve gradient flow and preserve image details.
• Train end-to-end to learn mappings from corrupted images to clean ones, optimizing the residual (Y-X) rather than the direct mapping.
• Use ReLU activations after each layer and train with Adam on patches from BSD, enabling deep architectures without pooling/unpooling.
• Allow testing on images of arbitrary size due to symmetric, fully convolutional structure; employ multi-orientation testing for ensembling gains.
• Compare to state-of-the-art methods across multiple restoration tasks without image priors.

실험 결과

연구 질문

• RQ1 Can a very deep fully convolutional encoder-decoder with symmetric skip connections improve image restoration performance across multiple tasks?
• RQ2 Do skip connections enable effective training of deeper networks and better preservation of fine image details in restoration?
• RQ3 Is a single RED-Net model capable of handling diverse corruptions (denoising, super-resolution, JPEG deblocking, non-blind deblurring, inpainting) with competitive or superior results?

주요 결과

• RED-Net achieves best reported performance on four image restoration tasks in their experiments.
• Deeper networks with symmetric skip connections train more effectively and preserve image details better than shallower or non-skipp-connected architectures.
• Skip connections facilitate gradient back-propagation to lower layers, reducing gradient vanishing in very deep networks.
• The model supports different corruption levels using a single, high-capacity network.
• Testing efficiency can be improved by down-sampling in early convolutional layers with up-sampling in symmetric deconvolutional layers, with minimal PSNR loss.
• Network variants RED10, RED20, and RED30 demonstrate the benefits of depth and skip connections for restoration tasks.