[Paper Review] Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search

Xiangxiang Chu, Bo Zhang+3 more|arXiv (Cornell University)|Jan 22, 2019

Advanced Image Processing Techniques18 references86 citations

TL;DR

The paper introduces FALSR, a multi-objective neural architecture search framework for fast, accurate, and lightweight image super-resolution, using elastic micro/macro search and a hybrid RL-EA generator to produce Pareto-optimal models.

ABSTRACT

Deep convolutional neural networks demonstrate impressive results in the super-resolution domain. A series of studies concentrate on improving peak signal noise ratio (PSNR) by using much deeper layers, which are not friendly to constrained resources. Pursuing a trade-off between the restoration capacity and the simplicity of models is still non-trivial. Recent contributions are struggling to manually maximize this balance, while our work achieves the same goal automatically with neural architecture search. Specifically, we handle super-resolution with a multi-objective approach. We also propose an elastic search tactic at both micro and macro level, based on a hybrid controller that profits from evolutionary computation and reinforcement learning. Quantitative experiments help us to draw a conclusion that our generated models dominate most of the state-of-the-art methods with respect to the individual FLOPS.

Motivation & Objective

• Automate the design of SR models that balance restoration quality and resource efficiency.
• Explore a multi-objective NAS with micro and macro search spaces for SR architectures.
• Leverage a hybrid controller (reinforcement learning + evolution) to generate diverse, high-performing models.
• Produce practical SR models under given constraints with a single run.

Proposed method

• Define a cell-level micro search space and an intercell macro search space for SR networks.
• Use a hybrid controller combining RL-based exploitation and an EA-based exploration within NSGA-II to generate model architectures.
• Treat SR as a constrained multi-objective optimization with objectives: PSNR, mult-adds, and parameter count.
• Encode models with a two-part chromosome (M_mic, M_mac) and evolve via crossover and mutation strategies.
• Employ incomplete training to quickly evaluate many models and select promising candidates for full training.
• Compare generated models against state-of-the-art SR methods on standard benchmarks (e.g., DIV2K) under comparable FLOPs.

Experimental results

Research questions

• RQ1 Can elastic micro/macro NAS spaces produce SR architectures that outperform manually designed networks under similar computational budgets?
• RQ2 Does a hybrid RL-EA controller improve exploration-exploitation trade-offs in multi-objective NAS for SR?
• RQ3 How do diverse, cell-level architectures affect SR performance versus dense/monolithic designs?
• RQ4 Are lightweight FALSR variants able to match or surpass state-of-the-art SR methods within constrained FLOPs and parameters?

Key findings

• FALSR-A, FALSR-B, and FALSR-C achieve competitive PSNR/SSIM with lower or comparable FLOPs and parameters across standard benchmarks.
• Elastic search across micro and macro spaces yields diverse cells and connection patterns that outperform several baselines under similar FLOPs.
• At a fixed FLOP budget, FALSR models often surpass or closely approach state-of-the-art methods like CARN, CARN-M, and DRRN on multiple datasets.
• Dense interconnections are not always optimal; sparse or mixed connectivity with diverse cells can yield better SR performance under resource constraints.
• Incomplete training enables efficient screening of thousands of models, with full training reserved for Pareto-front candidates.
• The Pareto front demonstrates a trade-off between PSNR accuracy and computational cost, with FALSR variants occupying favorable regions.