[Paper Review] Strict Enforcement of Conservation Laws and Invertibility in CNN-Based Super Resolution for Scientific Datasets

Andrew Geiss, Joseph Hardin|arXiv (Cornell University)|Nov 11, 2020

Advanced Image Processing Techniques4 citations

TL;DR

This paper proposes Downsampling Enforcement (DE), a differentiable operator applied after the final layer of CNN-based super-resolution networks to ensure that high-resolution outputs exactly reproduce low-resolution inputs under 2D-average downsampling. The method improves performance across seven SISR architectures on benchmark and scientific datasets while strictly preserving physical conservation laws, with no changes to training or loss functions.

ABSTRACT

Recently, deep Convolutional Neural Networks (CNNs) have revolutionized image super-resolution (SR), dramatically outperforming past methods for enhancing image resolution. They could be a boon for the many scientific fields that involve image or gridded datasets: satellite remote sensing, radar meteorology, medical imaging, numerical modeling etc. Unfortunately, while SR-CNNs produce visually compelling outputs, they may break physical conservation laws when applied to scientific datasets. Here, a method for ``Downsampling Enforcement" in SR-CNNs is proposed. A differentiable operator is derived that, when applied as the final transfer function of a CNN, ensures the high resolution outputs exactly reproduce the low resolution inputs under 2D-average downsampling while improving performance of the SR schemes. The method is demonstrated across seven modern CNN-based SR schemes on several benchmark image datasets, and applications to weather radar, satellite imager, and climate model data are also shown. The approach improves training time and performance while ensuring physical consistency between the super-resolved and low resolution data.

Motivation & Objective

• To address the lack of physical consistency in CNN-based super-resolution when applied to scientific datasets such as satellite, radar, and climate model data.
• To enforce exact invertibility under 2D-average downsampling without modifying training procedures or loss functions.
• To improve performance and training efficiency in super-resolution networks while ensuring statistical consistency between low- and high-resolution data.
• To demonstrate the method’s effectiveness across diverse datasets, including real-world scientific applications like weather radar and climate modeling.

Proposed method

• A differentiable downsampling enforcement (DE) operator is derived and applied as the final transfer function in a CNN-based super-resolution network.
• The DE operator ensures that the output of the network, when downsampled via 2D-average pooling, exactly reconstructs the original low-resolution input.
• The method is implemented as a learnable layer that computes corrections to the network’s intermediate high-resolution output to satisfy the invertibility constraint.
• The correction is computed using a closed-form expression derived from the inverse of the 2D-average downsampling operation, enabling gradient flow during backpropagation.
• The approach is compatible with any existing SISR architecture and does not require changes to data, loss function, or training pipeline.
• A regularization term can be added to the loss function to reduce the magnitude of corrections, improving intermediate output quality without sacrificing final performance.

Experimental results

Research questions

• RQ1 Can a differentiable operator be designed to enforce exact invertibility in CNN-based super-resolution under 2D-average downsampling?
• RQ2 Does enforcing invertibility via this operator improve performance on standard and scientific image datasets?
• RQ3 To what extent does the method preserve physical conservation laws in scientific applications such as satellite imaging and climate modeling?
• RQ4 How does the magnitude of correction required by the DE operator evolve during training, and can it be minimized?
• RQ5 Does the method remain effective across diverse architectures and upsampling ratios, including large ones like x16?

Key findings

• The DE method improved performance across all seven tested SISR architectures on benchmark datasets, with no degradation in PSNR or SSIM.
• On the DIV2K dataset, the DE-EDRN model achieved a PSNR of 30.07 and SSIM of 0.8359, comparable to the non-DE baseline (30.26/0.8393).
• The method ensured exact invertibility: when the DE-processed HR output was downsampled via 2D-average pooling, it perfectly reproduced the original LR input.
• The average magnitude of correction applied by the DE operator was 108 before regularization, but dropped to 0.3 with a λ=100 regularization term.
• Checkerboard artifacts were observed, particularly at high upsampling ratios (e.g., x16), suggesting a potential limitation for large-scale resolution increases.
• The method was successfully applied to scientific datasets including weather radar, satellite imager, and climate model output, ensuring physical consistency and conservation of large-scale statistics.