[Paper Review] COIN: COmpression with Implicit Neural representations

Emilien Dupont, Adam Goliński+3 more|arXiv (Cornell University)|Mar 3, 2021

Advanced Data Compression Techniques32 references25 citations

TL;DR

COIN compresses images by overfitting a small MLP to the image and storing the quantized MLP weights as the code; decoding evaluates the MLP at all pixel locations. It outperforms JPEG at low bit-rates without entropy coding.

ABSTRACT

We propose a new simple approach for image compression: instead of storing the RGB values for each pixel of an image, we store the weights of a neural network overfitted to the image. Specifically, to encode an image, we fit it with an MLP which maps pixel locations to RGB values. We then quantize and store the weights of this MLP as a code for the image. To decode the image, we simply evaluate the MLP at every pixel location. We found that this simple approach outperforms JPEG at low bit-rates, even without entropy coding or learning a distribution over weights. While our framework is not yet competitive with state of the art compression methods, we show that it has various attractive properties which could make it a viable alternative to other neural data compression approaches.

Motivation & Objective

• Motivate and demonstrate a simple neural data compression approach that treats image compression as model compression.
• Show that overfitting a small MLP to an image and storing its quantized weights can outperform JPEG at low bit-rates.
• Explore the trade-offs and practical properties of implicit neural representations for compression.
• Assess whether this approach can be competitive with state-of-the-art methods and identify future directions for enhancement.

Proposed method

• Represent images as functions f_theta: (x,y) -> RGB using an MLP with sine activations (SIREN) to capture high-frequency details.
• Overfit f_theta to the image by minimizing MSE between f_theta(x,y) and I[x,y].
• Quantize and store the MLP weights theta as the compressed code, effectively turning data compression into model compression.
• Decode by evaluating f_theta on every pixel location to reconstruct the image.
• Optionally explore architecture search and 16-bit weight quantization to balance rate and distortion.
• Note: encoding is computation-intensive per image, but decoding is straightforward and parallelizable.

Experimental results

Research questions

• RQ1 Can implicit neural representations learned per image provide competitive rate-distortion performance at low bit-rates compared to traditional codecs?
• RQ2 How does architecture choice and weight quantization impact the rate-distortion trade-off for COIN?
• RQ3 What are the practical advantages and limitations of encoding via per-image overfitted networks versus conventional autoencoder-based neural compression?
• RQ4 Can the approach be extended with learned weight distributions or meta-learning to close the gap to state-of-the-art methods?

Key findings

• At low bit-rates, COIN outperforms JPEG even without entropy coding.
• Quantizing weights from 32-bit to 16-bit preserves performance with minimal distortion versus 8-bit which degrades quality.
• The method achieves very small decoder-side memory (e.g., 14 kB at 0.3 bpp) compared with autoencoder baselines.
• Encoding is slow due to per-image optimization, but decoding is fast and highly parallelizable.
• Different architectures yield different optimalities for given bit-rate budgets, indicating a need for architecture search or pruning.