[Paper Review] Online Learned Continual Compression with Adaptive Quantization Modules

Lucas Caccia, Eugene Belilovsky+2 more|arXiv (Cornell University)|Nov 19, 2019

Domain Adaptation and Few-Shot Learning14 references38 citations

TL;DR

The paper introduces Online Continual Compression (OCC) using Adaptive Quantization Modules (AQM) built on VQ-VAE to compress and store a non-iid data stream within a fixed memory, while ensuring previous representations are decodable by future decoders and enabling online learning across tasks.

ABSTRACT

We introduce and study the problem of Online Continual Compression, where one attempts to simultaneously learn to compress and store a representative dataset from a non i.i.d data stream, while only observing each sample once. A naive application of auto-encoders in this setting encounters a major challenge: representations derived from earlier encoder states must be usable by later decoder states. We show how to use discrete auto-encoders to effectively address this challenge and introduce Adaptive Quantization Modules (AQM) to control variation in the compression ability of the module at any given stage of learning. This enables selecting an appropriate compression for incoming samples, while taking into account overall memory constraints and current progress of the learned compression. Unlike previous methods, our approach does not require any pretraining, even on challenging datasets. We show that using AQM to replace standard episodic memory in continual learning settings leads to significant gains on continual learning benchmarks. Furthermore we demonstrate this approach with larger images, LiDAR, and reinforcement learning environments.

Motivation & Objective

• Define the Online Continual Compression (OCC) problem and its challenges (representation drift, forgetting, non-stationary data).
• Propose Adaptive Quantization Modules (AQM) based on VQ-VAE to manage multi-level compression and memory.
• Develop self-replay and stream sampling to maintain memory quality under fixed capacity.
• Demonstrate effectiveness across online continual image classification, larger images, LiDAR, and RL settings.

Proposed method

• Use a sequence of VQ-VAEs with separate codebooks per level to discretize representations.
• Introduce an adaptive multi-level storage scheme where each level stores codebook indices with varying bit-rates.
• Employ self-replay to update the AQM using both new data and randomly sampled stored representations.
• Implement a stream sampling strategy (non-uniform, memory-aware) to delete stored samples while maximizing representation of the distribution.
• Stabilize representations by freezing codebooks for drift control while allowing encoder/decoder to adapt.
• Provide online algorithm outlining data flow, encoding, storage, and memory updates (Algorithm 1).

Experimental results

Research questions

• RQ1 Can online learned compression store and retrieve past representations despite representation drift as the encoder evolves?
• RQ2 Does adaptive, multi-level quantization improve memory efficiency and reconstruction quality in online continual learning?
• RQ3 Can internal replay and streaming sampling reduce forgetting while staying within fixed storage?
• RQ4 How does OCC with AQM perform on online continual image classification and extend to larger images, LiDAR, and RL observations?

Key findings

• AQM with vector-quantized VAEs effectively mitigates representation drift in online continual settings.
• Multi-level storage and self-replay enable superior accuracy and reduced forgetting compared to baselines like ER with JPEG, especially at smaller memory budgets.
• On CIFAR-10 continual learning benchmarks, AQM substantially outperforms ER and other baselines across memory sizes (e.g., improvements over ER by significant margins and lower forgetting).
• Offline evaluation on ImageNet shows gains from compressed memories over reservoir sampling of uncompressed data, with ablations confirming the importance of each component (codebook stabilization, decoupled training, adaptive compression, and drift control).
• AQM scales to larger modalities (ImageNet, LiDAR) and RL observations, delivering compression (e.g., 32x) with preserved task-relevant information.