[Paper Review] Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data

Zhipeng Cai, Ozan Şener+1 more|arXiv (Cornell University)|Aug 20, 2021

Domain Adaptation and Few-Shot Learning33 references4 citations

TL;DR

This paper introduces a new benchmark for online continual visual learning using geotagged images from YFCC100M, revealing that gradient-based optimization behaves fundamentally differently under natural distribution shifts compared to offline settings. It identifies batch size increases as harmful to both learning efficacy and retention, and proposes adaptive learning rate and replay buffer size strategies that significantly improve performance in real-world online continual learning scenarios.

ABSTRACT

Continual learning is the problem of learning and retaining knowledge through time over multiple tasks and environments. Research has primarily focused on the incremental classification setting, where new tasks/classes are added at discrete time intervals. Such an "offline" setting does not evaluate the ability of agents to learn effectively and efficiently, since an agent can perform multiple learning epochs without any time limitation when a task is added. We argue that "online" continual learning, where data is a single continuous stream without task boundaries, enables evaluating both information retention and online learning efficacy. In online continual learning, each incoming small batch of data is first used for testing and then added to the training set, making the problem truly online. Trained models are later evaluated on historical data to assess information retention. We introduce a new benchmark for online continual visual learning that exhibits large scale and natural distribution shifts. Through a large-scale analysis, we identify critical and previously unobserved phenomena of gradient-based optimization in continual learning, and propose effective strategies for improving gradient-based online continual learning with real data. The source code and dataset are available in: https://github.com/IntelLabs/continuallearning.

Motivation & Objective

• To address the gap in evaluating learning efficacy in continual learning by shifting from offline to online continual learning settings with real-world data.
• To construct a large-scale, naturally evolving visual benchmark using geotagged images to simulate real-world distribution shifts.
• To empirically analyze gradient-based optimization behavior in online continual learning under non-stationary data distributions.
• To identify critical, previously unobserved phenomena—such as the negative impact of batch size—under online continual learning.
• To propose adaptive strategies for learning rate and replay buffer size that improve performance in online continual learning.

Proposed method

• Constructs a new benchmark using 39 million geotagged images from YFCC100M spanning 9 years to simulate natural distribution shifts in visual data.
• Employs online continual geolocalization as the task, where models process data in small batches in real time, with immediate testing and incremental training.
• Uses a fixed computation and memory budget per time step, enforcing true online learning without task boundaries or multiple training epochs.
• Employs experience replay with First-In-First-Out (FIFO) buffer strategy, comparing it to reservoir replay for buffer update policies.
• Applies adaptive learning rate and replay buffer size adaptation algorithms to improve optimization stability and performance.
• Conducts ablation studies on batch size effects in both supervised learning and online continual learning with and without replay.

Experimental results

Research questions

• RQ1 How does gradient-based optimization behave in online continual learning under natural, non-stationary data distributions compared to offline settings?
• RQ2 What is the impact of increasing batch size on learning efficacy and information retention in online continual learning with real-world data?
• RQ3 How do different replay buffer update strategies (FIFO vs. reservoir) affect performance in online continual learning?
• RQ4 Can adaptive learning rate and replay buffer size strategies improve online continual learning performance under distribution shifts?
• RQ5 Is the negative effect of batch size in online continual learning due to data characteristics or inherent to the online learning setup?

Key findings

• Increasing batch size in online continual learning significantly harms both learning efficacy and information retention, even when using experience replay.
• The negative impact of batch size persists even without experience replay, indicating it is inherent to the online continual learning setup rather than a side effect of replay mechanisms.
• Supervised learning with shuffled data shows no such negative batch size effect, suggesting the issue is specific to online continual learning dynamics.
• FIFO and reservoir replay buffer strategies perform comparably, with no significant advantage for either, leading to the choice of FIFO for simplicity.
• Learning efficacy and information retention are conflicting objectives in online continual learning, requiring careful optimization trade-offs.
• Adaptive learning rate and replay buffer size strategies significantly improve performance, demonstrating their effectiveness in mitigating optimization challenges under distribution shifts.