[Paper Review] Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

Fei Mi, Liangwei Chen+3 more|arXiv (Cornell University)|Oct 2, 2020

Topic Modeling50 references4 citations

TL;DR

This paper proposes ARPER (Adaptively Regularized Prioritized Exemplar Replay), a continual learning method for natural language generation in task-oriented dialog systems that mitigates catastrophic forgetting by combining prioritized exemplar replay with adaptive regularization based on Elastic Weight Consolidation. ARPER significantly outperforms existing methods on MultiWoZ-2.0, reducing error rates by up to 4.45% in slot error rate across six domains while maintaining a small memory footprint.

ABSTRACT

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applications where new data come in a stream, we study NLG in a "continual learning" setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to forget the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on ElasticWeight Consolidation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outperforms other methods by effectively mitigating the detrimental catastrophic forgetting issue.

Motivation & Objective

• To address catastrophic forgetting in neural natural language generation (NLG) models when incrementally learning new domains and intents in real-world dialog systems.
• To develop a scalable continual learning framework for NLG that avoids retraining on all historical data.
• To improve knowledge retention of previously learned utterance patterns during incremental training on new data.
• To combine exemplar replay with adaptive regularization to strengthen knowledge consolidation without increasing model complexity.
• To evaluate the method across diverse NLG architectures and continual learning scenarios on a standard benchmark.

Proposed method

• Proposes ARPER, a method that combines prioritized exemplar replay with adaptive regularization based on Elastic Weight Consolidation (EWC).
• Uses a Herding-based selection strategy to identify diverse and representative exemplars from past data for efficient replay.
• Applies EWC regularization using an approximation of the Fisher Information Matrix computed on selected exemplars to protect important parameters.
• Introduces adaptive regularization weights that dynamically adjust based on the distributional shift between new and old data.
• Replays selected exemplars during each training phase to reinforce memory of prior knowledge.
• Employs a dual-component training strategy: exemplar replay for memory retention and adaptive regularization for parameter stability.

Experimental results

Research questions

• RQ1 Can exemplar replay with prioritized selection effectively reduce catastrophic forgetting in NLG for continual learning?
• RQ2 How does adaptive regularization based on EWC compare to fixed or non-adaptive regularization in maintaining performance on previously learned domains?
• RQ3 Does ARPER generalize across different base NLG models, such as SCLSTM, SCVAE, and GPT-2?
• RQ4 How does ARPER perform in comparison to state-of-the-art continual learning methods in a realistic, incremental learning setup?
• RQ5 What is the impact of exemplar selection strategy (prioritized vs. random) on long-term knowledge retention in NLG?

Key findings

• ARPER achieves a slot error rate of 4.45% on MultiWoZ-2.0 when using SCLSTM as the base model, outperforming the next best method (ER prio + Dropout) by 4.04 percentage points.
• The ablation study confirms that both prioritized exemplars and adaptive regularization are essential, as removing either component degrades performance significantly.
• On GPT-2, ARPER achieves a slot error rate of 5.32%, demonstrating generalization across different NLG architectures despite GPT-2’s inherent robustness to forgetting.
• Case studies show ARPER successfully preserves old slot patterns (e.g., 'Fee' and 'Type') while learning new domains, whereas baselines suffer from slot leakage and repetition errors.
• The method reduces catastrophic forgetting by effectively balancing memory replay and parameter regularization, even with only 250 exemplars stored.
• ARPER maintains strong performance across all domains, including the first domain, with a final error rate of 4.04% on the first domain, indicating minimal forgetting of initial knowledge.