[Paper Review] MEMORYLLM: Towards Self-Updatable Large Language Models

Yu Wang, Gao, Yifan+10 more|arXiv (Cornell University)|Feb 7, 2024

Topic Modeling4 citations

TL;DR

MemoryLLM introduces a self-updatable large language model with a fixed-size memory pool embedded in the transformer's latent space, enabling efficient knowledge injection and long-term retention without performance degradation. After nearly a million updates, the model maintains functionality and outperforms baselines in model editing and long-context reasoning tasks.

ABSTRACT

Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the latent space of the transformer. MEMORYLLM can self-update with text knowledge and memorize the knowledge injected earlier. Our evaluations demonstrate the ability of MEMORYLLM to effectively incorporate new knowledge, as evidenced by its performance on model editing benchmarks. Meanwhile, the model exhibits long-term information retention capacity, which is validated through our custom-designed evaluations and long-context benchmarks. MEMORYLLM also shows operational integrity without any sign of performance degradation even after nearly a million memory updates. Our code and model are open-sourced at https://github.com/wangyu-ustc/MemoryLLM.

Motivation & Objective

• To address the challenge of updating large language models with new knowledge after deployment, where existing methods suffer from redundancy, context overload, or performance degradation.
• To design a model that efficiently integrates new knowledge without requiring back-propagation or extensive fine-tuning.
• To ensure long-term knowledge retention by enabling gradual forgetting of outdated information while preserving model integrity.
• To maintain model performance and avoid degradation even after extensive memory updates, up to nearly one million steps.
• To reduce knowledge redundancy by using a compact, fixed-size memory pool that compresses and stores knowledge efficiently.

Proposed method

• MemoryLLM integrates a fixed-size memory pool within the latent space of each transformer layer, using learnable memory tokens to represent compressed knowledge.
• The model employs a self-update mechanism that propagates new knowledge to all layers, updating only a proportion of memory tokens per step to enable gradual forgetting.
• Knowledge injection is performed without back-propagation, relying on direct parameter updates to the memory pool for efficiency.
• The memory pool is trained to store and retrieve knowledge, with the model's attention mechanisms dynamically attending to relevant memory tokens during inference.
• The architecture separates persistent knowledge (modeled by the base LLM parameters) from updatable knowledge (stored in the memory pool), enabling targeted updates.
• A curated training process ensures that memory updates do not disrupt the model’s original capabilities, maintaining operational integrity.

(a) Generation

Experimental results

Research questions

• RQ1 Can a large language model effectively integrate new knowledge through a self-updating memory mechanism without requiring back-propagation?
• RQ2 How well does the model retain previously learned knowledge after repeated updates, and what is the rate of forgetting?
• RQ3 To what extent can the memory pool support long-context reasoning and model editing tasks compared to existing methods?
• RQ4 Does the model maintain performance and functionality after a very large number of memory updates, such as nearly one million?
• RQ5 Can a fixed-size memory pool store knowledge more compactly and with less redundancy than traditional retrieval or long-context approaches?

Key findings

• MemoryLLM demonstrated strong performance on model editing benchmarks, showing effective and efficient integration of new facts without requiring fine-tuning or back-propagation.
• The model maintained high accuracy across long-context QA tasks, indicating robust knowledge retention even after extensive updates.
• After nearly one million memory update steps, MemoryLLM showed no observable performance degradation, confirming its operational integrity.
• The model achieved significant improvements in knowledge retention, as validated through custom-designed retention evaluations and long-context benchmarks.
• The memory pool enabled effective compression of knowledge, reducing redundancy compared to traditional retrieval-based or long-context methods.
• The self-update mechanism allowed for gradual forgetting of outdated knowledge while preserving previously stored information, ensuring long-term utility.

(b) Self-Update