[Paper Review] Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy

Pengfei Yu, Heng Ji|arXiv (Cornell University)|May 29, 2023

Topic Modeling4 citations

TL;DR

This paper proposes a novel self-information updating (SIU) framework that mitigates language model-logical discrepancy by using self-prompting-based question-answer generation and associative distillation to improve factual consistency and generalization in LLM updating. The method achieves up to a 0.16 increase in factual consistency score and reduces replay buffer size by 97.7%, effectively minimizing catastrophic forgetting with only 2.3% of original training tokens.

ABSTRACT

Large Language Models~(LLMs) struggle with providing current information due to the outdated pre-training data. Existing methods for updating LLMs, such as knowledge editing and continual fine-tuning, have significant drawbacks in generalizability of new information and the requirements on structured updating corpus. We identify the core challenge behind these drawbacks: the LM-logical discrepancy featuring the difference between language modeling probabilities and logical probabilities. To evaluate and address the core challenge, we propose a new task formulation of the information updating task that only requires the provision of an unstructured updating corpus and evaluates the performance of information updating on the generalizability to question-answer pairs pertaining to the updating information. We further propose a novel and effective pipeline approach for the task, highlighting a self-prompting-based question-answer generation process and a associative distillation methods to bridge the LM-logical discrepancy. We develop two datasets for evaluation, one sourced from news articles published in March and April 2023, and the other from the Natural Questions benchmark. Experimental results demonstrate the superiority of our approach, significantly increasing the factual consistency score (on a scale from 0 to 1) by up to 0.16. Furthermore, our method effectively mitigates forgetting utilizing a compact replay buffer with only 2.3% of the training tokens.

Motivation & Objective

• To address the core challenge of LM-logical discrepancy, which limits generalizability and necessitates structured corpora in existing LLM updating methods.
• To propose a new task formulation, Self Information Updating (SIU), that enables updating with only unstructured corpora, such as news articles.
• To improve factual consistency and generalization of updated information across question-answer pairs without relying on structured knowledge triples.
• To mitigate catastrophic forgetting in continual learning by using a compact replay buffer derived through associative distillation.
• To evaluate the effectiveness of the method on unstructured updating corpora and its robustness across diverse text domains.

Proposed method

• The method introduces a self-prompting-based question-answer generation process to create QA pairs from unstructured updating corpora, enabling better association of new information with relevant contexts.
• It employs associative distillation to align language model probabilities with logical probabilities, reducing exposure bias toward pre-training data.
• The approach uses a compact replay buffer—reduced to 2.3% of original tokens—by distilling and re-generating answers from the updated corpus, preserving past knowledge.
• The pipeline integrates QA generation and distillation in a two-stage fine-tuning process: first on news data, then on Wikipedia, with evaluation on both new and past information.
• Factual consistency is evaluated using the Zhong et al. (2022) metric, emphasizing information acquisition over preference alignment.
• The method is designed to be compatible with existing knowledge editing and continual fine-tuning frameworks, enhancing their generalization and efficiency.

Experimental results

Research questions

• RQ1 How can language models be effectively updated with unstructured corpora while maintaining high factual consistency?
• RQ2 To what extent does LM-logical discrepancy limit the generalization of updated information in LLMs?
• RQ3 Can associative distillation effectively reduce exposure bias and improve the model's association of new information with relevant contexts?
• RQ4 How small can a replay buffer be while still preserving past knowledge and preventing catastrophic forgetting?
• RQ5 Does the proposed method generalize across different text domains, such as news and Wikipedia, without requiring structured editing inputs?

Key findings

• The proposed method increases factual consistency by up to 0.16 on a scale from 0 to 1, significantly outperforming baseline methods in information generalization.
• The associative distillation process reduces the required replay buffer size by 97.7%, from 919,624 to 21,124 tokens, while maintaining performance on past knowledge.
• The model shows only minor performance degradation on the NQ validation set when using a replay buffer of just 2.3% of the original training tokens, indicating strong forgetting mitigation.
• The method effectively generalizes updated information to question-answer pairs not explicitly in the training data, reducing reliance on pre-training data.
• The approach maintains strong performance across two distinct updating stages—news and Wikipedia—demonstrating robustness to domain shifts.
• The method is effective even with moderately sized updating corpora and shows promise for integration with existing editing and fine-tuning frameworks.