[Paper Review] A Note on Normalized Emergence Timing (in Pythia Language Model Evaluations)

Rooks, Tyler Cason|arXiv (Cornell University)|Apr 3, 2023

Topic Modeling163 citations

TL;DR

This paper introduces the Pythia suite of publicly available LLMs trained on identical data order and analyzes training dynamics, scaling effects, and case studies on bias, memorization, and term-frequency impacts.

ABSTRACT

How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce extit{Pythia}, a suite of 16 LLMs all trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters. We provide public access to 154 checkpoints for each one of the 16 models, alongside tools to download and reconstruct their exact training dataloaders for further study. We intend extit{Pythia} to facilitate research in many areas, and we present several case studies including novel results in memorization, term frequency effects on few-shot performance, and reducing gender bias. We demonstrate that this highly controlled setup can be used to yield novel insights toward LLMs and their training dynamics. Trained models, analysis code, training code, and training data can be found at \url{https://github.com/EleutherAI/pythia}.

Motivation & Objective

• Promote scientific research on large language models by providing a standardized, publicly accessible model suite.
• Investigate how training data order, deduplication, and model size influence learning dynamics and biases.
• Examine the role of pretraining term frequencies on downstream task performance across training progression.

Proposed method

• Provide a suite of 8 model sizes (70M to 12B params) trained on the same data order with public checkpoints (154 per model).
• Train two copies of the suite on the Pile and a deduplicated Pile to study data effects.
• Use dense parallel attention and rotary embeddings with untied embedding matrices for interpretability and efficiency.
• Employ large batch sizes (1024) with GPT-NeoX framework, ZeRO, data/tensor parallelism, and Flash Attention for scalability.
• Evaluate with Language Model Evaluation Harness on eight benchmarks to compare to OPT/BLOOM baselines.
• Release all models, checkpoints, and evaluation code under Apache 2.0 for full reproducibility.

Figure 1 : The CrowS-Pairs gender bias, shown as the percentage of times that the perplexity of the stereotyping sentence is lower than its less stereotyped counterpart (% Stereotype) for the Pythia models of different sizes at the end of training. We also show the effect of the gender swapping inte

Experimental results

Research questions

• RQ1 How does training data order and deduplication affect model performance and memorization across scales?
• RQ2 What is the influence of pretraining term frequencies on task performance during training?
• RQ3 Does the architecture choice of parallel attention plus MLP layers impact small vs. large model performance?
• RQ4 How does gender-bias intervention via pronoun frequency modification affect downstream bias measures across model sizes?

Key findings

• Deduplication shows no clear performance benefit on language modeling in Pythia models.
• Equi-token and equi-parameter performance is achieved with parallel attention + MLP across scales, contrary to some prior claims.
• There is a minimal and inconsistent curse of multilinguality across BLOOM depending on benchmark, suggesting reevaluation with diverse tasks.
• A Poisson point process models memorization timing well, indicating training order has limited impact on memorized sequences.
• A significant phase change occurs around 65,000 training steps (45% through training) where larger models (2.8B+) begin to show correlation between task accuracy and pretraining term frequencies.
• Pronoun-frequency interventions in the last 7% or 21% of training reduce gender bias on targeted benchmarks without majorly harming perplexity on baseline tasks.

Figure 2 : The WinoBias gender bias results, shown as the proportion of the time that the model placed a higher log probability on the more stereotyped pronoun as an answer to a multiple choice gender–occupation co-reference question.