[Paper Review] PEER: A Collaborative Language Model

Timo Schick, Jane Dwivedi-Yu+8 more|arXiv (Cornell University)|Aug 24, 2022

Topic Modeling44 citations

TL;DR

PEER trains a collaborative language model that plans, edits, explains, and repeats to simulate human writing, using Wikipedia edit histories and synthetic infilling to generalize to new domains.

ABSTRACT

Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process, today's language models are trained to generate only the final result. As a consequence, they lack several abilities crucial for collaborative writing: They are unable to update existing texts, difficult to control and incapable of verbally planning or explaining their actions. To address these shortcomings, we introduce PEER, a collaborative language model that is trained to imitate the entire writing process itself: PEER can write drafts, add suggestions, propose edits and provide explanations for its actions. Crucially, we train multiple instances of PEER able to infill various parts of the writing process, enabling the use of self-training techniques for increasing the quality, amount and diversity of training data. This unlocks PEER's full potential by making it applicable in domains for which no edit histories are available and improving its ability to follow instructions, to write useful comments, and to explain its actions. We show that PEER achieves strong performance across various domains and editing tasks.

Motivation & Objective

• Motivate the need for collaborative writing capabilities beyond single-pass generation.
• Propose a framework (Plan, Edit, Explain, Repeat) to model iterative text editing.
• Leverage Wikipedia edit histories and synthetic infilling to train PEER for domain-general editing.
• Enable self-training by infilling missing components of the editing process to boost data diversity and quality.
• Release models, data, and code to support research on collaborative language models.

Proposed method

• Model PEER as an iterative process that alternates between planning an edit, applying the edit, and explaining the change.
• Augment inputs with retrieved background documents to ground edits in evidence.
• Train multiple PEER variants to infer missing parts of the editing process (PEER-Edit, PEER-Undo, PEER-Explain, PEER-Document).
• Use synthetic data generation (infill) to extend training data beyond Wikipedia edits via PEER-Undo and related variants.
• Apply control tokens to steer outputs (plan, explanation length, termination behavior, document inclusion) during training and inference.
• Preprocess Wikipedia edit histories to build x_t to x_{t+1} tuples with plans p_t, edits, explanations e_t and associated documents D_t.

Experimental results

Research questions

• RQ1 Can PEER follow plans and perform edits in domains with no edit histories, aided by self-training with PEER-Undo?
• RQ2 Does following Wikipedia-based plans transfer to human-written instructions, and can synthetic plans from PEER-Explain improve this?
• RQ3 Can PEER effectively use citations and quotes to justify edits, and does PEER-Document help?
• RQ4 How does iterative PEER editing compare to single-pass generation for autonomous and plan-guided use?
• RQ5 What is the impact of domain adaptation and synthetic data on PEER’s performance across tasks?

Key findings

• PEER outperforms baselines on natural edits across Wikipedia and non-Wikipedia domains.
• Domain-adapted variants using synthetic edits (PEER-Undo) improve cross-domain performance, with notable gains on Gardening, Politics, and Movies subsets.
• Plans and documents provide complementary information; removing both substantially degrades performance.
• Following human-written plans transfers to downstream editing tasks, and synthetic plans from PEER-Explain further enhance results.
• PEER variants that utilize citations and generated documents show strong explanatory capabilities and improved update quality.
• Iterative PEER editing yields competitive results relative to single-pass generation, especially when guided by plans.