[Paper Review] Toolformer: Language Models Can Teach Themselves to Use Tools

Timo Schick, Jane Dwivedi-Yu+6 more|arXiv (Cornell University)|Feb 9, 2023

Topic Modeling370 citations

TL;DR

Toolformer trains a language model to decide when and how to call external tools via APIs in a self-supervised loop, yielding strong zero-shot results across tasks while preserving core language modeling abilities.

ABSTRACT

Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.

Motivation & Objective

• Enable a language model to autonomously learn when to call which external tools via APIs.
• Improve zero-shot and few-shot task performance by augmenting data with tool calls without human annotations.
• Maintain the model's general language modeling capabilities while adding tool-use behavior.
• Demonstrate that tool-use can scale across diverse tasks and tools without task-specific supervision.

Proposed method

• Represent each API call as a text-embedded token sequence using <API> ... </API> annotations.
• Create C* by prompting the LM to annotate data with candidate API calls, execute them, and filter based on a self-supervised loss that measures future token prediction gain.
• Compute L_i^+ as the loss when an API call and result are included; L_i^- as the minimum loss without the API call or with only the input."
• Retain only API calls with sufficient loss reduction (τ_f) and interleave useful calls into the original text to form C*, then finetune the LM on C* with standard LM objective.
• During inference, generate normally until the model outputs →, fetch the API result, and continue decoding by inserting the API response.

Experimental results

Research questions

• RQ1 Can a language model learn to decide when to call external tools without human supervision?
• RQ2 Which tools (QA, search, calculator, translation, calendar) provide the most consistent gains across diverse tasks?
• RQ3 Does finetuning on tool-augmented data degrade or preserve the base language modeling abilities?
• RQ4 How does model size affect the ability to leverage tools in zero-shot settings?

Key findings

• Toolformer substantially improves zero-shot performance across diverse tasks compared to baselines of similar size.
• On LAMA subsets (SQuAD, Google-RE, T-REx), Toolformer outperforms GPT-J baselines and is competitive with GPT-3 (175B).
• On math benchmarks (ASDiv, SVAMP, MAWPS), Toolformer with calculator calls vastly outperforms baselines, often by large margins, with most examples using the calculator.
• On QA datasets (WebQuestions, Natural Questions, TriviaQA), Toolformer improves over GPT-J baselines and relies heavily on Wikipedia search for retrieval.
• Multilingual QA (MLQA) shows Toolformer benefits from machine translation usage, though gains vary by language; larger models still outperform Toolformer in some regimes.
• Temporal datasets (TempLAMA, Dateset) show Toolformer excels when a calendar tool is useful, with Dateset showing strong gains when the calendar is used.
• Perplexity on WikiText and CCNet subsets indicates that adding API calls does not increase LM perplexity when API usage is disabled at inference, preserving core LM abilities.