[Paper Review] The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning

Seungone Kim, Se June Joo+5 more|arXiv (Cornell University)|May 23, 2023

Topic Modeling4 citations

TL;DR

This paper introduces the CoT Collection, a large-scale instruction-tuning dataset with 1.84 million chain-of-thought (CoT) rationales across 1,060 tasks, to enhance zero-shot and few-shot reasoning in smaller language models. Fine-tuning Flan-T5 (3B and 11B) on this dataset yields significant gains—+4.34% and +2.60% average accuracy on BIG-Bench-Hard—while outperforming even GPT-3.5 and Claude in few-shot settings with a 13.98% margin.

ABSTRACT

Language models (LMs) with less than 100B parameters are known to perform poorly on chain-of-thought (CoT) reasoning in contrast to large LMs when solving unseen tasks. In this work, we aim to equip smaller LMs with the step-by-step reasoning capability by instruction tuning with CoT rationales. In order to achieve this goal, we first introduce a new instruction-tuning dataset called the CoT Collection, which augments the existing Flan Collection (including only 9 CoT tasks) with additional 1.84 million rationales across 1,060 tasks. We show that CoT fine-tuning Flan-T5 (3B & 11B) with CoT Collection enables smaller LMs to have better CoT capabilities on unseen tasks. On the BIG-Bench-Hard (BBH) benchmark, we report an average improvement of +4.34% (Flan-T5 3B) and +2.60% (Flan-T5 11B), in terms of zero-shot task accuracy. Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2.24% (Flan-T5 3B) and +2.37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13.98% margin. Our code, the CoT Collection data, and model checkpoints are publicly available.

Motivation & Objective

• To address the poor zero-shot and few-shot reasoning performance of smaller language models (under 100B parameters) on unseen tasks.
• To overcome the limitations of existing CoT instruction tuning datasets, which are small and imbalanced, leading to weak generalization in smaller models.
• To develop a comprehensive, scalable instruction-tuning dataset that enables smaller LMs to generalize CoT reasoning across diverse, unseen tasks.
• To demonstrate that CoT fine-tuning with a large, diverse dataset significantly improves both zero-shot and few-shot performance on downstream benchmarks.

Proposed method

• The CoT Collection is constructed by augmenting the Flan Collection with 1.84 million CoT rationales across 1,060 tasks, significantly expanding the original 9 CoT tasks.
• The dataset is used to fine-tune Flan-T5 models (3B and 11B), resulting in a new model called CoT-T5, which is optimized for step-by-step reasoning.
• The method applies instruction tuning with CoT demonstrations, enabling the model to generate intermediate reasoning steps before final predictions during inference.
• Ablation studies evaluate performance using a subset of 163 tasks (T0 setting) and multilingual data (80K instances in 5 languages), assessing low-resource and cross-lingual generalization.
• Evaluation is conducted on zero-shot (BIG-Bench-Hard) and few-shot (domain-specific legal/medical tasks) benchmarks with CoT prompting.
• Multilingual adaptation is performed by fine-tuning mT0 (3B) on translated CoT data, measuring performance on MGSM across five languages.

Figure 1: An illustration of the overall task group and dataset source of where we obtained the instances to augment the rationales in CoT Collection . Compared to the 9 datasets that provide publicly available rationales (included within ‘Flan-T5 ExQA’, ‘Flan-T5 Arithmetic’, ‘Flan-T5 MCQA’, ‘Flan-T

Experimental results

Research questions

• RQ1 Can a large-scale, diverse CoT instruction-tuning dataset significantly improve the zero-shot reasoning performance of smaller language models?
• RQ2 Does CoT fine-tuning with a broad range of tasks generalize better than fine-tuning on a small number of CoT tasks?
• RQ3 Can CoT fine-tuning enhance few-shot learning capabilities on domain-specific tasks, outperforming large models with demonstrations?
• RQ4 To what extent does multilingual CoT fine-tuning improve zero-shot reasoning in low-resource languages?
• RQ5 How does CoT fine-tuning compare to in-context learning with large models like GPT-3.5 and Claude in few-shot settings?

Key findings

• CoT-T5 (3B) achieves a +4.34% average accuracy improvement over Flan-T5 (3B) on 27 BIG-Bench-Hard datasets under zero-shot CoT prompting.
• CoT-T5 (11B) improves zero-shot accuracy by +2.60% on the same benchmark compared to Flan-T5 (11B).
• On four domain-specific few-shot tasks (LEDGAR, Case Hold, MedNLI, PubMedQA), CoT-T5 (3B) and (11B) outperform Flan-T5 by +2.24% and +2.37% respectively.
• CoT-T5 (3B) surpasses GPT-3.5 and Claude by +13.98% and +8.11% respectively in few-shot learning, even when those models use maximal input length demonstrations.
• Fine-tuning T0 (3B) on a subset of 163 CoT tasks from the CoT Collection yields an +8.65% average accuracy gain on 11 P3 Evaluation benchmarks.
• Multilingual CoT fine-tuning of mT0 (3B) on 80K translated instances improves MGSM performance by 2x to 10x across French, Japanese, Korean, Russian, and Chinese.

Figure 2: MCQA Prompt used to augment rationales from P3 dataset. Through ICL, the large LM generates a rationale that is conditioned on the ground-truth label.