[Paper Review] Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer

Bowen Tan, Yun Zhu+4 more|arXiv (Cornell University)|Nov 12, 2023

Topic Modeling4 citations

TL;DR

Cappy is a 360M-parameter pretrained scorer that enhances large multi-task language models (LLMs) by scoring candidate responses to instructions, enabling performance boosts without fine-tuning. It outperforms much larger LLMs on 11 classification tasks and significantly improves FLAN-T5's performance on 45 complex BIG-Bench tasks through efficient, parameter-efficient response selection.

ABSTRACT

Large language models (LLMs) such as T0, FLAN, and OPT-IML, excel in multi-tasking under a unified instruction-following paradigm, where they also exhibit remarkable generalization abilities to unseen tasks. Despite their impressive performance, these LLMs, with sizes ranging from several billion to hundreds of billions of parameters, demand substantial computational resources, making their training and inference expensive and inefficient. Furthermore, adapting these models to downstream applications, particularly complex tasks, is often unfeasible due to the extensive hardware requirements for finetuning, even when utilizing parameter-efficient approaches such as prompt tuning. Additionally, the most powerful multi-task LLMs, such as OPT-IML-175B and FLAN-PaLM-540B, are not publicly accessible, severely limiting their customization potential. To address these challenges, we introduce a pretrained small scorer, Cappy, designed to enhance the performance and efficiency of multi-task LLMs. With merely 360 million parameters, Cappy functions either independently on classification tasks or serve as an auxiliary component for LLMs, boosting their performance. Moreover, Cappy enables efficiently integrating downstream supervision without requiring LLM finetuning nor the access to their parameters. Our experiments demonstrate that, when working independently on 11 language understanding tasks from PromptSource, Cappy outperforms LLMs that are several orders of magnitude larger. Besides, on 45 complex tasks from BIG-Bench, Cappy boosts the performance of the advanced multi-task LLM, FLAN-T5, by a large margin. Furthermore, Cappy is flexible to cooperate with other LLM adaptations, including finetuning and in-context learning, offering additional performance enhancement.

Motivation & Objective

• To address the high computational cost and inefficiency of fine-tuning large multi-task LLMs for downstream tasks.
• To enable performance enhancement of powerful but inaccessible LLMs (e.g., OPT-IML-175B, FLAN-PaLM-540B) without requiring access to their parameters.
• To develop a lightweight, efficient, and flexible component that can be used independently or in collaboration with LLMs for improved task generalization.
• To create a weakly-supervised pretraining data construction method using LLM-generated correctness scores for regression-style training of the scorer.

Proposed method

• Cappy is a 360M-parameter RoBERTa-based model trained to predict a correctness score (0–1) for a given (instruction, response) pair.
• It uses a weakly-supervised data construction pipeline: existing multi-task LLMs generate responses, and Rouge-L scores are used as weak supervision to create regression labels.
• The pretraining data is constructed via data augmentation using existing LLMs, generating diverse (instruction, response, score) triplets from pretraining datasets.
• Cappy operates in a candidate selection paradigm: for a given instruction, it scores multiple candidate responses and selects the one with the highest score.
• It can be used independently for classification tasks or as an auxiliary component to boost LLMs without fine-tuning.
• It is compatible with other adaptation techniques such as in-context learning and fine-tuning, enabling further performance gains.

Experimental results

Research questions

• RQ1 Can a small, lightweight scorer outperform much larger multi-task LLMs on zero-shot classification tasks?
• RQ2 Can a small scorer effectively boost the performance of a large multi-task LLM on complex, unseen tasks without parameter updates?
• RQ3 How effective is weak supervision via LLM-generated responses and Rouge-L scores in training a high-performing correctness scorer?
• RQ4 Can the scorer be flexibly integrated with other LLM adaptation strategies like in-context learning or fine-tuning?

Key findings

• Cappy outperforms larger multi-task LLMs on 11 language understanding tasks from PromptSource, achieving higher accuracy despite being orders of magnitude smaller.
• On 45 complex tasks from BIG-Bench, Cappy boosts FLAN-T5’s performance by a large margin without requiring any fine-tuning of the LLM.
• Ablation studies show that both continual pretraining and LLM-based data augmentation are critical for Cappy’s performance, with data augmentation having a stronger impact.
• Cappy enhances performance when combined with other adaptation methods, such as in-context learning and fine-tuning, demonstrating its compatibility and extensibility.
• The use of Rouge-L as a weak supervision metric is effective, though not perfect, and future work may explore better correctness metrics for multi-task settings.