[Paper Review] Black-Box Tuning for Language-Model-as-a-Service

Tianxiang Sun, Yunfan Shao+3 more|arXiv (Cornell University)|Jan 10, 2022

Topic Modeling56 citations

TL;DR

Black-box tuning optimizes continuous prompts for LMaaS by using derivative-free optimization in a random subspace, achieving competitive or superior results to gradient-based methods while keeping PTMs as a service.

ABSTRACT

Extremely large pre-trained language models (PTMs) such as GPT-3 are usually released as a service. It allows users to design task-specific prompts to query the PTMs through some black-box APIs. In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable. Can we optimize the task prompts by only accessing the model inference APIs? This paper proposes the black-box tuning framework to optimize the continuous prompt prepended to the input text via derivative-free optimization. Instead of optimizing in the original high-dimensional prompt space, which is intractable for traditional derivative-free optimization, we perform optimization in a randomly generated subspace due to the low intrinsic dimensionality of large PTMs. The experimental results show that the black-box tuning with RoBERTa on a few labeled samples not only significantly outperforms manual prompt and GPT-3's in-context learning, but also surpasses the gradient-based counterparts, i.e., prompt tuning and full model tuning.

Motivation & Objective

• Motivate LMaaS as a scenario where gradients are unavailable and prompts must be optimized via API access only.
• Propose Black-Box Tuning (BBT) to optimize continuous prompts through derivative-free optimization in a randomly embedded subspace.
• Show that BBT can outperform manual prompts, in-context learning, and gradient-based prompt/full-model tuning under few-shot settings.
• Demonstrate deployment and efficiency advantages of BBT for LMaaS over gradient-based and feature-based baselines.

Proposed method

• Formulate tasks as classification with a continuous prompt p added to inputs and a black-box PTM API f that outputs logits.
• Reduce prompt space dimensionality by optimizing z in a low-dimensional subspace via a random projection Az + p0, with p0 initialized from random tokens.
• Optimize using CMA-ES, a derivative-free optimizer, instead of backpropagation.
• Use a cross-entropy or hinge loss (or negative accuracy) as the objective to guide optimization.
• Pre-train a prompt embedding p0 on MNLI for sentence-pair tasks or initialize with random embeddings for other tasks.
• Evaluate under a few-shot setting across multiple datasets using RoBERTa LARGE as backbone.

Experimental results

Research questions

• RQ1 Can continuous prompts be effectively optimized when only PTM inference APIs are accessible (no gradients)?
• RQ2 Does derivative-free optimization in a random low-dimensional subspace match or exceed gradient-based prompt tuning and full-model tuning on standard tasks in LMaaS settings?
• RQ3 How do loss functions (cross-entropy, hinge, negative accuracy) and subspace dimensionality affect performance in black-box prompt tuning?
• RQ4 What are the deployment, training-time, memory, and data-transfer implications of black-box tuning compared to gradient-based and feature-based baselines?
• RQ5 To what extent does pre-training prompt embeddings improve performance for sentence-pair tasks in the LMaaS framework?

Key findings

• BBT often outperforms manual prompts and in-context learning on the evaluated tasks.
• BBT surpasses gradient-based methods (prompt tuning, P-Tuning v2) and full model tuning in average performance across seven tasks in the 16-shot setting.
• Cross-entropy and hinge losses provide better guidance than negative accuracy in the few-shot regime.
• A smaller subspace (d in 100–1000) with CMA-ES can be effective, with task-dependent optimal d.
• Pre-trained prompt embeddings can improve NLI task performance when using prompt-based approaches in LMaaS.
• BBT offers deployment and efficiency advantages, with optimization cost decoupled from PTM size and viable server-side hosting.