[Paper Review] Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning

Yinpeng Liu, Jiawei Liu+4 more|arXiv (Cornell University)|Feb 16, 2024

Online and Blended Learning4 citations

TL;DR

This paper proposes In-Context Curriculum Learning (ICCL), a method that enhances few-shot in-context learning in open-source LLMs by ordering demonstration examples from simple to complex, improving model performance during inference. The approach, which relies on instruction tuning to develop curriculum-aware capabilities, shows significant gains on scientific reasoning tasks—especially for smaller models—while human-designed ordering outperforms LLM-generated ordering.

ABSTRACT

Demonstration ordering, which is an important strategy for in-context learning (ICL), can significantly affects the performance of large language models (LLMs). However, most of the current approaches of ordering require high computational costs to introduce the priori knowledge. In this paper, inspired by the human learning process, we propose a simple but effective demonstration ordering method for ICL, named the few-shot In-Context Curriculum Learning (ICCL). The ICCL implies gradually increasing the complexity of prompt demonstrations during the inference process. The difficulty can be assessed by human experts or LLMs-driven metrics, such as perplexity. Then we design extensive experiments to discuss the effectiveness of the ICCL at both corpus-level and instance-level. Moreover, we also investigate the formation mechanism of LLM's ICCL capability. Experimental results demonstrate that ICCL, developed during the instruction-tuning stage, is effective for representative open-source LLMs. To facilitate further research and applications by other scholars, we make the code publicly available.

Motivation & Objective

• To investigate whether organizing in-context demonstrations by increasing difficulty improves few-shot in-context learning performance.
• To determine when and how LLMs acquire the ability to benefit from curriculum-based demonstration ordering.
• To evaluate the effectiveness of human-designed versus LLM-generated curriculum schedules in in-context learning.
• To explore the feasibility of applying curriculum learning directly within the inference phase without retraining.

Proposed method

• Propose In-Context Curriculum Learning (ICCL), a demonstration ordering strategy that arranges examples from low to high complexity to improve in-context learning.
• Use human experts or LLMs as 'sorters' to rank demonstrations based on perceived difficulty, forming a curriculum schedule.
• Apply the ordered demonstrations as context in inference prompts, maintaining the same model parameters and only changing the demonstration order.
• Conduct experiments on base and instruction-tuned LLMs across scientific reasoning datasets (SciCite, SciNLI) to compare ICCL vs. baseline in-context learning (ICL).
• Design prompts to evaluate whether LLMs can autonomously generate effective curriculum orders, comparing their performance to human-curated orders.
• Measure performance using F1 scores and analyze improvement or decline relative to random ordering and standard ICL.

Experimental results

Research questions

• RQ1 Does ordering in-context demonstrations by increasing difficulty (ICCL) improve few-shot in-context learning performance in LLMs?
• RQ2 At what stage—pre-training or instruction-tuning—does the model acquire the ability to benefit from curriculum-based demonstration ordering?
• RQ3 How does the performance of human-designed curriculum schedules compare to LLM-generated curriculum schedules in ICCL?
• RQ4 Why does ICCL show limited or negative effects on high-performing models like GPT-4?

Key findings

• ICCL significantly improves performance on instruction-tuned open-source LLMs, with an average F1 score improvement of 5.62% on SciNLI for Mixtral 8x7B.
• Instruction-tuned Llama 2 70B shows a 5.62% improvement in F1 when using ICCL, while the base model shows a 4.12% decrease, indicating that ICCL capability emerges during instruction tuning.
• Human-designed curriculum ordering outperforms LLM-generated ordering, with the latter showing only a 0.29% average performance decline compared to random ordering.
• ICCL is ineffective for GPT-4, which shows a 2.56% average F1 score decrease when using ICCL, suggesting high-performing models may be less sensitive to demonstration ordering.
• The base models show unstable performance with ICCL, with an average 7.16% drop in F1 compared to standard ICL, indicating no inherent sensitivity to curriculum ordering without instruction tuning.
• The study confirms that human experts play a critical role in ICCL by providing a knowledge-rich, effective curriculum that maximizes parameter utilization in the model.