[Paper Review] How is ChatGPT's behavior changing over time?

Lingjiao Chen, Matei Zaharia+1 more|arXiv (Cornell University)|Jul 18, 2023

Artificial Intelligence in Healthcare and Education167 citations

TL;DR

This study analyzes March 2023 vs. June 2023 updates of GPT-3.5 and GPT-4 across multiple tasks, showing substantial performance and instruction-following drift over time and highlighting the need for continuous monitoring of LLM services.

ABSTRACT

GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on several diverse tasks: 1) math problems, 2) sensitive/dangerous questions, 3) opinion surveys, 4) multi-hop knowledge-intensive questions, 5) generating code, 6) US Medical License tests, and 7) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was reasonable at identifying prime vs. composite numbers (84% accuracy) but GPT-4 (June 2023) was poor on these same questions (51% accuracy). This is partly explained by a drop in GPT-4's amenity to follow chain-of-thought prompting. Interestingly, GPT-3.5 was much better in June than in March in this task. GPT-4 became less willing to answer sensitive questions and opinion survey questions in June than in March. GPT-4 performed better at multi-hop questions in June than in March, while GPT-3.5's performance dropped on this task. Both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. We provide evidence that GPT-4's ability to follow user instructions has decreased over time, which is one common factor behind the many behavior drifts. Overall, our findings show that the behavior of the "same" LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLMs.

Motivation & Objective

• Motivated by opacity in model updates, assess how two major LLM services (GPT-3.5 and GPT-4) change over time.
• Evaluate performance and behavior drift across diverse tasks to understand potential trade-offs and stability.
• Provide evidence and resources to motivate continuous monitoring and robust integration of LLMs in workflows.

Proposed method

• Compare March 2023 vs. June 2023 versions of GPT-3.5 and GPT-4 using prompts with default system prompts and a low temperature (0.1).
• Evaluate eight tasks: math problems (prime vs. composite, happy numbers), sensitive/dangerous questions, OpinionQA surveys, LangChain HotpotQA multi-hop QA, code generation, USMLE medical exams, and visual reasoning.
• Use task-specific metrics (accuracy, response rate, exact match, executability) and two common metrics (verbosity and mismatch) to quantify drift.
• Analyze dependence on chain-of-thought prompting by testing with and without CoT instructions.
• Curate and release prompts and responses to enable reproducibility; open-source analysis code.

Experimental results

Research questions

• RQ1 Do GPT-4 and GPT-3.5 exhibit measurable performance drift between March 2023 and June 2023 across diverse tasks?
• RQ2 How does the ability to follow user instructions (e.g., chain-of-thought prompts) drift over time, and how does this relate to task performance?
• RQ3 What patterns emerge in safety-related and opinion-generation behaviors as models drift over time?
• RQ4 To what extent do code formatting, multi-hop reasoning, and prompting stability contribute to downstream pipeline brittleness over time?
• RQ5 What implications do these drifts have for deploying LLMs in real-world systems and workflows?

Key findings

• Both GPT-4 and GPT-3.5 show substantial performance and behavior drift between March and June 2023 across several tasks.
• GPT-4's prime/composite accuracy dropped from 84% to 51%, while GPT-3.5 improved from 49.6% to 76.2% on the prime test, with changes in chain-of-thought usage.
• GPT-4 became less willing to answer sensitive and opinion survey questions in June than in March, while GPT-3.5 showed mixed changes.
• GPT-4 improved on some multi-hop questions in June, whereas GPT-3.5 declined on this task.
• Code generation outputs became less directly executable in June for both models due to formatting changes, with a notable rise in non-code text.
• GPT-4's ability to follow user instructions generally decreased over time, contributing to broader behavior drift.
• There are notable drifts in safety defense against jailbreaking attacks and in producing verbose rationales for refusals, with GPT-4 showing stronger safety shifts than GPT-3.5.
• Across tasks, substantial changes in output length and content (verbosity, mismatch) accompany accuracy drift, signaling instability in model behavior.