[Paper Review] Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models

Boyu Zhang, Hongyang Yang+1 more|arXiv (Cornell University)|Jun 22, 2023

Stock Market Forecasting Methods11 citations

TL;DR

This paper tunes a general-purpose LLM (LLaMA-7B) via instruction tuning on a small financial sentiment dataset to outperform FinBERT and ChatGPT in financial sentiment analysis, with emphasis on numerical sensitivity and contextual understanding.

ABSTRACT

Sentiment analysis is a vital tool for uncovering insights from financial articles, news, and social media, shaping our understanding of market movements. Despite the impressive capabilities of large language models (LLMs) in financial natural language processing (NLP), they still struggle with accurately interpreting numerical values and grasping financial context, limiting their effectiveness in predicting financial sentiment. In this paper, we introduce a simple yet effective instruction tuning approach to address these issues. By transforming a small portion of supervised financial sentiment analysis data into instruction data and fine-tuning a general-purpose LLM with this method, we achieve remarkable advancements in financial sentiment analysis. In the experiment, our approach outperforms state-of-the-art supervised sentiment analysis models, as well as widely used LLMs like ChatGPT and LLaMAs, particularly in scenarios where numerical understanding and contextual comprehension are vital.

Motivation & Objective

• Demonstrate that instruction tuning of a general-purpose LLM can improve financial sentiment analysis.
• Address numerical sensitivity in financial texts to better interpret sentiment from numbers.
• Assess the role of contextual understanding enhanced by the LLM's prior knowledge.
• Compare instruction-tuned LLaMA-7B against FinBERT and ChatGPT on financial sentiment tasks.

Proposed method

• Transform the sentiment classification dataset into an instruction-tuning format with 10 human-written instructions.
• Fine-tune LLaMA-7B on the formatted instruction data using supervised sequence-to-sequence loss.
• Map the autoregressive outputs to three sentiment labels (positive, negative, neutral).
• Evaluate the model against FinBERT and LLaMA-7B to assess context and numerical sensitivity.
• Use 8 A100 GPUs with DeepSpeed for training over 10 epochs with specified hyperparameters.

Experimental results

Research questions

• RQ1 How can numerical sensitivity be improved in financial sentiment analysis using instruction-tuned LLMs?
• RQ2 What is the impact of contextual understanding, gained from general LLM knowledge, on financial sentiment prediction?
• RQ3 How does instruction-tuned FinGPT compare to traditional FinBERT and generic LLMs in financial sentiment tasks?
• RQ4 Can a small amount of instruction data yield state-of-the-art performance with a general-purpose LLM?

Key findings

• Instruct-FinGPT-7B outperforms FinBERT and LLaMA-7B on all evaluated datasets in accuracy and F1.
• The model shows strong numerical sensitivity, correctly interpreting sentiment related to financial numbers in several examples.
• Contextual understanding from an instruction-tuned LLM leads to better sentiment interpretation when context is missing or vague.
• Zero-shot FPB evaluation favors Instruct-FinGPT-7B over ChatGPT-3.5 and LLaMA-7B, indicating good generalization.
• Training required is modest (≈58 minutes on 8 A100 GPUs) with a small amount of instruction data.
• The approach achieves superior performance with substantially lower training resources compared to BloombergGPT.