[Paper Review] Financial News Analytics Using Fine-Tuned Llama 2 GPT Model

Bohdan M. Pavlyshenko|arXiv (Cornell University)|Aug 24, 2023

Stock Market Forecasting Methods17 citations

TL;DR

The paper fine-tunes Llama 2 using PEFT/LoRA for multitask financial news analysis, producing structured responses and JSON outputs for downstream predictive use.

ABSTRACT

The paper considers the possibility to fine-tune Llama 2 GPT large language model (LLM) for the multitask analysis of financial news. For fine-tuning, the PEFT/LoRA based approach was used. In the study, the model was fine-tuned for the following tasks: analysing a text from financial market perspectives, highlighting main points of a text, summarizing a text and extracting named entities with appropriate sentiments. The obtained results show that the fine-tuned Llama 2 model can perform a multitask financial news analysis with a specified structure of response, part of response can be a structured text and another part of data can have JSON format for further processing. Extracted sentiments for named entities can be considered as predictive features in supervised machine learning models with quantitative target variables.

Motivation & Objective

• Motivate multitask financial news analysis using open-source LLMs.
• Demonstrate PEFT/LoRA-based fine-tuning on a financial news dataset.
• Show that fine-tuned Llama 2 can output structured responses and JSON for downstream tasks.
• Explore how extracted named-entity sentiments can serve as predictive features.

Proposed method

• Fine-tune Llama 2-7b-chat-hf with PEFT/LoRA on a financial news dataset from Kaggle.
• Use 4-bit or 8-bit quantization to optimize GPU usage during fine-tuning.
• Train with SFTTrainer and specified training arguments (e.g., learning_rate 5e-4, 10 epochs, max_seq_length 2048).
• Provide prompts that guide the model to analyze text from market perspectives, highlight main points, summarize, and extract named entities with sentiments.
• Evaluate loss curves on training and validation sets to compare 4-bit vs 8-bit quantization.
• Assess outputs that combine structured text and JSON for downstream processing.

Figure 1: Loss curves for training ( loss ) and validation ( eval_loss ) datasets in case of 4bit model quantization.

Experimental results

Research questions

• RQ1 Can a fine-tuned Llama 2 model perform multitask financial news analysis with a structured response?
• RQ2 Do PEFT/LoRA fine-tuned models produce reliable named-entity sentiment data suitable as predictive features?
• RQ3 Is there a difference in performance between 4-bit and 8-bit quantization during fine-tuning?
• RQ4 Can the model generate outputs that interleave natural language and JSON data for easy downstream processing.

Key findings

• The fine-tuned Llama 2 model can perform multitask financial news analysis with a specified structure of response.
• Extracted sentiments for named entities can be considered predictive features for supervised models with quantitative targets.
• Loss curves for 4-bit and 8-bit quantization are very similar during fine-tuning.
• The approach enables outputs that mix structured text with JSON formatted data for further processing.
• Training dataset from Kaggle includes multiple finance news sources used for fine-tuning and evaluation.
• Inaccuracy in naming entities in JSON output can be addressed with a more precise training dataset and instructions.

Figure 2: Loss curves for training ( loss ) and validation ( eval_loss ) datasets in case of 8bit model quantization.