[Paper Review] FinGPT: Democratizing Internet-scale Data for Financial Large Language Models

Xiaoyang Liu, Guoxuan Wang+2 more|arXiv (Cornell University)|Jul 19, 2023

Stock Market Forecasting Methods22 citations

TL;DR

FinGPT presents an open-source, data-centric framework that automates real-time collection and curation of Internet-scale financial data from 34+ sources and enables lightweight fine-tuning of LLMs for finance using LoRA/QLoRA and market feedback.

ABSTRACT

Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating human-like texts, which may potentially revolutionize the finance industry. However, existing LLMs often fall short in the financial field, which is mainly attributed to the disparities between general text data and financial text data. Unfortunately, there is only a limited number of financial text datasets available, and BloombergGPT, the first financial LLM (FinLLM), is close-sourced (only the training logs were released). In light of this, we aim to democratize Internet-scale financial data for LLMs, which is an open challenge due to diverse data sources, low signal-to-noise ratio, and high time-validity. To address the challenges, we introduce an open-sourced and data-centric framework, Financial Generative Pre-trained Transformer (FinGPT), that automates the collection and curation of real-time financial data from 34 diverse sources on the Internet, providing researchers and practitioners with accessible and transparent resources to develop their FinLLMs. Additionally, we propose a simple yet effective strategy for fine-tuning FinLLM using the inherent feedback from the market, dubbed Reinforcement Learning with Stock Prices (RLSP). We also adopt the Low-rank Adaptation (LoRA, QLoRA) method that enables users to customize their own FinLLMs from general-purpose LLMs at a low cost. Finally, we showcase several FinGPT applications, including robo-advisor, sentiment analysis for algorithmic trading, and low-code development. FinGPT aims to democratize FinLLMs, stimulate innovation, and unlock new opportunities in open finance. The codes have been open-sourced.

Motivation & Objective

• Demonstrate the feasibility of democratizing Internet-scale financial data for FinLLMs.
• Introduce a four-layer FinGPT framework focused on data sources, curation, LLMs, and applications.
• Showcase data-driven fine-tuning via lightweight methods and market-based reinforcement signals.
• Validate applications such as robo-advisors, sentiment analysis for trading, and low-code development.

Proposed method

• Four-layer FinGPT architecture (Data Source, Data Curation, LLM, Application).
• Automated real-time data collection from 34+ diverse sources including news, social media, filings, and datasets.
• Data cleaning and document filtering to improve quality and reduce noise.
• Low-rank adaptation (LoRA/QLoRA) for lightweight model fine-tuning.
• Reinforcement Learning with Stock Prices (RLSP) using market feedback as labels for sentiment.
• Low-cost adaptation enabling plug-and-play FinLLMs with privacy considerations.

Experimental results

Research questions

• RQ1 Can open, real-time, Internet-scale financial data be collected and curated to train FinLLMs?
• RQ2 Does data-centricFinGPT improve FinLLM performance on financial tasks compared with baselines?
• RQ3 Are lightweight fine-tuning methods (LoRA/QLoRA) and market-feedback signals sufficient for effective FinLLMs?
• RQ4 What are practical FinGPT applications and their performance in robo-advising, sentiment-driven trading, and low-code development?

Key findings

• The FinGPT pipeline integrates 34+ data sources to democratize access to financial data.
• RLSP enables market-informed labeling without human feedback, reducing labeling costs.
• FinGPT with LoRA/QLoRA achieves lower adaptation costs and maintains data privacy.
• Fine-tuning on FinGPT-curated data improves sentiment analysis and trading metrics over baselines.
• Table 1 shows FinGPT achieving higher accuracy and cumulative returns compared with LLaMA on news sentiment tasks.
• Fine-tuning with financial datasets (FPB, FiQA-SA, TFNS, NWGI) yields substantial F1 and accuracy gains over pretrained baselines.