[Paper Review] FinGPT: Open-Source Financial Large Language Models

Hongyang Yang, Xiaoyang Liu+1 more|arXiv (Cornell University)|Jun 9, 2023

FinTech, Crowdfunding, Digital FinanceBusiness, Management and Accounting23 citations

TL;DR

FinGPT presents an open-source, data-centric framework for financial LLMs, enabling lightweight fine-tuning (LoRA/RLSP) on real-time data to democratize FinLLMs and enable finance-focused applications.

ABSTRACT

Large language models (LLMs) have shown the potential of revolutionizing natural language processing tasks in diverse domains, sparking great interest in finance. Accessing high-quality financial data is the first challenge for financial LLMs (FinLLMs). While proprietary models like BloombergGPT have taken advantage of their unique data accumulation, such privileged access calls for an open-source alternative to democratize Internet-scale financial data. In this paper, we present an open-source large language model, FinGPT, for the finance sector. Unlike proprietary models, FinGPT takes a data-centric approach, providing researchers and practitioners with accessible and transparent resources to develop their FinLLMs. We highlight the importance of an automatic data curation pipeline and the lightweight low-rank adaptation technique in building FinGPT. Furthermore, we showcase several potential applications as stepping stones for users, such as robo-advising, algorithmic trading, and low-code development. Through collaborative efforts within the open-source AI4Finance community, FinGPT aims to stimulate innovation, democratize FinLLMs, and unlock new opportunities in open finance. Two associated code repos are https://github.com/AI4Finance-Foundation/FinGPT and https://github.com/AI4Finance-Foundation/FinNLP

Motivation & Objective

• Democratize financial data and FinLLMs through an open-source framework.
• Highlight the importance of data acquisition, cleaning, and preprocessing for FinLLMs.
• Propose an end-to-end four-layer architecture (Data Source, Data Engineering, LLMs, Applications) tailored to finance.
• Showcase practical applications and demos to illustrate FinGPT’s potential in finance.

Proposed method

• Adopt a data-centric approach prioritizing high-quality data curation and preprocessing.
• Build an end-to-end four-layer framework: Data Source, Data Engineering, LLMs, Applications.
• Use lightweight fine-tuning of open-source LLMs via LoRA to reduce trainable parameters (from 6.17B to 3.67M).
• Offer alternative RL-based fine-tuning by Reinforcement Learning on Stock Prices (RLSP) using stock movements as feedback.
• Leverage RLHF-like methods and prompt engineering to align model outputs with financial tasks.

Experimental results

Research questions

• RQ1 How can an open-source, data-centric pipeline support effective FinLLMs in finance?
• RQ2 What are the trade-offs in using LoRA versus RLSP for fine-tuning financial LLMs?
• RQ3 Can real-time data engineering overcome high temporal sensitivity and low SNR in financial data?
• RQ4 What practical finance applications can FinGPT enable and demonstrate the value of open FinLLMs?

Key findings

• FinGPT demonstrates a cost-effective fine-tuning approach: adapting open-source LLMs at under $300 per training versus BloombergGPT’s higher cost.
• The framework emphasizes real-time data ingestion, cleaning, and NLP processing to address temporal sensitivity and noise in financial data.
• LoRA-based fine-tuning reduces trainable parameters from 6.17B to 3.67M while leveraging domain-relevant financial signals.
• RLSP provides a market-driven feedback loop by using stock price changes as objective signals for model alignment with market responses.
• The open-source ecosystem and tutorials enable applications such as robo-advisors, quantitative trading, risk management, and financial education.