[Paper Review] Ploutos: Towards interpretable stock movement prediction with financial large language model

Hanshuang Tong, Jun Li+4 more|arXiv (Cornell University)|Feb 18, 2024

Stock Market Forecasting Methods6 citations

TL;DR

Ploutos introduces a multimodal financial LLM framework (PloutosGen and PloutosGPT) that fuses text and numerical data with adaptive expert weighting to predict stock movement and generate faithful, informative rationales.

ABSTRACT

Recent advancements in large language models (LLMs) have opened new pathways for many domains. However, the full potential of LLMs in financial investments remains largely untapped. There are two main challenges for typical deep learning-based methods for quantitative finance. First, they struggle to fuse textual and numerical information flexibly for stock movement prediction. Second, traditional methods lack clarity and interpretability, which impedes their application in scenarios where the justification for predictions is essential. To solve the above challenges, we propose Ploutos, a novel financial LLM framework that consists of PloutosGen and PloutosGPT. The PloutosGen contains multiple primary experts that can analyze different modal data, such as text and numbers, and provide quantitative strategies from different perspectives. Then PloutosGPT combines their insights and predictions and generates interpretable rationales. To generate accurate and faithful rationales, the training strategy of PloutosGPT leverage rearview-mirror prompting mechanism to guide GPT-4 to generate rationales, and a dynamic token weighting mechanism to finetune LLM by increasing key tokens weight. Extensive experiments show our framework outperforms the state-of-the-art methods on both prediction accuracy and interpretability.

Motivation & Objective

• Motivate interpretable stock movement prediction by fusing textual and numerical market data.
• Develop a modular expert pool (sentiment, technical, human) to provide diverse signals.
• Everyday data is leveraged through a finetuned LLM to generate transparent rationales for decisions.

Proposed method

• Propose PloutosGen as a pipeline of diverse experts analyzing multimodal stock data.
• Introduce PloutosGPT with rearview-mirror prompting to harvest faithful rationales from past cases.
• Apply dynamic token weighting to finetune the LLM by emphasizing key rationale tokens.
• Define faithfulness and informativeness metrics to evaluate rationale quality.

Experimental results

Research questions

• RQ1 RQ1: How does Ploutos perform relative to current LLM-based and traditional prediction models?
• RQ2 RQ2: How do the different components of Ploutos affect predictive effectiveness?
• RQ3 RQ3: Are the generated decision rationales faithful and informative?

Key findings

• Ploutos outperforms state-of-the-art traditional and LLM-based methods on ACL18 and CIKM18 datasets.
• Ablation shows each component (sentiment, technical, rearview-mirror prompting, dynamic token weighting) contributes to performance.
• Ploutos achieves higher accuracy and MCC than baselines across both datasets (ACL18 and CIKM18).
• Ploutos exhibits superior faithfulness and informativeness in its rationales compared to other models.
• Dynamic token weighting with an optimal temperature (~0.5) improves both accuracy and rationale quality.
• FinMA-7B struggles to produce informative rationales, highlighting the benefit of Ploutos’ training strategy.