[論文レビュー] Ensemble Methodology:Innovations in Credit Default Prediction Using LightGBM, XGBoost, and LocalEnsemble

Mengran Zhu, Ye Zhang+4 more|arXiv (Cornell University)|Feb 28, 2024

Financial Distress and Bankruptcy Prediction被引用数 20

ひとこと要約

本論文は LightGBM、XGBoost、LocalEnsemble を組み合わせた Ensemble Methods フレームワークを提案し、クレジットデフォルト予測の精度を向上させる。American Express データセットで検証。エnsemble は公開・非公開評価のいずれも個別モデルを上回る。

ABSTRACT

In the realm of consumer lending, accurate credit default prediction stands as a critical element in risk mitigation and lending decision optimization. Extensive research has sought continuous improvement in existing models to enhance customer experiences and ensure the sound economic functioning of lending institutions. This study responds to the evolving landscape of credit default prediction, challenging conventional models and introducing innovative approaches. By building upon foundational research and recent innovations, our work aims to redefine the standards of accuracy in credit default prediction, setting a new benchmark for the industry. To overcome these challenges, we present an Ensemble Methods framework comprising LightGBM, XGBoost, and LocalEnsemble modules, each making unique contributions to amplify diversity and improve generalization. By utilizing distinct feature sets, our methodology directly tackles limitations identified in previous studies, with the overarching goal of establishing a novel standard for credit default prediction accuracy. Our experimental findings validate the effectiveness of the ensemble model on the dataset, signifying substantial contributions to the field. This innovative approach not only addresses existing obstacles but also sets a precedent for advancing the accuracy and robustness of credit default prediction models.

研究の動機と目的

• クレジットデフォルト予測における消費者ローン提供時の課題に対処する。
• 一般化と頑健性を向上させる多様なアンサンブルフレームワークを開発する。
• 異なる特徴セットとローカルアンサンブル技術を活用して精度を高める。
• 大規模で匿名化された American Express データセット で有効性を示す。

提案手法

• ノイズを除去し、型を変換し、外れ値を処理してデータを前処理する。
• 集計、ラグ特徴量、メタ特徴量を含む特徴量エンジニアリング。
• 異なる特徴セットで3つのモジュール（LightGBM、XGBoost、LocalEnsemble）を訓練し、多様性を促進。
• 初期モデルからのアウトオブフォールド予測をメタ特徴として組み込む。
• モジュール予測を重み付きアンサンブルで結合する（y_hat_e = sum w_i * y_hat_i）。
• 正規化ジニ係数と4%デフォルト率を組み合わせた複合指標で評価する。

実験結果

リサーチクエスチョン

• RQ1 Can an ensemble of LightGBM, XGBoost, and LocalEnsemble achieve higher credit default prediction accuracy than individual models?
• RQ2 Do diverse feature sets and a LocalEnsemble component improve generalization and robustness on large time-series credit data?
• RQ3 How does the proposed Ensemble Model perform on public and private subsets of the American Express dataset compared to neural networks and other boosting models?

主な発見

モデル	公開スコア (49%)	非公開スコア (51%)
GRU	0.78877	0.79832
Transformer	0.78916	0.79832
Tabtransformer	0.78271	0.79236
Neural Networks	0.78705	0.79698
XGBoost	0.79982	0.80757
LightGBM	0.80006	0.80809
CatBoost (Local)	0.79804	0.80629
LightGBM (Local)	0.79967	0.80697
Local Ensemble	0.80094	0.80842
Ensemble Model	0.80128	0.80872

• The Ensemble Model achieves the highest scores on both public (0.80128) and private (0.80872) datasets.
• The LocalEnsemble and LightGBM+XGBoost components contribute to performance gains through feature diversity.
• XGBoost and LightGBM provide strong baseline performance, with LocalEnsemble further enhancing generalization.
• Feature importance analysis shows top features explain over 90% of predictive power across XGBoost and LightGBM.
• The proposed fusion of three modules outperforms several deep learning and traditional models in the study.