[Paper Review] Ensemble Methodology:Innovations in Credit Default Prediction Using LightGBM, XGBoost, and LocalEnsemble

Mengran Zhu, Ye Zhang+4 more|arXiv (Cornell University)|Feb 28, 2024

Financial Distress and Bankruptcy Prediction20 citations

TL;DR

The paper proposes an Ensemble Methods framework combining LightGBM, XGBoost, and LocalEnsemble to boost credit default prediction accuracy, validated on the American Express dataset. The ensemble outperforms individual models in both public and private evaluations.

ABSTRACT

In the realm of consumer lending, accurate credit default prediction stands as a critical element in risk mitigation and lending decision optimization. Extensive research has sought continuous improvement in existing models to enhance customer experiences and ensure the sound economic functioning of lending institutions. This study responds to the evolving landscape of credit default prediction, challenging conventional models and introducing innovative approaches. By building upon foundational research and recent innovations, our work aims to redefine the standards of accuracy in credit default prediction, setting a new benchmark for the industry. To overcome these challenges, we present an Ensemble Methods framework comprising LightGBM, XGBoost, and LocalEnsemble modules, each making unique contributions to amplify diversity and improve generalization. By utilizing distinct feature sets, our methodology directly tackles limitations identified in previous studies, with the overarching goal of establishing a novel standard for credit default prediction accuracy. Our experimental findings validate the effectiveness of the ensemble model on the dataset, signifying substantial contributions to the field. This innovative approach not only addresses existing obstacles but also sets a precedent for advancing the accuracy and robustness of credit default prediction models.

Motivation & Objective

• Address challenges in credit default prediction within consumer lending.
• Develop a diverse ensemble framework to improve generalization and robustness.
• Leverage distinct feature sets and local ensemble techniques to enhance accuracy.
• Demonstrate efficacy on a large, anonymized American Express dataset.

Proposed method

• Preprocess data by removing noise, converting types, and handling outliers.
• Engineer features including aggregations, lag features, and meta features.
• Train three modules (LightGBM, XGBoost, LocalEnsemble) with different feature sets to promote diversity.
• Incorporate out-of-fold predictions from initial models as meta features.
• Combine module predictions using a weighted ensemble (y_hat_e = sum w_i * y_hat_i).
• Evaluate using a composite metric combining Normalized Gini Coefficient and 4% default rate.

Experimental results

Research questions

• RQ1 Can an ensemble of LightGBM, XGBoost, and LocalEnsemble achieve higher credit default prediction accuracy than individual models?
• RQ2 Do diverse feature sets and a LocalEnsemble component improve generalization and robustness on large time-series credit data?
• RQ3 How does the proposed Ensemble Model perform on public and private subsets of the American Express dataset compared to neural networks and other boosting models?

Key findings

• The Ensemble Model achieves the highest scores on both public (0.80128) and private (0.80872) datasets.
• The LocalEnsemble and LightGBM+XGBoost components contribute to performance gains through feature diversity.
• XGBoost and LightGBM provide strong baseline performance, with LocalEnsemble further enhancing generalization.
• Feature importance analysis shows top features explain over 90% of predictive power across XGBoost and LightGBM.
• The proposed fusion of three modules outperforms several deep learning and traditional models in the study.