[Paper Review] Formulating A Strategic Plan Based On Statistical Analyses And Applications For Financial Companies Through A Real-World Use Case

Saman Sarraf|arXiv (Cornell University)|Jul 10, 2023

Big Data and Business IntelligenceBusiness, Management and Accounting3 citations

TL;DR

This paper proposes a data-driven strategic plan for LendingClub, a peer-to-peer lending platform, using statistical analysis and machine learning to enhance loan approval speed and reduce default risk. By analyzing 2 million loan records, the study identifies that higher loan amounts significantly increase charge-off rates, leading to recommendations for adopting a cloud-based big data platform with advanced feature engineering and ML models to improve predictive accuracy and operational efficiency.

ABSTRACT

Business statistics play a crucial role in implementing a data-driven strategic plan at the enterprise level to employ various analytics where the outcomes of such a plan enable an enterprise to enhance the decision-making process or to mitigate risks to the organization. In this work, a strategic plan informed by the statistical analysis is introduced for a financial company called LendingClub, where the plan is comprised of exploring the possibility of onboarding a big data platform along with advanced feature selection capacities. The main objectives of such a plan are to increase the company's revenue while reducing the risks of granting loans to borrowers who cannot return their loans. In this study, different hypotheses formulated to address the company's concerns are studied, where the results reveal that the amount of loans profoundly impacts the number of borrowers charging off their loans. Also, the proposed strategic plan includes onboarding advanced analytics such as machine learning technologies that allow the company to build better generalized data-driven predictive models.

Motivation & Objective

• To develop a data-driven strategic plan for LendingClub to improve loan risk assessment and decision-making speed.
• To investigate the relationship between loan amount and borrower charge-off rates using statistical hypothesis testing.
• To evaluate the feasibility of implementing a scalable big data analytics platform to support real-time loan approval and predictive modeling.
• To recommend enhanced feature selection and data collection strategies to improve model performance while managing infrastructure costs.

Proposed method

• Conducted hypothesis testing (t-tests, ANOVA) and correlation analysis to assess relationships between loan amount and charge-off flags.
• Performed data preprocessing, including cleaning, handling missing values, and feature engineering on a dataset with 145 attributes and 2M+ observations.
• Applied statistical models including linear and logistic regression, cluster analysis, and correlation analysis to identify key predictors of default.
• Designed a cloud-based big data architecture using AWS to support batch and stream processing for real-time analytics.
• Proposed an end-to-end pipeline with modular components: data ingestion, preprocessing, statistical/ML modeling, results storage, and visualization via user interface.
• Integrated automated scheduling for retraining machine learning models and optimizing feature selection based on quarterly statistical analysis.

Figure 1: Left: Distribution of borrowers Right: Distribution of loan amounts charged off

Experimental results

Research questions

• RQ1 Is there a statistically significant relationship between loan amount and the likelihood of borrower charge-off?
• RQ2 Do higher loan amounts correlate with increased default rates compared to lower loan amounts?
• RQ3 Can advanced feature selection and big data analytics improve the accuracy of default prediction models?
• RQ4 What are the operational and technical steps required to migrate LendingClub’s analytics pipeline to a cloud-based big data platform?
• RQ5 How can targeted data collection improve model performance while reducing infrastructure costs?

Key findings

• Higher loan amounts are significantly correlated with increased charge-off rates, indicating a strong relationship between loan size and default risk.
• The dataset contains 32 features with no missing values, supporting robust statistical modeling and predictive analytics.
• Hypothesis testing confirmed a statistically significant relationship between loan amount and charge-off flag (p < 0.05), supporting the need for risk-adjusted lending criteria.
• The proposed cloud-based big data architecture enables real-time data processing and supports scalable deployment of machine learning models.
• Targeted feature selection based on quarterly statistical analysis can reduce data volume and infrastructure load without compromising model performance.
• The integration of automated model retraining and enhanced feature engineering is expected to improve predictive accuracy and support 30-minute loan approval decisions.

Figure 2: Steps for migrating data pipeline to a cloud environment