[Paper Review] Enhanced Gradient Boosting for Zero-Inflated Insurance Claims and Comparative Analysis of CatBoost, XGBoost, and LightGBM

Banghee So|arXiv (Cornell University)|Jul 15, 2023

Probability and Risk ModelsDecision Sciences9 citations

TL;DR

This paper compares CatBoost, XGBoost, and LightGBM for zero-inflated insurance claim frequency modeling, introduces zero-inflated Poisson boosted trees (ZIPBT), and evaluates performance on MTPL and telematics datasets.

ABSTRACT

The property and casualty (P&C) insurance industry faces challenges in developing claim predictive models due to the highly right-skewed distribution of positive claims with excess zeros. To address this, actuarial science researchers have employed "zero-inflated" models that combine a traditional count model and a binary model. This paper investigates the use of boosting algorithms to process insurance claim data, including zero-inflated telematics data, to construct claim frequency models. Three popular gradient boosting libraries - XGBoost, LightGBM, and CatBoost - are evaluated and compared to determine the most suitable library for training insurance claim data and fitting actuarial frequency models. Through a comprehensive analysis of two distinct datasets, it is determined that CatBoost is the best for developing auto claim frequency models based on predictive performance. Furthermore, we propose a new zero-inflated Poisson boosted tree model, with variation in the assumption about the relationship between inflation probability $p$ and distribution mean $μ$, and find that it outperforms others depending on data characteristics. This model enables us to take advantage of particular CatBoost tools, which makes it easier and more convenient to investigate the effects and interactions of various risk features on the frequency model when using telematics data.

Motivation & Objective

• Motivate the use of boosting methods for zero-inflated insurance claim data, including telematics-driven features.
• Develop and compare zero-inflated Poisson boosted tree models against standard Poisson and ZIP GLMs.
• Evaluate library-specific implementations (CatBoost, XGBoost, LightGBM) for actuarial frequency modeling.
• Propose ZIP boosted tree formulations that link inflation probability p with mean μ to improve interpretability and interaction analysis.

Proposed method

• Review of gradient boosting basics and how XGBoost, LightGBM, and CatBoost implement GBDT with regularized loss.
• Development of zero-inflated Poisson boosted tree (ZIPBT) with two parameterizations: (a) p as a function of μ, (b) p and μ unrelated, via two coordinated training regimes.
• Derivation of the ZIP loss, gradients, and Hessians for the p(μ) link and the independent-parts approach.
• Coordinate descent style training to alternate optimization between μ-related and p-related components (Algorithms 1 and 2 in Appendix).
• Empirical evaluation on French MTPL and a synthetic telematics dataset, with 11 models including Poisson boosted trees, ZIPBT variants, and GLMs, using cross-validation and test split.

Experimental results

Research questions

• RQ1 Can CatBoost, XGBoost, and LightGBM effectively fit zero-inflated insurance claim data, and which library provides superior predictive performance for auto claim frequency?
• RQ2 Do ZIP boosted tree variants that link inflation p to mean μ outperform standard Poisson, ZIP, and GLMs on zero-inflated datasets?
• RQ3 What are the relative gains of modeling p as a function of μ versus modeling p and μ independently in ZIP boosted trees?
• RQ4 How do telematics-derived features interact with traditional risk factors in a boosted-tree framework for frequency modeling?

Key findings

• CatBoost achieves the best predictive performance for auto claim frequency among the three libraries on the evaluated datasets.
• ZIP boosted tree variants (ZIPB1 and ZIPB2) can outperform Poisson boosted trees and standard ZIP/GLM approaches depending on data characteristics.
• Model evaluation using Deviance, Pseudo R-squared, Vuong test, and Randomized Quantile Residuals favors ZIP-based approaches over traditional Poisson models in zero-inflated settings.
• In MTPL and telematics datasets, zero-inflation handling and interaction-capable boosting improve frequency prediction for highly imbalanced claim data.