[Paper Review] An Interpolating Family of Size Distributions

Corinne Sinner, Yves Dominicy+3 more|arXiv (Cornell University)|Jun 14, 2016

Statistical Distribution Estimation and Applications31 references3 citations

TL;DR

This paper introduces a novel five-parameter interpolating family of size distributions on $[x_0, ∞)$ that smoothly bridges power laws (e.g., Pareto) and power laws with exponential cutoff (e.g., Weibull), offering enhanced tractability and flexibility over the generalized beta distribution. The model enables seamless modeling across diverse data shapes with closed-form expressions for key functions and robust maximum likelihood estimation.

ABSTRACT

We introduce a new five-parameter family of size distributions on the semi-finite interval $[x_0, \infty), x_0 \geqslant 0$, with two attractive features. First, it interpolates between power laws, such as the Pareto distribution, and power laws with exponential cut-off, such as the Weibull distribution. The proposed family is thus very flexible and spans over a broad range of well-known size distributions which are special cases of our family. Second, it has important tractability advantages over the popular five-parameter Generalized Beta distribution. We derive the hazard function, survival function, modes and quantiles, propose a random number generation procedure and discuss maximum likelihood estimation issues. Finally, we illustrate the wide applicability and fitting capacities of our new model on basis of three real data sets from very diverse domains, namely actuarial science, environmental science and survival analysis.

Motivation & Objective

• Address the challenge of selecting between power law and power law with exponential cutoff distributions in diverse empirical applications.
• Develop a unified, flexible size distribution family that interpolates between Pareto and Weibull types while maintaining mathematical tractability.
• Overcome limitations of the generalized beta distribution by providing a model with simpler normalizing constants and clearer parameter interpretation.
• Ensure the model supports practical statistical inference through derivable hazard, survival, quantile, and mode functions.
• Demonstrate broad applicability across real-world domains such as actuarial science, environmental science, and survival analysis.

Proposed method

• Propose a new probability density function on $[x_0, ∞)$ that interpolates between power laws and power laws with exponential cutoff via a flexible transformation.
• Define the distribution using a location parameter $x_0$, scale parameter $c$, and shape parameters $b$, $p$, and $q$, with a closed-form density function.
• Derive the survival and hazard functions analytically, enabling reliability and risk modeling applications.
• Provide a closed-form expression for the mode by solving the first derivative of the density, with separate analysis for subfamilies IF1, IF2, and IF3.
• Develop a random variate generation procedure based on inverse transform sampling using the quantile function.
• Implement maximum likelihood estimation with discussion on identifiability and numerical stability, particularly for censored data in survival analysis.

Experimental results

Research questions

• RQ1 Can a single flexible distribution family effectively interpolate between power law and power law with exponential cutoff behaviors?
• RQ2 How does the proposed model compare in tractability and fitting performance to the generalized beta distribution across diverse data types?
• RQ3 What are the analytical properties of the hazard, survival, and mode functions in the new family?
• RQ4 Can the model accurately fit real-world data from disparate domains such as actuarial claims, environmental extremes, and survival data?
• RQ5 How do the shape parameters influence the distribution's behavior, particularly in transitioning between heavy-tailed and light-tailed regimes?

Key findings

• The proposed interpolating family includes the Pareto and Weibull distributions as special cases, and the generalized beta distribution as a limiting case, demonstrating broad inclusivity.
• The normalizing constant of the new distribution does not involve special functions, enhancing computational tractability compared to the generalized beta distribution.
• The mode of the distribution is derived in closed form for subfamilies IF1 ($p=0$), IF2 ($p \to \infty$), and IF3 ($b=1$), with explicit expressions provided.
• For IF1, the mode is $x_0 + c\left(\frac{b-1}{bq+1}\right)^{1/b}$ when $b > 1$ or $b < -1/q$, and occurs at the boundary otherwise.
• For IF3, the mode is $x_0 + c(p+1)^{-1/q}\left(\left(\frac{q+1}{(p+1)q+1}\right)^{-1/q} - 1\right)$, with convergence to the boundary as $p, q \to 0$.
• The model demonstrates strong fitting performance on three real datasets: actuarial claim sizes, environmental wind speeds, and survival data, outperforming standard alternatives in terms of fit and interpretability.