[Paper Review] Hidden Regular Variation: Detection and Estimation

Abhimanyu Mitra, Sidney I. Resnick|arXiv (Cornell University)|Jan 27, 2010

Financial Risk and Volatility Modeling23 references3 citations

TL;DR

This paper extends hidden regular variation (HRV) to higher-dimensional sub-cones beyond the standard $\mathbb{E}^{(2)}$ in $\mathbb{E} = [0,\infty]^d \setminus \{0\}$, proposing a detection procedure and a semi-parametric estimation method for the limit measure $\nu^{(l)}$ on $\mathbb{E}^{(l)}$. The key contribution is a consistent estimator that improves risk probability estimates—especially for rare events—by exploiting the semi-parametric structure of $\nu^{(l)}$, outperforming non-parametric methods like Heffernan-Resnick in numerical examples.

ABSTRACT

Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on $\mathbb{E} = [0, \infty]^d \backslash \{(0,0, ..., 0) \} $ and models another regular variation on the sub-cone $\mathbb{E}^{(2)} = \mathbb{E} \backslash \cup_{i=1}^d \mathbb{L}_i$, where $\mathbb{L}_i$ is the $i$-th axis. We extend the concept of hidden regular variation to sub-cones of $\mathbb{E}^{(2)}$ as well. We suggest a procedure for detecting the presence of hidden regular variation, and if it exists, propose a method of estimating the limit measure exploiting its semi-parametric structure. We exhibit examples where hidden regular variation yields better estimates of probabilities of risk sets.

Motivation & Objective

• To extend the theory of hidden regular variation (HRV) beyond two dimensions to sub-cones $\mathbb{E}^{(l)}$ for $3 \leq l \leq d$.
• To develop a detection procedure for HRV on $\mathbb{E}^{(l)}$ applicable in arbitrary finite dimensions.
• To propose a semi-parametric estimation method for the limit measure $\nu^{(l)}$ on $\mathbb{E}^{(l)}$, improving on non-parametric approaches.
• To demonstrate that HRV can yield non-zero estimates for risk probabilities that standard regular variation on $\mathbb{E}$ fails to capture.
• To address limitations in existing HRV frameworks, such as ignoring intermediate tail indices and lack of confidence intervals for estimates.

Proposed method

• Define HRV on nested sub-cones $\mathbb{E} \supset \mathbb{E}^{(2)} \supset \cdots \supset \mathbb{E}^{(d)}$, where $\mathbb{E}^{(l)}$ removes all $l$-dimensional coordinate planes.
• Propose a detection method based on empirical scaling behavior and angular measure analysis to identify HRV on $\mathbb{E}^{(l)}$.
• Transform the limit measure $\nu^{(l)}$ using a non-standard coordinate system to decompose it into a Pareto measure $\nu_{\alpha^{(l)}}$ and a probability measure $S^{(l)}$ on the unit sphere of $\mathbb{E}^{(l)}$, enabling semi-parametric estimation.
• Estimate the parametric part (tail index $\alpha^{(l)}$) and non-parametric part ($S^{(l)}$) of $\nu^{(l)}$ separately, improving efficiency.
• Construct a consistent non-parametric estimator for $\nu^{(l)}$ via empirical measures, serving as a benchmark.
• Use numerical experiments to compare the semi-parametric estimator with the Heffernan-Resnick estimator, particularly for tail risk probabilities.

Experimental results

Research questions

• RQ1 Can hidden regular variation be consistently defined and detected on higher-dimensional sub-cones $\mathbb{E}^{(l)}$ for $l \geq 3$?
• RQ2 Does HRV on $\mathbb{E}^{(l)}$ provide better estimates of rare event probabilities than standard regular variation on $\mathbb{E}$?
• RQ3 Can the semi-parametric structure of the limit measure $\nu^{(l)}$ be exploited to improve estimation efficiency over non-parametric methods?
• RQ4 Is the proposed detection procedure effective in identifying HRV on $\mathbb{E}^{(l)}$ in dimensions $d > 2$?
• RQ5 What are the statistical limitations of current HRV theory, particularly regarding confidence intervals and handling of degenerate limit measures?

Key findings

• The proposed semi-parametric estimator for $\nu^{(l)}$ yields non-zero estimates for risk probabilities that the Heffernan-Resnick non-parametric estimator reports as zero, particularly for extreme events like $P[\text{Size} > 2 \times 10^8, \text{Rate} > 10^6]$.
• Numerical results show the semi-parametric method consistently outperforms the non-parametric Heffernan-Resnick estimator in estimating tail probabilities, suggesting improved efficiency by leveraging structural assumptions.
• The method successfully detects HRV on $\mathbb{E}^{(l)}$ for $l \geq 3$, demonstrating that asymptotic independence is not a necessary condition for HRV in higher dimensions.
• The paper identifies a key limitation: HRV models may overlook intermediate tail indices (e.g., on $\mathbb{E}^{(2)}$ planes with $\alpha^{(2),2}$) that are more relevant than the $\mathbb{E}^{(3)}$-level HRV.
• A moment condition is derived to check the finiteness of $\nu^{(l)}(\{\mathbf{x} \in \mathbb{E}^{(l)} : \|\mathbf{x}\| > 1\})$, but it depends on the unknown angular measure $S^{(l)}$, highlighting a need for statistical tests for finiteness.
• The authors acknowledge the lack of confidence intervals for parameter or risk probability estimates, indicating a critical open statistical challenge in HRV inference.