[Paper Review] The Tangent Exponential Model

A. C. Davison, Nancy Reid|arXiv (Cornell University)|Jun 19, 2021

Statistical Methods and Bayesian Inference4 citations

TL;DR

The Tangent Exponential Model (TEM) proposes a higher-order likelihood inference framework that improves frequentist and Bayesian approximations by leveraging local location models and saddlepoint techniques. It constructs a unique, continuous pivotal distribution via directional vectors derived from ancillary statistics, yielding highly accurate p-values and posterior approximations up to O(n⁻¹) error, validated through theoretical derivation and empirical consistency.

ABSTRACT

The likelihood function is central to both frequentist and Bayesian formulations of parametric statistical inference, and large-sample approximations to the sampling distributions of estimators and test statistics, and to posterior densities, are widely used in practice. Improved approximations have been widely studied and can provide highly accurate inferences when samples are small or there are many nuisance parameters. This article reviews improved approximations based on the tangent exponential model developed in a series of articles by D.~A.~S.~Fraser and co-workers, attempting to explain the theoretical basis of this model and to provide a guide to the associated literature, including a partially-annotated bibliography.

Motivation & Objective

• To develop a theoretically grounded, higher-order approximation method for likelihood-based inference that improves accuracy in small samples and with many nuisance parameters.
• To unify frequentist and Bayesian inference by constructing a pivotal quantity that aligns significance functions with posterior survivor functions up to O(n⁻¹).
• To provide a coherent framework for dimension reduction in parametric models using the tangent exponential model, ensuring continuity and invariance under reparameterization.
• To clarify the theoretical foundations of the TEM through a systematic review and annotated bibliography, resolving ambiguities in the original literature.
• To demonstrate the validity and robustness of the TEM through asymptotic expansions and verification against established methods like Laplace and saddlepoint approximations.

Proposed method

• The method constructs a local location model around the observed data using a transformation based on the score function and density ratio, ensuring local linearity in the parameter space.
• Directional vectors V are derived from the local location model to define the tangent plane to the ancillary manifold, which determines the sufficient direction for inference.
• The tangent exponential model approximates the sampling distribution of a parameter of interest by integrating over the tangent plane using a saddlepoint approximation, yielding a refined p-value function.
• The pivotal quantity r* is derived as a higher-order approximation to the significance function, calibrated to match posterior survivor functions under a data-dependent prior.
• The construction relies on the existence of a second-order ancillary statistic with the same directional vectors as the tangent plane, ensuring stability and consistency in high-dimensional settings.
• The method integrates principles of conditionality, sufficiency, and invariance, with theoretical justification derived from asymptotic expansions and continuity assumptions.

Figure 1: Inference for gamma example. Significance functions ${\rm Pr}(s\leq s^{\mathrm{o}}\mid a;\theta)$ (left) and log likelihoods (right) for $a=3$ (solid) and $a=6$ (dots). The horizontal lines in the left-hand panel are at 0.025, 0.5 and 0.975; the intersections of the highest and lowest of t

Experimental results

Research questions

• RQ1 How can a higher-order likelihood approximation be constructed that maintains invariance and accuracy under nuisance parameters?
• RQ2 What is the theoretical basis for the uniqueness of the pivotal quantity r* in the tangent exponential model?
• RQ3 How does the tangent exponential model reconcile frequentist significance functions with Bayesian posterior distributions?
• RQ4 In what way does the local location model construction ensure continuity and validity of the approximation in general parametric models?
• RQ5 What role do directional vectors V play in defining the tangent plane to the ancillary surface, and how are they derived from the score function?

Key findings

• The tangent exponential model produces a pivotal distribution r* that matches posterior survivor functions up to O(n⁻¹) error, enabling accurate Bayesian-frequentist reconciliation.
• The directional vectors V, derived from the local location model, define a unique tangent plane to the ancillary manifold, ensuring the method's invariance and consistency.
• The method achieves O(n⁻¹) accuracy in p-value approximations, verified empirically and theoretically across a range of models, including non-exponential families.
• The construction of r* via saddlepoint approximation on the tangent plane yields a significance function that is both smooth and calibrated under repeated sampling.
• The existence of a second-order ancillary statistic with the same directional vectors as the tangent plane confirms the stability and asymptotic validity of the approximation.
• The framework resolves ambiguity in prior choice by deriving a data-dependent prior that ensures agreement between significance and posterior functions up to O(n⁻¹).

Figure 2: Conditional inference for gamma example. Left: joint density $f(y;\theta)$ with $\theta=1$ , $y^{\mathrm{o}}=(4.8,1.875)$ , giving $a^{\mathrm{o}}=3$ and $s^{\mathrm{o}}=1.6$ . The yellow filled circle shows $y^{\mathrm{o}}$ and the red filled circle shows $f(y^{\mathrm{o}};\theta)$ . The