[Paper Review] Schwarz type model comparison for LAQ models

Shoichi Eguchi, H. Masuda|arXiv (Cornell University)|Jun 6, 2016

Bayesian Methods and Mixture Models31 references3 citations

TL;DR

This paper proposes a quasi-Bayesian information criterion (QBIC) for model selection in locally asymptotically quadratic (LAQ) models, extending Schwarz's BIC to non-ergodic and dependent data models with random asymptotic information matrices. The method uses stochastic expansions of marginal quasi-likelihood and establishes consistency of model selection under general LAQ conditions, including ergodic diffusion and non-ergodic processes.

ABSTRACT

For model-specification purpose, we study asymptotic behavior of the marginal quasi-log likelihood associated with a family of locally asymptotically quadratic (LAQ) statistical experiments. Our result entails a far-reaching extension of applicable scope of the classical approximate Bayesian model comparison due to Schwarz, with frequentist-view theoretical foundation. In particular, the proposed statistics can deal with both ergodic and non-ergodic stochastic-process models, where the corresponding $M$-estimator is of multi-scaling type and the asymptotic quasi-information matrix is random. Focusing on the ergodic diffusion model, we also deduce the consistency of the multistage optimal-model selection where we may select an optimal sub-model structure step by step, so that computational cost can be much reduced. We illustrate the proposed method by the Gaussian quasi-likelihood for diffusion-type models in details, together with several numerical experiments.

Motivation & Objective

• To extend the classical Schwarz BIC to a broader class of statistical models, particularly those with dependent or non-ergodic data structures.
• To provide a frequentist foundation for Bayesian model comparison in locally asymptotically quadratic (LAQ) models, where the asymptotic information matrix may be random.
• To develop a unified framework for quasi-likelihood-based model selection that remains valid even under model misspecification or infinite-dimensional nuisance parameters.
• To establish consistency of multistage model selection in ergodic diffusion models, reducing computational cost through sequential structure selection.
• To generalize existing information criteria to handle stochastic processes with multi-scaling M-estimators and random limiting Fisher information.

Proposed method

• Derives an asymptotic expansion of the marginal quasi-likelihood using polynomial-type large deviation inequality (PLDI), enabling analysis under random asymptotic information.
• Introduces a quasi-Bayesian information criterion (QBIC) based on the stochastic expansion, incorporating observed information matrix and log-determinant terms.
• Applies the QBIC to Gaussian quasi-likelihood for diffusion-type models, ensuring theoretical validity under LAQ structure.
• Uses a two-stage selection procedure in ergodic diffusion models, where sub-models are selected sequentially to reduce computational burden.
• Employs a scaling parameter $ a_n $ to handle multi-scaling behavior of M-estimators, with asymptotic distributions derived via normalized score processes.
• Establishes consistency of model selection by showing the probability that the true model achieves the lowest QBIC converges to one as $ n \to \infty $.

Experimental results

Research questions

• RQ1 Can Schwarz-type model comparison be extended to non-ergodic stochastic processes where the asymptotic Fisher information matrix is random?
• RQ2 Does the proposed QBIC maintain consistency for LAQ models with multi-scaling M-estimators and random limiting information?
• RQ3 Can multistage model selection be consistently applied in ergodic diffusion models to reduce computational cost?
• RQ4 How does the QBIC perform under model misspecification or in semiparametric settings with infinite-dimensional parameters?
• RQ5 Is the classical BIC form recoverable in non-i.i.d. settings such as non-stationary time series or cointegrated models?

Key findings

• The proposed QBIC achieves model selection consistency for LAQ models, even when the asymptotic information matrix is random, under mild regularity conditions.
• For ergodic diffusion models, the multistage optimal model selection procedure is consistent, with the probability of selecting the true model converging to one as sample size increases.
• The QBIC asymptotically approximates the marginal quasi-likelihood, with the difference between models converging in probability to a negative value under the true model.
• The method generalizes classical BIC to non-ergodic and dependent data models, including cointegrated and non-stationary time series, where previous criteria fail.
• The asymptotic expansion of the quasi-likelihood relies on PLDI, allowing the derivation of the QBIC even when the observed information matrix converges in distribution to a random limit.
• Numerical experiments confirm the validity of the QBIC in Gaussian quasi-likelihood settings, demonstrating robustness and consistency across various model structures.