[Paper Review] Small area estimation of general finite-population parameters based on grouped data

Yuki Kawakubo, Genya Kobayashi|arXiv (Cornell University)|Mar 18, 2019

Spatial and Panel Data Analysis16 references4 citations

TL;DR

This paper proposes a novel model-based small area estimation method for general finite-population parameters using grouped data, such as income class frequencies. It employs a latent variable approach with a linear mixed model and multinomial likelihood to link group probabilities to auxiliary variables, enabling empirical Bayes estimation via Gibbs sampling and a Monte Carlo EM algorithm with importance sampling.

ABSTRACT

This paper proposes a new model-based approach to small area estimation of general finite-population parameters based on grouped data or frequency data, which is often available from sample surveys. Grouped data contains information on frequencies of some pre-specified groups in each area, for example the numbers of households in the income classes, and thus provides more detailed insight about small areas than area-level aggregated data. A direct application of the widely used small area methods, such as the Fay-Herriot model for area-level data and nested error regression model for unit-level data, is not appropriate since they are not designed for grouped data. The newly proposed method adopts the multinomial likelihood function for the grouped data. In order to connect the group probabilities of the multinomial likelihood and the auxiliary variables within the framework of small area estimation, we introduce the unobserved unit-level quantities of interest which follows the linear mixed model with the random intercepts and dispersions after some transformation. Then the probabilities that a unit belongs to the groups can be derived and are used to construct the likelihood function for the grouped data given the random effects. The unknown model parameters (hyperparameters) are estimated by a newly developed Monte Carlo EM algorithm using an efficient importance sampling. The empirical best predicts (empirical Bayes estimates) of small area parameters can be calculated by a simple Gibbs sampling algorithm. The numerical performance of the proposed method is illustrated based on the model-based and design-based simulations. In the application to the city level grouped income data of Japan, we complete the patchy maps of the Gini coefficient as well as mean income across the country.

Motivation & Objective

• Address the lack of reliable small area estimators for general finite-population parameters when only grouped data (e.g., income class frequencies) are available.
• Overcome the limitations of existing Fay–Herriot and nested error models, which are not applicable to grouped data due to absence of unit-level information.
• Develop a unified framework that connects grouped data frequencies to auxiliary variables through latent unit-level variables and random effects.
• Enable estimation of complex parameters such as Gini coefficients and means at the small area level using only frequency data.
• Provide a computationally feasible estimation procedure using Monte Carlo EM and Gibbs sampling for empirical Bayes prediction.

Proposed method

• Model grouped data using a multinomial likelihood function based on observed frequencies in predefined groups.
• Introduce unobserved latent unit-level variables representing the true values of interest, constrained to lie within group intervals.
• Assume the latent variables follow a linear mixed model with random intercepts and heteroscedastic errors to link to auxiliary variables.
• Derive the likelihood of grouped data conditional on the latent variables, random effects, and variance components.
• Use a Monte Carlo EM algorithm with efficient importance sampling to estimate hyperparameters (e.g., variance components).
• Compute empirical best predictors via Gibbs sampling from the full conditional distributions of the latent variables and random effects.

Experimental results

Research questions

• RQ1 Can a model-based small area estimation framework be developed for general finite-population parameters when only grouped data (e.g., income class frequencies) are available?
• RQ2 How can latent unit-level variables be used to connect grouped data frequencies to auxiliary variables in a mixed model framework?
• RQ3 What is an efficient computational method for estimating hyperparameters in a model with grouped data and latent variables?
• RQ4 How does the proposed method perform in estimating complex parameters such as the Gini coefficient and mean income at the small area level?
• RQ5 Can the method produce reliable and stable small area estimates even when sample sizes are small and direct estimators are unreliable?

Key findings

• The proposed method successfully enables small area estimation of general finite-population parameters, including the Gini coefficient and mean income, using only grouped data.
• The Monte Carlo EM algorithm with importance sampling provides stable and accurate estimation of hyperparameters, even in high-dimensional latent variable spaces.
• Empirical Bayes estimates of small area parameters are obtained efficiently via Gibbs sampling, with full conditional distributions derived analytically.
• In simulation studies, the method outperforms direct estimators in terms of mean squared error, especially for small areas with limited sample sizes.
• Application to Japanese city-level income data produced patchy maps of the Gini coefficient and mean income across 1,265 municipalities, demonstrating practical utility.
• The method effectively handles the uncertainty in grouped data by borrowing strength across areas through the hierarchical mixed model structure.