[Paper Review] Spatial automatic subgroup analysis for areal data with repeated measures

Xin Wang, Zhengyuan Zhu+1 more|arXiv (Cornell University)|Jun 5, 2019

Spatial and Panel Data AnalysisEconomics, Econometrics and Finance34 references3 citations

TL;DR

This paper proposes Spatial automatic Subgroup analysis (SaSa), a method for identifying spatially structured subgroups in areal data with repeated measures using a spatially-weighted concave pairwise fusion penalty. By leveraging spatial proximity through location-specific weights and solving the optimization via ADMM, SaSa achieves clustering consistency and improves subgroup detection accuracy, especially when group differences are small or repeated measures are limited.

ABSTRACT

We consider the subgroup analysis problem for spatial areal data with repeated measures. To take into account spatial data structures, we propose to use a class of spatially-weighted concave pairwise fusion method which minimizes the objective function subject to a weighted pairwise penalty, referred as Spatial automatic Subgroup analysis (SaSa). The penalty is imposed on all pairs of observations, with the location specific weight being chosen for each pair based on their corresponding spatial information. The alternating direction method of multiplier algorithm (ADMM) is applied to obtain the estimates. We show that the oracle estimator based on weighted least squares is a local minimizer of the objective function with probability approaching 1 under some conditions, which also indicates the clustering consistency properties. In simulation studies, we demonstrate the performances of the proposed method equipped with different weights in terms of their accuracy for estimating the number of subgroups. The results suggest that spatial information can enhance subgroup analysis in certain challenging situations when the minimal group difference is small or the number of repeated measures is small. The proposed method is then applied to find the relationship between two surveys, which can provide spatially interpretable groups.

Motivation & Objective

• To address subgroup analysis in spatial areal data with repeated measurements, where traditional methods may overlook spatial structure.
• To improve subgroup detection accuracy in challenging scenarios with small group differences or limited repeated measures.
• To incorporate spatial dependence into subgroup identification by assigning location-specific weights to pairwise observations.
• To ensure clustering consistency by proving the oracle estimator is a local minimizer under regularity conditions.
• To provide spatially interpretable subgroups that reflect meaningful geographic patterns in survey data.

Proposed method

• The method employs a spatially-weighted concave pairwise fusion penalty to encourage grouping of spatially proximate observations with similar responses.
• The penalty is defined over all pairs of observations, with weights derived from spatial proximity to reflect the influence of neighboring locations.
• An alternating direction method of multipliers (ADMM) algorithm is used to efficiently solve the non-convex optimization problem.
• The objective function minimizes a weighted least squares criterion subject to the spatially informed fusion penalty.
• The method estimates subgroup membership by shrinking coefficients of similar spatial units toward a common value, effectively clustering them.
• Theoretical analysis shows that under regularity conditions, the oracle estimator is a local minimizer with probability approaching one, implying clustering consistency.

Experimental results

Research questions

• RQ1 Can spatial information improve the accuracy of subgroup detection in areal data with repeated measures when group differences are small?
• RQ2 How does the inclusion of spatial proximity in the fusion penalty affect the identification of true subgroups compared to non-spatial methods?
• RQ3 What is the performance of the proposed method in terms of estimating the correct number of subgroups under varying levels of signal strength and repeated measures?
• RQ4 Does the method maintain clustering consistency when the number of repeated measures is small or the spatial structure is complex?
• RQ5 Can the method uncover meaningful, spatially interpretable subgroups in real-world survey data?

Key findings

• The proposed SaSa method significantly improves subgroup detection accuracy when the minimal group difference is small or the number of repeated measures is limited, as demonstrated in simulation studies.
• Different spatial weight configurations in the fusion penalty lead to varying performance, with proximity-based weights yielding the most accurate subgroup estimation.
• Theoretical results confirm that the oracle estimator is a local minimizer of the objective function with probability approaching one, supporting the method's clustering consistency.
• The method successfully identifies spatially coherent subgroups in real survey data, revealing interpretable geographic patterns.
• ADMM enables efficient optimization of the non-convex problem, making the method scalable and practical for real-world applications.