[Paper Review] Estimating Counterfactual Matrix Means with Short Panel Data

Lihua Lei, Brad Ross|arXiv (Cornell University)|Dec 12, 2023

Spatial and Panel Data Analysis4 citations

TL;DR

This paper proposes a novel method for estimating counterfactual matrix means in short panel data with missing outcomes, leveraging low-rank factor models to handle general missingness patterns. It enables consistent, asymptotically normal estimation under fixed outcome dimensions and computationally efficient inference via a single eigendecomposition of aggregated factor estimates, outperforming TWFE-based estimators in semi-synthetic simulations.

ABSTRACT

We develop a spectral approach for identifying and estimating average counterfactual outcomes under a low-rank factor model with short panel data and general outcome missingness patterns. Applications include event studies and studies of outcomes of "matches" between agents of two types, e.g. people and places, typically conducted using less-flexible Two-Way Fixed Effects (TWFE) models of outcomes. Given finite observed outcomes per unit, we show our approach identifies all counterfactual outcome means, including those not identified by existing methods, if a particular graph algorithm determines that units' sets of observed outcomes have sufficient overlap. Our analogous, computationally efficient estimation procedure yields consistent, asymptotically normal estimates of counterfactual outcome means under fixed-$T$ (number of outcomes), large-$N$ (sample size) asymptotics. When estimating province-level averages of held-out wages from an Italian matched employer-employee dataset, our estimator outperforms a TWFE-model-based estimator.

Motivation & Objective

• To address the challenge of estimating counterfactual outcomes when only a few outcomes are observed per unit in large cross-sections.
• To develop a method that handles general outcome missingness patterns not accommodated by existing factor model-based approaches.
• To enable consistent and asymptotically normal estimation of counterfactual means under low-dimensional unobserved confounders.
• To ensure computational efficiency by requiring only a single eigendecomposition of aggregated factor estimates.
• To bridge the gap between the flexibility of factor models and the robustness of TWFE estimators in short panel settings.

Proposed method

• Uses a low-rank factor model to represent outcomes, allowing multidimensional unobserved confounders to affect outcomes differently across outcomes.
• Estimates factor loadings and common shocks via subset-based aggregation of units with identical observed outcome patterns.
• Applies eigendecomposition to a covariance-like matrix constructed from estimated factor components to extract counterfactual mean estimates.
• Implements a projection-based estimator for the counterfactual mean matrix using the eigenspace of the aggregated factor estimate matrix.
• Derives asymptotic normality and inference validity under regularity conditions, including bounded operator norms and strict exogeneity.
• Employs a two-step estimation procedure: first estimate factor structures per outcome pattern group, then aggregate and decompose to recover counterfactual means.

Experimental results

Research questions

• RQ1 Can counterfactual matrix means be consistently estimated when only a few outcomes are observed per unit in a large cross-section?
• RQ2 How can general outcome missingness patterns—beyond those supported by TWFE or existing factor models—be accommodated in short panel data?
• RQ3 What is the computational and statistical efficiency of a method that relies on a single eigendecomposition of aggregated factor estimates?
• RQ4 Does the proposed method outperform TWFE-based estimators in settings with complex missingness and confounding?
• RQ5 Under what conditions is the estimator asymptotically normal and valid for inference?

Key findings

• The proposed method yields consistent and asymptotically normal estimates of counterfactual outcome means under fixed outcome dimensions and general missingness patterns.
• The estimator achieves computational efficiency by requiring only a single eigendecomposition of a matrix constructed from subset-based factor estimates.
• In a semi-synthetic simulation using matched employer-employee data, the method outperforms a TWFE-based estimator in terms of bias and mean squared error.
• Theoretical results show that the estimator’s asymptotic distribution is valid under mild regularity conditions, including bounded operator norms and strict exogeneity.
• The method maintains validity even when missingness is non-ignorable, provided confounders are low-dimensional and captured by the factor model.
• The convergence rate of the estimator is $O_p(N^{-1/2})$, and the asymptotic distribution is derived via a perturbation-theoretic approach to eigenspace estimation.