[Paper Review] Fast and Feasible Estimation of Generalized Linear Models with High-Dimensional k-way Fixed Effects

Amrei Stammann|arXiv (Cornell University)|Jul 6, 2017

Control Systems and Identification20 citations

TL;DR

This paper presents a fast, memory-efficient algorithm for estimating generalized linear models (GLMs) with high-dimensional k-way fixed effects using a Newton-Raphson framework combined with a weighted Frisch-Waugh-Lovell theorem and alternating projections. The method enables feasible estimation of logit, probit, and Poisson models with complex fixed effects structures by iteratively concentrating out fixed effects, achieving near-perfect accuracy and dramatic speedups over dummy variable approaches.

ABSTRACT

We present a fast and memory efficient algorithm for the estimation of generalized linear models with an additive separable k-way error component. The brute force approach uses dummy variables to account for the unobserved heterogeneity, but quickly faces computational limits. Thus, we show how a weighted version of the Frisch-Waugh-Lovell theorem combined with the method of alternating projections can be incorporated into a Newton-Raphson algorithm to dramatically reduce the computational costs. The algorithm is especially useful in situations, where generalized linear models with k-way fixed effects based on dummy variables are computationally demanding or even infeasible due to time or memory limitations. In a simulation study and an empirical application we demonstrate the performance of our algorithm.

Motivation & Objective

• To address the computational infeasibility of estimating generalized linear models with high-dimensional k-way fixed effects using standard dummy variable methods.
• To extend efficient fixed effects estimation—previously limited to linear models—into the generalized linear model framework.
• To develop a memory- and time-efficient algorithm that maintains high numerical accuracy for logit, probit, and Poisson models.
• To enable practical estimation of complex fixed effects models in large panel datasets, such as those in labor economics and trade econometrics.
• To provide a robust, scalable alternative to brute-force dummy variable estimation in high-dimensional settings.

Proposed method

• The algorithm integrates the weighted Frisch-Waugh-Lovell theorem into each Newton-Raphson iteration to project out fixed effects from the linear predictor.
• It employs the method of alternating projections to iteratively demean the data with respect to each fixed effect dimension, reducing computational burden.
• A pseudo-demeaning step is applied in each iteration to concentrate out fixed effects without explicitly estimating them, preserving numerical efficiency.
• The approach uses Neumann-Halperin projection with adjustable tolerance levels to balance speed and accuracy.
• Fixed effect estimates are recovered ex-post after optimization, avoiding direct inversion of large design matrices.
• The method is implemented in the R package alpaca, designed specifically for non-linear GLMs where lfe already handles linear models.

Experimental results

Research questions

• RQ1 Can a computationally efficient algorithm be developed for generalized linear models with high-dimensional k-way fixed effects?
• RQ2 How accurately can fixed effects be estimated using iterative projection methods compared to the dummy variable benchmark?
• RQ3 To what extent does the proposed algorithm reduce computation time and memory usage in large panel datasets?
• RQ4 Does the method maintain numerical stability and convergence across diverse model specifications and data sizes?
• RQ5 Can the algorithm be generalized across different GLM families such as logit, probit, and Poisson?

Key findings

• The algorithm achieves near-perfect numerical accuracy, with exactness frequencies of 1.00 (100%) for up to 5 and 8-digit precision across all tested N-T combinations.
• For 16-digit precision, exactness drops to 0.00 in some cases, but this reflects the inherent limitations of floating-point arithmetic rather than algorithmic failure.
• Computation time for the alpaca algorithm remains under 1.5 seconds even for N=200, T=50, while the dummy variable approach takes over 140 seconds.
• The average computation time for alpaca scales sublinearly with sample size, whereas the dummy variable method scales quadratically or worse.
• The method maintains high accuracy across all tested GLM families, including logit, probit, and Poisson, with consistent performance across multiple data configurations.
• The algorithm enables estimation of three-way fixed effects PPML models in large datasets (e.g., N=200, T=50) that would be infeasible with standard dummy variable methods.