[Paper Review] Forecasting racial dynamics at the neighborhood scale using Density-functional Fluctuation Theory

Yunus A. Kinkhabwala, Boris Barron+3 more|arXiv (Cornell University)|Aug 5, 2021

Urban, Neighborhood, and Segregation Studies33 references4 citations

TL;DR

This paper introduces Density-Functional Fluctuation Theory (DFFT) to forecast neighborhood-scale racial dynamics in the U.S. using only coarse demographic data, bypassing the need for individual-level migration records. By extracting density-dependent segregation functions from 1990–2010 U.S. Census block group data, DFFT accurately predicts changes in racial/ethnic composition, achieving log-likelihoods of 4.13×10⁻³ (White), 3.74×10⁻³ (Hispanic), and 3.41×10⁻³ (Black), with robust uncertainty quantification via bootstrapping.

ABSTRACT

Racial residential segregation is a defining and enduring feature of U.S. society, shaping inter-group relations, racial disparities in income and health, and access to high-quality public goods and services. The design of policies aimed at addressing these inequities would be better informed by descriptive models of segregation that are able to predict neighborhood scale racial sorting dynamics. While coarse regional population projections are widely accessible, small area population changes remain challenging to predict because granular data on migration is limited and mobility behaviors are driven by complex social and idiosyncratic dynamics. Consequently, to account for such drivers, it is necessary to develop methods that can extract effective descriptions of their impacts on population dynamics based solely on statistical analysis of available data. Here, we develop and validate a Density-Functional Fluctuation Theory (DFFT) that quantifies segregation using density-dependent functions extracted from population counts and uses these functions to accurately forecast how the racial/ethnic compositions of neighborhoods across the US are likely to change. Importantly, DFFT makes minimal assumptions about the nature of the underlying causes of segregation and is designed to quantify segregation for neighborhoods with different total populations in regions with different compositions. This quantification can be used to accurately forecast both average changes in neighborhood compositions and the likelihood of more drastic changes such as those associated with gentrification and neighborhood tipping. As such, DFFT provides a powerful framework for researchers and policy makers alike to better quantify and forecast neighborhood-scale segregation and its associated dynamics.

Motivation & Objective

• To develop a method for forecasting small-area racial/ethnic population changes at the neighborhood scale despite limited granular migration data.
• To overcome the limitations of scalar segregation indices, which fail to distinguish between distinct migration dynamics leading to the same index value.
• To create a statistical framework that extracts effective segregation functions from available demographic counts to model neighborhood-scale population dynamics.
• To provide a policy-relevant tool for predicting neighborhood-level changes such as gentrification and tipping, informed by nationwide statistical patterns.
• To validate the model using 2010 census data and quantify forecast uncertainty through bootstrapping and data quality filtering.

Proposed method

• DFFT applies principles from statistical physics to model segregation as a function of population density, using a frustration function that quantifies deviation from expected racial/ethnic distributions.
• The theory derives a probability distribution over neighborhood compositions based on a log-likelihood function that penalizes improbable configurations, normalized by total population to allow cross-regional comparison.
• Frustration functions are inferred via statistical fitting to 1990–2010 U.S. Census block group data, with time scales τ(C) optimized using longitudinally aligned block group geometries.
• Outliers are mitigated by applying a log-likelihood floor of -10 and excluding neighborhoods with total population changes exceeding 25%.
• Uncertainty in nationwide average frustration functions is quantified using bootstrapped subsamples of counties with at least 50 million people.
• The model is validated by forecasting 2010 compositions from 1990–2000 data and comparing results to actual 2010 block group counts.

Experimental results

Research questions

• RQ1 Can segregation dynamics at the neighborhood scale be accurately forecast using only coarse-grained demographic data, without individual-level migration records?
• RQ2 How can segregation be modeled beyond scalar indices to capture distinct migration dynamics that yield the same index value?
• RQ3 To what extent can a density-functional approach extract effective, predictive functions of segregation from available census data?
• RQ4 How do forecast uncertainties vary across different racial/ethnic groups and population sizes?
• RQ5 Can the model predict not only average composition changes but also extreme shifts such as gentrification or tipping?

Key findings

• DFFT achieves a log-likelihood of 4.13×10⁻³ for White population forecasts, 3.74×10⁻³ for Hispanic, and 3.41×10⁻³ for Black subgroups, indicating strong predictive accuracy.
• The model successfully forecasts 2010 neighborhood compositions using only 1990–2000 data, demonstrating robustness across time scales.
• Bootstrapping reveals that the nationwide average frustration function has a standard deviation within the shaded uncertainty regions shown in figures 3 and the supplementary materials.
• Excluding neighborhoods with population changes >25% and applying a log-likelihood floor of -10 significantly improves forecast reliability by reducing outlier influence.
• The framework enables geospatial exploration of forecast accuracy and future projections via open-source tools hosted on OSF and GitHub.
• The method provides a scalable, data-driven alternative to ad hoc models, with applications in policy planning and segregation research.