[Paper Review] Identification with Latent Choice Sets: The Case of the Head Start Impact Study

Vishal Kamat|arXiv (Cornell University)|Nov 1, 2017

Early Childhood Education and Development3 citations

TL;DR

This paper develops a nonparametric model to identify program effects under unobserved heterogeneity in treatment choice sets, applying it to the Head Start Impact Study. Using linear programming, it constructs identified sets for policy-relevant parameters and finds that a significant share of children voluntarily enroll in Head Start if given access, with positive short-term test score impacts across multiple policy scenarios.

ABSTRACT

This paper studies identification of program effects in settings with latent choice sets. Here, by latent choice sets, I mean the unobserved heterogeneity that arises when the choice set from which the agent selects treatment is heterogeneous and unobserved by the researcher. The analysis is developed in the context of the Head Start Impact Study, a social experiment designed to evaluate preschools as part of Head Start, the largest early childhood education program in the United States. In this setting, resource constraints limit preschool slots to only a few eligible children through an assignment mechanism that is not observed in the data, which in turn introduces unobserved heterogeneity in the child's choice set of care settings. I propose a nonparametric model that explicitly accounts for latent choice sets in the care setting enrollment decision. In this model, I study various parameters that evaluate Head Start in terms of policies that mandate enrollment and also those that allow voluntary enrollment into Head Start. I show that the identified set for these parameters given the information provided by the study and by various institutional details of the setting can be constructed using a linear programming method. Applying the developed analysis, I find that a significant proportion of children voluntarily enroll into Head Start if provided access and that Head Start is effective in terms of improving short-term test scores across multiple policy dimensions.

Motivation & Objective

• To address identification challenges in program evaluation when treatment choice sets are unobserved and heterogeneous across individuals.
• To model unobserved heterogeneity in care setting choice sets arising from resource-constrained assignment mechanisms not recorded in data.
• To evaluate Head Start's effectiveness under both mandatory and voluntary enrollment policies using nonparametric methods.
• To construct identified sets for policy-relevant parameters using observable data and institutional details from the Head Start Impact Study.
• To provide a framework for causal inference in settings with latent choice sets, applicable to social programs with constrained access.

Proposed method

• Proposes a nonparametric structural model that accounts for unobserved heterogeneity in treatment choice sets.
• Incorporates institutional details of the Head Start assignment process to inform the structure of latent choice sets.
• Uses linear programming to construct the identified set for policy-relevant parameters under partial identification.
• Characterizes the set of parameters consistent with the observed data and institutional constraints.
• Employs revealed preference logic to infer potential choice sets from enrollment patterns and assignment rules.
• Applies the model to the Head Start Impact Study to estimate bounds on treatment effects under different enrollment policies.

Experimental results

Research questions

• RQ1 What is the identified set for the average treatment effect of Head Start under voluntary enrollment, given unobserved choice sets?
• RQ2 How does the identified set for Head Start’s impact change under a policy mandating enrollment for all eligible children?
• RQ3 To what extent do children voluntarily choose Head Start when access is provided, based on the observed data and institutional constraints?
• RQ4 How do different institutional rules governing slot allocation affect the identification of treatment effects in the presence of latent choice sets?
• RQ5 What is the range of plausible treatment effects on short-term test scores under various policy assumptions?

Key findings

• A significant proportion of eligible children would voluntarily enroll in Head Start if given access, indicating strong demand.
• Head Start produces positive short-term impacts on test scores across multiple policy scenarios, including both mandatory and voluntary enrollment.
• The identified set for treatment effects is non-degenerate, indicating meaningful identification despite unobserved choice set heterogeneity.
• The linear programming method successfully constructs sharp bounds on policy-relevant parameters using the available data and institutional details.
• The model reveals that institutional constraints on access significantly affect the interpretation of treatment effects and must be explicitly modeled.
• The results suggest that Head Start is effective not only under mandatory enrollment but also under voluntary access, with substantial potential for positive outcomes.