[Paper Review] Semiparametric Estimation of a CES Demand System with Observed and Unobserved Product Characteristics

Alı Hortaçsu, Joonhwi Joo|arXiv (Cornell University)|Nov 17, 2015

Consumer Market Behavior and Pricing82 references3 citations

TL;DR

This paper develops a semiparametric estimation framework for a CES demand system that jointly models the whether-to-buy and how-much-to-buy decisions using observed and unobserved product characteristics. By embedding a quality kernel in a two-stage discrete-continuous choice model, it enables consistent estimation even with zero market shares—preventing upward-sloping demand curves that arise when such shares are ignored or imputed.

ABSTRACT

We develop a characteristics based demand estimation framework for the Marshallian demand system obtained by solving a budget-constrained constant elasticity of substitution (CES) utility maximization problem. From our Marshallian CES demand system, we derive the same market share equation of Berry (1994); Berry, Levinsohn, and Pakes (1995)'s characteristics based logit demand system. Our CES demand estimation framework can accommodate zero predicted and observed market shares by conceptually separating the whether-to-buy decision and how-much-to-buy decision. Furthermore, the estimator we suggest allows a tractable semiparametric estimation strategy that is flexible regarding the distribution of unobservable product characteristics. We apply our framework to scanner data on cola sales, where we show estimated demand curves can be upward sloping if zero market shares are not accommodated properly.

Motivation & Objective

• To reconcile the CES utility-based demand model with the widely used Berry (1994) logit framework using aggregate market-level data.
• To address the problem of zero observed and predicted market shares in demand estimation, which can lead to biased or inconsistent estimates.
• To provide a microeconomically grounded framework that separates intensive and extensive margins of demand using a two-stage discrete-continuous choice model.
• To develop a tractable semiparametric estimation strategy flexible to the distribution of unobservable product characteristics.
• To demonstrate that ignoring consideration set selection—especially in the presence of zero shares—can produce misleading or even upward-sloping demand curves.

Proposed method

• Formalizes a Marshallian demand system derived from a budget-constrained CES utility maximization problem.
• Derives the same market share equation as Berry (1994) and Berry et al. (1995), establishing equivalence under non-zero shares.
• Introduces a two-stage model: first, a binary choice for whether to buy any product (consideration set), second, a continuous choice of quantity given purchase.
• Uses a quality kernel to model unobserved heterogeneity, allowing flexible semiparametric estimation of the distribution of unobservables.
• Employs instrumental variables and simulation-based estimation to handle endogeneity and unobserved product characteristics.
• Applies the framework to scanner data on cola sales, comparing results under different treatments of zero market shares.

Experimental results

Research questions

• RQ1 Can a CES-based demand system be estimated consistently when zero market shares are present in the data?
• RQ2 How does the consideration set selection process affect the identification and estimation of price and quality parameters in demand models?
• RQ3 What are the consequences of dropping or imputing zero market shares in standard logit-based demand estimation?
• RQ4 To what extent does the distribution of unobserved product characteristics affect estimation results, and can a semiparametric approach mitigate this?
• RQ5 Does the proposed framework prevent the counterintuitive result of upward-sloping demand curves that arise from ignoring zero shares?

Key findings

• Ignoring zero market shares—especially by dropping or imputing them—leads to upward-sloping demand curves, even when true price coefficients are negative.
• The proposed two-stage model with a quality kernel successfully separates intensive and extensive margins, providing a microfoundation for handling zero shares.
• Estimation results show that the correlation between propensity scores from Klein-Spady and Probit models is high (r ≈ 0.7), suggesting robustness of selection model estimates.
• Imputing zero shares with small positive values causes upward bias in price coefficient estimates, and the direction of bias is unpredictable.
• The semiparametric estimator performs well even when unobservables are non-Gaussian, indicating robustness to distributional misspecification.
• The framework allows direct application of Berry et al. (1995)’s identification and estimation strategy when zero shares are absent, while extending it to cases with zero shares through a clean exclusion restriction on the whether-to-buy decision.