[Paper Review] On the Nuisance of Control Variables in Regression Analysis

Paul Hünermund, Beyers Louw|arXiv (Cornell University)|May 20, 2020

Advanced Causal Inference Techniques17 references21 citations

TL;DR

This paper argues that control variables in multivariate regression are often endogenous and their estimated coefficients lack causal interpretation due to confounding mechanisms. The authors recommend omitting or relegating control variable coefficients to appendices, focusing instead on causal identification of main treatment effects to prevent misleading interpretations in management and organizational research.

ABSTRACT

Control variables are included in regression analyses to estimate the causal effect of a treatment on an outcome. In this paper, we argue that the estimated effect sizes of controls are unlikely to have a causal interpretation themselves, though. This is because even valid controls are possibly endogenous and represent a combination of several different causal mechanisms operating jointly on the outcome, which is hard to interpret theoretically. Therefore, we recommend refraining from interpreting marginal effects of controls and focusing on the main variables of interest, for which a plausible identification argument can be established. To prevent erroneous managerial or policy implications, coefficients of control variables should be clearly marked as not having a causal interpretation or omitted from regression tables altogether. Moreover, we advise against using control variable estimates for subsequent theory building and meta-analyses.

Motivation & Objective

• To challenge the widespread practice of interpreting control variable coefficients as having causal meaning in regression analyses.
• To highlight that even valid controls may be endogenous and represent combined causal mechanisms, making their marginal effects uninterpretable.
• To address the risk of erroneous managerial or policy implications arising from overinterpreting control variable estimates.
• To advocate for treating control variables as nuisance parameters rather than substantive findings in empirical research.
• To discourage the use of control variable estimates in theory building and meta-analyses due to their lack of causal clarity.

Proposed method

• Adopting the potential outcomes framework and causal diagrams (d-separation) to assess which variables should be controlled for in regression models.
• Using graphical causal models to distinguish between valid controls and bad controls, emphasizing that control variables may still be endogenous.
• Applying the backdoor criterion to identify adjustment sets that block backdoor paths, ensuring consistent estimation of treatment effects.
• Recommending that researchers focus on identifying and justifying main treatment variables with plausible identification arguments, rather than relying on control variable signs or significance.
• Proposing that control variable coefficients be omitted from main regression tables or moved to appendices, with clear labeling in table notes.
• Critiquing the use of R² improvement and incremental F-tests for control variable inclusion, advocating instead for theory-driven model specification.

Experimental results

Research questions

• RQ1 Why do control variable coefficients in regression models often lack a valid causal interpretation despite being included to improve estimation?
• RQ2 What are the implications of interpreting control variable coefficients as causal when they may be endogenous or confounded by unobserved factors?
• RQ3 How should researchers report control variable results to avoid misleading conclusions in empirical management and organizational research?
• RQ4 To what extent should control variable estimates be used in meta-analyses or theory development, given their uncertain causal status?
• RQ5 What is the appropriate role of control variables in regression models when their coefficients are likely biased or uninterpretable?

Key findings

• 47% of regression-based manuscripts in Organization Science and Strategic Management Journal (2015–2020) explicitly discussed the estimated effect sizes of control variables, indicating widespread interpretive practice.
• Control variable coefficients are unlikely to have a causal interpretation because they may reflect a combination of multiple unobserved causal mechanisms operating jointly on the outcome.
• Even when controls are valid, they can be endogenous due to unobserved confounders, undermining their interpretability in isolation.
• Coefficients of control variables can vary substantially across different adjustment sets, as shown in Table 1 (columns 6–8), indicating instability and lack of unique causal meaning.
• Meta-analyses should be restricted to main treatment variables with plausible identification arguments, as control variable estimates add little to cumulative evidence due to their ambiguous causal status.
• Omitting or relegating control variable coefficients to appendices prevents misleading interpretations and streamlines manuscript presentation without loss of methodological rigor.