[Paper Review] Dual Instrumental Variable Regression

Krikamol Muandet, Arash Mehrjou+2 more|arXiv (Cornell University)|Oct 27, 2019

Statistical Methods and Inference49 references16 citations

TL;DR

This paper introduces DualIV, a novel non-linear instrumental variable regression method that reformulates two-stage IV estimation as a convex-concave saddle-point problem, enabling direct estimation of the structural causal function without first-stage regression. The approach uses kernel methods in reproducing kernel Hilbert spaces and achieves consistent estimation with theoretical guarantees and competitive empirical performance.

ABSTRACT

We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation. Inspired by problems in stochastic programming, we show that two-stage procedures for non-linear IV regression can be reformulated as a convex-concave saddle-point problem. Our formulation enables us to circumvent the first-stage regression which is a potential bottleneck in real-world applications. We develop a simple kernel-based algorithm with an analytic solution based on this formulation. Empirical results show that we are competitive to existing, more complicated algorithms for non-linear instrumental variable regression.

Motivation & Objective

• To address the limitations of traditional two-stage IV methods in non-linear settings, particularly the sensitivity to first-stage regression errors.
• To develop a consistent, non-linear IV estimation method that avoids the computational and statistical bottlenecks of first-stage regression.
• To provide a theoretically grounded, convex-concave optimization framework for non-linear IV regression inspired by stochastic programming.
• To enable practical, scalable estimation via kernel-based methods with analytic solutions in RKHS.

Proposed method

• Reformulate non-linear IV regression as a convex-concave saddle-point problem by leveraging duality, avoiding the need for first-stage regression.
• Use a dual formulation that directly estimates the structural causal function $ f $ via optimization over function spaces.
• Employ reproducing kernel Hilbert spaces (RKHS) to represent the unknown structural function and conditional expectations.
• Derive an analytic solution using kernelized covariance and cross-covariance operators, minimizing a regularized empirical risk.
• Apply Tikhonov regularization with two hyperparameters $ \lambda_1 $ and $ \lambda_2 $ to ensure stability and convergence.
• Establish consistency of the estimator in RKHS norm under regularity conditions, with convergence rates dependent on $ \lambda_1, \lambda_2 $, and sample size $ n $.

Experimental results

Research questions

• RQ1 Can a dual formulation of two-stage IV regression be derived that avoids first-stage regression and enables direct estimation of the structural function?
• RQ2 How can the duality between the structural and reduced-form models be exploited to construct a convex-concave optimization problem?
• RQ3 What conditions ensure the consistency of the resulting estimator in RKHS under non-linear and non-parametric settings?
• RQ4 How does the performance of the proposed DualIV method compare to existing non-linear IV methods in terms of estimation accuracy and robustness?
• RQ5 What is the theoretical convergence rate of the kernel-based DualIV estimator, and how do hyperparameters affect its consistency?

Key findings

• The DualIV method achieves consistent estimation of the structural causal function $ f $ in the RKHS norm under appropriate regularization and sample size conditions.
• The estimator converges to the true function at a rate of $ \mathcal{O}(1/\lambda_1^2\lambda_2^2\sqrt{n}) $, provided $ \lambda_1, \lambda_2 \to 0 $ and $ 1/(\lambda_1^2\lambda_2^2\sqrt{n}) \to 0 $.
• Empirical results show that DualIV performs competitively with more complex existing algorithms for non-linear IV regression.
• The method avoids the first-stage regression bottleneck by directly solving a dual optimization problem, improving robustness and scalability.
• Theoretical analysis confirms that the estimator is consistent, with convergence established through bounds on operator norms and $ \sqrt{n} $-consistency of empirical covariance operators.
• The kernel-based formulation allows for analytic solutions and enables non-parametric estimation without assuming linearity in the treatment-outcome relationship.