[Paper Review] The Knowledge Gradient Policy Using A Sparse Additive Belief Model

Yan Li, Han Liu+1 more|arXiv (Cornell University)|Mar 18, 2015

Statistical Methods and Inference34 references3 citations

TL;DR

This paper proposes KGSpLin and KGSpAM, a knowledge gradient policy for high-dimensional, sparse additive models in sequential decision-making under uncertainty. By combining group Lasso with Bayesian inference and B-spline basis expansion, the method efficiently identifies relevant features and selects the optimal alternative under a finite budget, achieving optimal estimation error rates and outperforming linear model baselines in controlled experiments.

ABSTRACT

We propose a sequential learning policy for noisy discrete global optimization and ranking and selection (R\&S) problems with high dimensional sparse belief functions, where there are hundreds or even thousands of features, but only a small portion of these features contain explanatory power. We aim to identify the sparsity pattern and select the best alternative before the finite budget is exhausted. We derive a knowledge gradient policy for sparse linear models (KGSpLin) with group Lasso penalty. This policy is a unique and novel hybrid of Bayesian R\&S with frequentist learning. Particularly, our method naturally combines B-spline basis expansion and generalizes to the nonparametric additive model (KGSpAM) and functional ANOVA model. Theoretically, we provide the estimation error bounds of the posterior mean estimate and the functional estimate. Controlled experiments show that the algorithm efficiently learns the correct set of nonzero parameters even when the model is imbedded with hundreds of dummy parameters. Also it outperforms the knowledge gradient for a linear model.

Motivation & Objective

• To address sequential decision-making in high-dimensional, sparse belief models where only a few features are relevant.
• To develop a policy that identifies both the best alternative and the underlying sparsity structure under a finite budget.
• To generalize knowledge gradient from linear models to nonparametric additive models using B-spline approximation.
• To provide theoretical error bounds for posterior mean and functional estimates in sparse settings.
• To empirically validate the method’s ability to recover true sparsity patterns despite hundreds of irrelevant features.

Proposed method

• Proposes a knowledge gradient policy for sparse linear models (KGSpLin) using ℓ1,∞ group Lasso to promote group sparsity in coefficient estimation.
• Employs B-spline basis expansion to represent unknown smooth functions in additive models, enabling nonparametric estimation.
• Introduces a Beta-Bernoulli conjugate prior over indicator variables to model uncertainty about which features are active.
• Generalizes KGSpLin to nonparametric sparse additive models (KGSpAM) and functional ANOVA models via tensor-product B-splines.
• Uses a homotopy algorithm for recursive group Lasso optimization to maintain computational efficiency during sequential learning.
• Derives theoretical estimation error bounds for both posterior mean and functional estimates under sparsity assumptions.

Experimental results

Research questions

• RQ1 Can a knowledge gradient policy effectively identify the best alternative in high-dimensional, sparse optimization problems with noisy measurements?
• RQ2 How well can the method recover the true sparsity pattern when hundreds of features are present, most of which are irrelevant?
• RQ3 What theoretical guarantees can be provided for the estimation error of the posterior mean and functional estimates in sparse additive models?
• RQ4 Does the hybrid Bayesian-frequentist approach using group Lasso improve performance over standard knowledge gradient for linear models?
• RQ5 How does the method scale and perform in controlled experiments with high-dimensional, sparse signal structures?

Key findings

• The method achieves an estimation error bound of O(σε² s* d̄² log p / n) for the posterior mean estimate, where s* is the true sparsity and d̄ is the average number of basis functions per feature.
• The functional estimate error is bounded by O(σε² log p / n²/³), demonstrating convergence under optimal sparsity and basis choice.
• Controlled experiments show the algorithm successfully identifies the correct set of nonzero parameters even when embedded with hundreds of dummy features.
• KGSpAM outperforms standard knowledge gradient for linear models in both selection accuracy and sparsity recovery.
• Theoretical analysis confirms that the posterior covariance matrix shrinks at a rate proportional to 1/n², supporting consistent estimation.
• With optimal basis dimension d̄ = O(n¹/⁶), the method achieves a minimax-optimal convergence rate of O(σε² log p / n²/³) for functional estimation.