[Paper Review] Causal Bandits without Graph Learning

Mikhail Konobeev, Jalal Etesami+1 more|arXiv (Cornell University)|Jan 26, 2023

Advanced Bandit Algorithms ResearchDecision Sciences3 citations

TL;DR

This paper proposes RAndomized Parent Search (raps), an algorithm for causal bandits that discovers the parent nodes of the reward variable without prior knowledge of the causal graph. It achieves optimal intervention complexity by using atomic interventions and derives exact expected intervention counts, proving optimality via a universal lower bound, with sublinear scaling under general conditions and logarithmic scaling under specific graphical assumptions.

ABSTRACT

We study the causal bandit problem when the causal graph is unknown and develop an efficient algorithm for finding the parent node of the reward node using atomic interventions. We derive the exact equation for the expected number of interventions performed by the algorithm and show that under certain graphical conditions it could perform either logarithmically fast or, under more general assumptions, slower but still sublinearly in the number of variables. We formally show that our algorithm is optimal as it meets the universal lower bound we establish for any algorithm that performs atomic interventions. Finally, we extend our algorithm to the case when the reward node has multiple parents. Using this algorithm together with a standard algorithm from bandit literature leads to improved regret bounds.

Motivation & Objective

• To address the limitation in causal bandit literature where the causal graph is assumed known upfront.
• To develop an efficient algorithm that discovers the parent nodes of the reward variable using only atomic interventions, without requiring knowledge of the causal graph or its essential graph.
• To derive exact expected intervention counts for the algorithm on any DAG and establish theoretical optimality.
• To extend the method to cases with multiple reward parents and improve regret bounds using standard bandit algorithms.

Proposed method

• The raps algorithm performs randomized atomic interventions to identify the parent nodes of the reward variable by testing whether intervened nodes are ancestors of the reward node.
• It uses a statistical test based on intervention outcomes to determine if a node is an ancestor of the reward node, assuming perfect information from each intervention.
• The expected number of interventions is derived exactly using a closed-form equation based on the graph structure and the probability of correctly identifying parent nodes.
• The algorithm is extended to handle multiple parents by applying the same principle iteratively, with theoretical guarantees on intervention complexity.
• The method combines raps with the Upper Confidence Bound (UCB) bandit algorithm to improve regret bounds in the causal bandit setting.
• Theoretical analysis proves that raps meets the universal lower bound on intervention complexity, establishing its optimality.

Experimental results

Research questions

• RQ1 Can we design an algorithm that discovers the parent nodes of the reward variable in causal bandits without prior knowledge of the causal graph?
• RQ2 What is the exact expected number of atomic interventions required by such an algorithm on any DAG?
• RQ3 Under what graphical conditions does the intervention complexity scale logarithmically or sublinearly with the number of variables?
• RQ4 Is the proposed algorithm optimal in terms of intervention complexity, and how does it compare to the universal lower bound?
• RQ5 How can the algorithm be extended to handle multiple parent nodes of the reward variable, and what regret improvements can be achieved?

Key findings

• The raps algorithm achieves optimal intervention complexity by matching the universal lower bound derived in the paper, proving it is information-theoretically optimal.
• For Erdős-Rényi random DAGs with edge probability $ p = 1 - \left(\frac{0.5}{\log_2 n - 1}\right)^{1/(\log_2 n - 1)} $, the expected number of interventions scales as $ \mathcal{O}(\log n) $.
• For Erdős-Rényi graphs with $ p = \frac{\log n}{n} $, the expected number of interventions scales as $ \frac{n}{\log n} $, which is sublinear but slower than logarithmic.
• When the reward node has $ |\mathcal{P}| $ parents, the intervention complexity grows as $ (|\mathcal{P}| + 1)\log n $, confirming the scalability of the method.
• Empirical results show that the expected intervention count predicted by the theoretical equation matches experimental averages across 20 runs on Erdős-Rényi graphs with $ n = 1000 $.
• The raps + UCB combination achieves improved regret bounds compared to UCB alone, though with higher variance due to dependency on the budget and problem parameters.