[Paper Review] Contextual Constrained Learning for Dose-Finding Clinical Trials

Hyun-Suk Lee, Cong Shen+2 more|arXiv (Cornell University)|Jan 8, 2020

Advanced Bandit Algorithms Research22 references4 citations

TL;DR

This paper proposes C3T-Budget, a contextual multi-armed bandit algorithm for dose-finding in clinical trials under budget and safety constraints. It dynamically selects patients and doses by balancing treatment efficacy and estimation accuracy, using group-specific context, estimated efficacy, and credibility to prioritize recruitment—achieving lower dose error rates and higher efficacy per patient than state-of-the-art methods, especially under limited budgets.

ABSTRACT

Clinical trials in the medical domain are constrained by budgets. The number of patients that can be recruited is therefore limited. When a patient population is heterogeneous, this creates difficulties in learning subgroup specific responses to a particular drug and especially for a variety of dosages. In addition, patient recruitment can be difficult by the fact that clinical trials do not aim to provide a benefit to any given patient in the trial. In this paper, we propose C3T-Budget, a contextual constrained clinical trial algorithm for dose-finding under both budget and safety constraints. The algorithm aims to maximize drug efficacy within the clinical trial while also learning about the drug being tested. C3T-Budget recruits patients with consideration of the remaining budget, the remaining time, and the characteristics of each group, such as the population distribution, estimated expected efficacy, and estimation credibility. In addition, the algorithm aims to avoid unsafe dosages. These characteristics are further illustrated in a simulated clinical trial study, which corroborates the theoretical analysis and demonstrates an efficient budget usage as well as a balanced learning-treatment trade-off.

Motivation & Objective

• To address the challenge of dose-finding in heterogeneous patient populations under limited budgets and safety constraints.
• To balance the trade-off between clinical research (learning optimal doses) and clinical practice (treating patients effectively) in early-phase trials.
• To improve estimation accuracy for under-represented subgroups while avoiding unsafe dosages.
• To develop an algorithm that efficiently uses limited budgets by prioritizing recruitment based on estimation credibility and expected efficacy.
• To demonstrate that the algorithm outperforms existing methods in both dose accuracy and treatment efficacy under constrained settings.

Proposed method

• C3T-Budget models patient groups as contextual arms in a multi-armed bandit framework, with context including population distribution, estimated efficacy, and estimation credibility.
• For each group, the algorithm selects the dose with the highest estimated efficacy while satisfying safety constraints based on historical data.
• The algorithm decides whether to treat a patient based on the potential information gain, prioritizing groups with high uncertainty or low confidence in efficacy estimates.
• C3T-Budget-E is an extension that emphasizes recruiting from the most promising subgroup when budgets are extremely limited, sacrificing precision for less effective groups.
• The algorithm uses a constrained optimization approach that integrates budget, time horizon, and safety constraints into patient recruitment and dose allocation decisions.
• It employs a sequential decision-making process where patient arrival rates vary by group, and the agent adapts recruitment and dosing in real time.

Experimental results

Research questions

• RQ1 How can a dose-finding algorithm effectively balance learning and treatment in clinical trials with limited budgets and heterogeneous patient groups?
• RQ2 What is the impact of budget constraints on the accuracy of dose recommendations and patient efficacy in subgroup-specific trials?
• RQ3 How does the time horizon influence the performance of a contextual clinical trial algorithm under budget constraints?
• RQ4 Can a contextual bandit approach improve estimation accuracy for under-represented subgroups while avoiding unsafe dosages?
• RQ5 How does the algorithm’s recruitment strategy adapt to varying group sizes and arrival rates in a heterogeneous population?

Key findings

• C3T-Budget achieves the lowest dose error rate among all algorithms when the budget B ≥ 80, with near-zero error for B ≥ 500.
• For budgets B < 80, C3T-Budget-E achieves the lowest error rate, indicating superior performance in ultra-low-budget settings.
• C3T-Budget-E consistently achieves the highest efficacy per patient across all budget levels, demonstrating its strength in treatment-focused scenarios.
• As the budget increases, the dose error rate of C3T-Budget decreases drastically, showing effective budget utilization compared to baseline methods.
• When the time-horizon T/B ratio increases beyond 1, C3T-Budget’s error rate decreases due to greater flexibility in patient recruitment, while C3T-Budget-E’s error rate increases due to over-concentration on high-efficacy subgroups.
• Both algorithms saturate in performance at high T/B ratios, indicating that the limiting factor shifts from time to budget, with minimal gains from further increasing T/B.