[Paper Review] Micro-Randomized Trials in mHealth

Peng Liao, Predrag Klasnja+2 more|arXiv (Cornell University)|Apr 1, 2015

Mobile Health and mHealth Applications27 references17 citations

TL;DR

This paper proposes micro-randomized trials (MRTs) as an experimental design for testing the proximal effects of just-in-time mobile health (mHealth) interventions, where treatments are sequentially randomized across hundreds to thousands of decision points per participant. The method enables estimation of treatment effects in real time and includes a sample size calculator and test statistic for statistical inference, validated through simulations and applied to the HeartSteps physical activity app.

ABSTRACT

The use and development of mobile interventions is experiencing rapid growth. In mobile interventions, treatments are provided via a mobile device that are intended to help an individual make healthy decisions in the moment, and thus have a proximal, near future impact. Currently the development of mobile interventions is proceeding at a much faster pace than that of associated data science methods. A first step toward developing data-based methods is to provide an experimental design for use in testing the proximal effects of these just-in-time treatments. In this paper, we propose a trial design for this purpose. In a micro-randomized trial, treatments are sequentially randomized throughout the conduct of the study, with the result that each participant may be randomized at the 100s or 1000s of occasions at which a treatment might be provided. Further, we develop a test statistic for assessing the proximal effect of a treatment as well as an associated sample size calculator. We conduct simulation evaluations of the sample size calculator in various settings. Rules of thumb that might be used in designing the micro-randomized trial are discussed. This work is motivated by our collaboration on the HeartSteps mobile application designed to increase physical activity.

Motivation & Objective

• To address the growing gap between rapid mHealth intervention development and the lack of robust data science methods for evaluating their proximal effects.
• To develop an experimental design that allows for testing the immediate impact of mobile health treatments in real-world, dynamic contexts.
• To provide a statistical framework—including a test statistic and sample size calculator—for analyzing treatment effects in micro-randomized trials.
• To guide researchers in designing MRTs through practical rules of thumb and simulation-based evaluation.

Proposed method

• Treatments are sequentially randomized at numerous decision points (e.g., 100s to 1000s per participant) throughout the study duration, enabling assessment of immediate treatment effects.
• Each randomization is independent and occurs based on real-time context, such as current activity level or location, to test the effectiveness of just-in-time interventions.
• A test statistic is developed to assess the proximal effect of a treatment on the outcome at the time of delivery.
• A sample size calculator is proposed to ensure adequate statistical power for detecting meaningful treatment effects in MRTs.
• Simulation studies are conducted to evaluate the performance of the sample size calculator under various settings, including different effect sizes and variability levels.
• Rules of thumb for trial design are derived from simulation results to guide practical implementation.

Experimental results

Research questions

• RQ1 What is the optimal design for testing the proximal effects of mobile health interventions delivered in real time?
• RQ2 How can statistical power be ensured in micro-randomized trials with high-frequency treatment decisions?
• RQ3 What sample size is required to detect meaningful treatment effects in MRTs under realistic conditions?
• RQ4 How do varying levels of context-dependent variability and treatment effect size influence the performance of the sample size calculator?
• RQ5 What practical guidelines can be derived for designing effective micro-randomized trials in mHealth research?

Key findings

• The proposed sample size calculator performs well in simulations across diverse settings, maintaining appropriate Type I error rates and statistical power.
• The test statistic effectively detects proximal treatment effects even when effects are small or context-dependent.
• Simulation results show that higher frequency of randomizations increases statistical power, supporting the use of dense decision points in MRTs.
• Rules of thumb derived from simulations provide practical guidance for researchers in determining the number of randomizations and required sample size.
• The method successfully supports causal inference of treatment effects in real-time mHealth interventions, as demonstrated in the HeartSteps physical activity application.
• The framework enables data-driven optimization of just-in-time interventions by isolating the immediate impact of each treatment delivery.