[Paper Review] Persuading to Prepare for Quitting Smoking with a Virtual Coach: Using States and User Characteristics to Predict Behavior

Nele Albers, Mark A. Neerincx+1 more|arXiv (Cornell University)|Apr 5, 2023

Behavioral Health and InterventionsPsychology3 citations

TL;DR

This study evaluates the impact of user states and characteristics on predicting behavior in a virtual coach for smoking cessation using Reinforcement Learning (RL). It finds that states significantly improve behavior and future state prediction, while user characteristics like involvement only help when combined with states, supporting their integration in personalized eHealth persuasion algorithms.

ABSTRACT

Despite their prevalence in eHealth applications for behavior change, persuasive messages tend to have small effects on behavior. Conditions or states (e.g., confidence, knowledge, motivation) and characteristics (e.g., gender, age, personality) of persuadees are two promising components for more effective algorithms for choosing persuasive messages. However, it is not yet sufficiently clear how well considering these components allows one to predict behavior after persuasive attempts, especially in the long run. Since collecting data for many algorithm components is costly and places a burden on users, a better understanding of the impact of individual components in practice is welcome. This can help to make an informed decision on which components to use. We thus conducted a longitudinal study in which a virtual coach persuaded 671 daily smokers to do preparatory activities for quitting smoking and becoming more physically active, such as envisioning one's desired future self. Based on the collected data, we designed a Reinforcement Learning (RL)-approach that considers current and future states to maximize the effort people spend on their activities. Using this RL-approach, we found, based on leave-one-out cross-validation, that considering states helps to predict both behavior and future states. User characteristics and especially involvement in the activities, on the other hand, only help to predict behavior if used in combination with states rather than alone. We see these results as supporting the use of states and involvement in persuasion algorithms. Our dataset is available online.

Motivation & Objective

• To understand the individual impact of algorithm components—user states and characteristics—on predicting behavior after persuasive interventions in eHealth applications.
• To reduce data collection burden by identifying which components are most effective for behavior prediction, minimizing user effort and improving cost-efficiency.
• To evaluate whether states (e.g., confidence, motivation) or user characteristics (e.g., age, gender, involvement) are more effective predictors of behavior in longitudinal behavior change interventions.
• To assess whether combining states and characteristics improves prediction accuracy beyond using either alone.
• To support the design of more effective, user-friendly persuasion algorithms by identifying high-impact components for inclusion in eHealth systems.

Proposed method

• Conducted a longitudinal study with 671 daily smokers using a virtual coach to encourage preparatory activities for smoking cessation and increased physical activity.
• Collected data on user states (via COM-B self-evaluation questionnaire) and user characteristics (e.g., age, gender, personality, involvement) at multiple time points.
• Applied a Reinforcement Learning (RL) framework that models both current and future states to maximize user effort on preparatory activities.
• Used leave-one-out cross-validation to evaluate predictive performance of models using states, characteristics, or both.
• Trained RL models to learn transition probabilities and expected effort based on state-action pairs, with state features derived from behavior change theory (COM-B).
• Evaluated model performance using self-reported effort as the behavioral outcome, assessing prediction accuracy for both behavior and future states.

Figure 1. Left axis: Mean $L_{1}$ -error with 95% CIs for predicting rewards based on 1) the mean reward per action and 2) the mean reward per action and state. Right axis: Mean reward overall and per state.

Experimental results

Research questions

• RQ1 How well do user states predict subsequent behavior and future states in a longitudinal eHealth intervention?
• RQ2 How well do user characteristics (e.g., age, gender, personality, involvement) predict behavior when used in isolation?
• RQ3 Does combining user states and characteristics improve prediction accuracy compared to using either component alone?
• RQ4 What is the relative contribution of states versus characteristics in predicting behavior change over time?
• RQ5 Can RL models that incorporate both current and future states effectively predict user engagement in preparatory health activities?

Key findings

• Considering user states significantly improves the prediction of both current behavior and future states in the virtual coach intervention.
• User characteristics alone, including involvement, do not reliably predict behavior unless combined with state information.
• The combination of states and involvement in the activities yields better predictive performance than using either component in isolation.
• The RL-based approach successfully modeled dynamic transitions between states and actions, demonstrating the value of considering future states in persuasion strategies.
• The study's dataset is publicly available, supporting reproducibility and further research in personalized eHealth interventions.
• Findings suggest that states are more effective than characteristics alone for modeling behavior change, supporting their integration into RL-driven persuasion algorithms.

Figure 2. Comparison of three approaches to predicting next states with regards to the mean likelihood of next states with 95% CIs for each state.