[Paper Review] Reinforcement Learning-Based Adaptive Insulin Advisor for Individuals with Type 1 Diabetes Patients under Multiple Daily Injections Therapy

Qingnan Sun, Marko V. Jankovic+1 more|arXiv (Cornell University)|Jun 7, 2019

Diabetes Management and Research15 references4 citations

TL;DR

This paper proposes a reinforcement learning-based adaptive insulin advisor for type 1 diabetes patients using multiple daily injections (MDI), integrating self-monitoring of blood glucose (SMBG) and insulin pens. The system improves glycemic control by increasing time in range and reducing hypoglycemic and hyperglycemic events compared to conventional MDI therapy, validated via in silico simulations using the DMMS.R simulator.

ABSTRACT

The existing adaptive basal-bolus advisor (ABBA) was further developed to benefit patients under insulin therapy with multiple daily injections (MDI). Three different in silico experiments were conducted with the DMMS.R simulator to validate the approach of combined use of self-monitoring of blood glucose (SMBG) and insulin injection devices, e.g. insulin pen, as are used by the majority of type 1 diabetes patients under insulin therapy. The proposed approach outperforms the conventional method, as it increases the time spent within the target range and simultaneously reduces the risks of hyperglycaemic and hypoglycaemic events.

Motivation & Objective

• To develop an adaptive insulin advisor for type 1 diabetes patients using multiple daily injections (MDI) and self-monitoring of blood glucose (SMBG).
• To enhance glycemic control by reducing time spent in hyperglycemic and hypoglycemic ranges.
• To validate the proposed RL-based insulin advisor using in silico simulations with the DMMS.R simulator.
• To demonstrate superiority over conventional MDI therapy in maintaining blood glucose within target range.

Proposed method

• The system employs a reinforcement learning (RL) agent trained to recommend insulin doses based on real-time SMBG data and insulin pen usage.
• The RL agent learns optimal insulin dosing strategies through trial-and-error interactions within a simulated environment.
• The DMMS.R simulator models physiological responses of type 1 diabetes patients to insulin and meal intake.
• The system integrates SMBG readings and insulin injection data from insulin pens to inform real-time dosing decisions.
• The agent is trained to maximize time in the target glucose range while minimizing hypoglycemic and hyperglycemic events.
• The approach is validated across three distinct in silico experiments under varying meal and insulin conditions.

Experimental results

Research questions

• RQ1 Can a reinforcement learning-based insulin advisor improve glycemic control in type 1 diabetes patients using MDI therapy?
• RQ2 How does the RL-based advisor compare to conventional MDI therapy in terms of time in range and hypoglycemic/hyperglycemic risk?
• RQ3 To what extent can the integration of SMBG and insulin pen data enhance insulin dosing accuracy in MDI patients?
• RQ4 Does the RL agent maintain robust performance across diverse meal and insulin bolus scenarios in simulation?

Key findings

• The proposed RL-based insulin advisor significantly increased the time spent in the target glucose range compared to conventional MDI therapy.
• The system reduced the frequency and duration of hypoglycemic events, improving safety for patients.
• The system also reduced the incidence of hyperglycemic episodes, indicating better postprandial glucose control.
• The three in silico experiments confirmed consistent performance improvements across varied meal and insulin bolus conditions.