[Paper Review] Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey

Alaa Awad Abdellatif, Naram Mhaisen+4 more|arXiv (Cornell University)|Aug 5, 2021

Digital Mental Health Interventions18 citations

TL;DR

This paper presents a comprehensive survey of reinforcement learning (RL) applications in intelligent healthcare (I-health) systems, focusing on edge intelligence, smart core networks, and dynamic treatment regimes. It integrates RL, deep RL, and multi-agent RL to optimize real-time decision-making, reduce latency, and improve scalability in chronic disease management and critical care, offering a unified framework for next-generation healthcare systems.

ABSTRACT

The rapid increase in the percentage of chronic disease patients along with the recent pandemic pose immediate threats on healthcare expenditure and elevate causes of death. This calls for transforming healthcare systems away from one-on-one patient treatment into intelligent health systems, to improve services, access and scalability, while reducing costs. Reinforcement Learning (RL) has witnessed an intrinsic breakthrough in solving a variety of complex problems for diverse applications and services. Thus, we conduct in this paper a comprehensive survey of the recent models and techniques of RL that have been developed/used for supporting Intelligent-healthcare (I-health) systems. This paper can guide the readers to deeply understand the state-of-the-art regarding the use of RL in the context of I-health. Specifically, we first present an overview for the I-health systems challenges, architecture, and how RL can benefit these systems. We then review the background and mathematical modeling of different RL, Deep RL (DRL), and multi-agent RL models. After that, we provide a deep literature review for the applications of RL in I-health systems. In particular, three main areas have been tackled, i.e., edge intelligence, smart core network, and dynamic treatment regimes. Finally, we highlight emerging challenges and outline future research directions in driving the future success of RL in I-health systems, which opens the door for exploring some interesting and unsolved problems.

Motivation & Objective

• Address the growing burden of chronic diseases and aging populations by transforming traditional healthcare into intelligent, scalable systems.
• Identify key challenges in I-health systems, including real-time data processing, low-latency communication, and dynamic treatment personalization.
• Survey and categorize recent RL, deep RL, and multi-agent RL models for healthcare applications.
• Propose a unified architecture integrating RL with edge and core networks to enhance decision-making and system efficiency.
• Highlight open challenges and future research directions for RL in I-health, especially in communication efficiency and critical care applications.

Proposed method

• Conduct a systematic literature review of RL applications in I-health across three domains: edge intelligence, smart core networks, and dynamic treatment regimes.
• Model I-health systems using hierarchical layers: services, requirements, enabling technologies, and RL methodologies (e.g., Q-learning, DQN, MARL).
• Integrate RL into edge and cloud computing infrastructures to enable real-time, low-latency decision-making for remote monitoring and emergency response.
• Apply deep reinforcement learning (DRL) to handle high-dimensional state spaces from IoT and medical sensor data.
• Utilize multi-agent RL (MARL) to coordinate distributed agents in dynamic, decentralized healthcare environments with communication constraints.
• Propose communication-efficient MARL designs to reduce overhead while maintaining performance in time-varying wireless networks.

Experimental results

Research questions

• RQ1 How can reinforcement learning optimize decision-making in intelligent healthcare systems under dynamic and uncertain conditions?
• RQ2 What are the key challenges in applying RL to edge-based and core-network healthcare systems, and how can they be mitigated?
• RQ3 How can RL-based dynamic treatment regimes personalize care while ensuring safety and adaptability?
• RQ4 What are the trade-offs between communication efficiency and policy performance in multi-agent RL for healthcare?
• RQ5 What emerging challenges remain in deploying RL for mission-critical healthcare applications like remote surgery and critical care?

Key findings

• RL-based solutions significantly improve real-time decision-making in I-health systems, particularly in remote monitoring and dynamic treatment personalization.
• Deep reinforcement learning enables effective handling of high-dimensional medical and sensor data from IoT devices in edge computing environments.
• Multi-agent RL shows promise in coordinating distributed healthcare agents, though communication efficiency remains a major challenge.
• Communication-efficient MARL designs reduce overhead while maintaining performance, especially in time-varying and bandwidth-constrained wireless networks.
• RL can optimize blockchain configurations in I-health systems by balancing latency, storage, and cost, enabling secure and scalable data sharing.
• Critical care applications such as remote surgery benefit from RL’s ability to minimize latency and ensure ultra-reliable data transmission in MEC environments.