[Paper Review] When Machine Learning Meets Congestion Control: A Survey and Comparison

Huiling Jiang, Qing Li+5 more|arXiv (Cornell University)|Oct 22, 2020

Network Traffic and Congestion Control90 references4 citations

TL;DR

This paper surveys and compares machine learning-based congestion control (CC) algorithms, focusing on reinforcement learning (RL) as a key approach. It evaluates RL-based CC against traditional rule-based methods, demonstrating superior performance in high-bandwidth, low-delay scenarios, while identifying challenges like convergence, overfitting, and engineering feasibility for real-world deployment.

ABSTRACT

Machine learning (ML) has seen a significant surge and uptake across many diverse applications. The high flexibility, adaptability and computing capabilities it provides extends traditional approaches used in multiple fields including network operation and management. Numerous surveys have explored ML in the context of networking, such as traffic engineering, performance optimization and network security. Many ML approaches focus on clustering, classification, regression and reinforcement learning (RL). The innovation of this research and contribution of this paper lies in the detailed summary and comparison of learning-based congestion control (CC) approaches. Compared with traditional CC algorithms which are typically rule-based, capabilities to learn from historical experience are highly desirable. From the literature, it is observed that RL is a crucial trend among learning-based CC algorithms. In this paper, we explore the performance of RL-based CC algorithms and present current problems with RL-based CC algorithms. We outline challenges and trends related to learning-based CC algorithms.

Motivation & Objective

• To analyze the state of the art in learning-based congestion control (CC) algorithms, particularly those using reinforcement learning (RL).
• To compare RL-based CC algorithms with traditional rule-based CC mechanisms in terms of performance, adaptability, and robustness.
• To identify key challenges in deploying RL-based CC in real-world networks, including convergence, overfitting, and computational complexity.
• To outline future research trends, such as lightweight models and realistic simulation platforms, for scalable and efficient CC solutions.
• To provide a comprehensive overview of current limitations and opportunities in ML-driven network congestion management.

Proposed method

• Conducts a systematic survey of learning-based CC approaches, with a focus on reinforcement learning (RL) as the dominant paradigm.
• Reviews traditional CC mechanisms—loss-based, delay-based, and hybrid—highlighting their limitations in dynamic, complex networks.
• Analyzes RL-based CC algorithms through simulation-based evaluation across diverse network scenarios, including data centers, 5G, and satellite networks.
• Identifies engineering challenges such as hard convergence, overfitting, and high computational demands in RL-based CC systems.
• Proposes the need for lightweight, domain-knowledge-informed models to reduce complexity and improve deployability.
• Advocates for the development of open, realistic simulation platforms (e.g., Pantheon) to support evaluation and training of next-generation CC algorithms.

Experimental results

Research questions

• RQ1 How do RL-based congestion control algorithms compare to traditional rule-based CC algorithms in terms of throughput, fairness, and delay in diverse network environments?
• RQ2 What are the primary challenges hindering the real-world deployment of RL-based congestion control systems?
• RQ3 How can lightweight and efficient learning-based models be designed to reduce computational overhead and improve scalability?
• RQ4 What role do realistic simulation platforms play in advancing the development and evaluation of learning-based CC algorithms?
• RQ5 What future trends and research directions are most promising for improving the robustness and adaptability of ML-driven congestion control?

Key findings

• RL-based congestion control algorithms outperform traditional CC algorithms in high-bandwidth, low-delay network scenarios, achieving higher throughput and better fairness.
• Despite strong simulation results, RL-based CC faces significant challenges in real-world deployment, including hard convergence and overfitting to specific network topologies.
• Current RL-based CC algorithms suffer from high computational complexity and memory demands, limiting their practicality in resource-constrained environments.
• There is a critical need for lightweight models that integrate domain knowledge to reduce parameter count and improve training efficiency.
• Existing simulation platforms like Pantheon are insufficient for modeling complex, dynamic network environments such as flexible ad hoc wireless networks.
• Future research must prioritize engineering feasibility, including parameter selection and computational efficiency, to enable real-world adoption of learning-based CC solutions.