[論文レビュー] ReLMXEL: Adaptive RL-Based Memory Controller with Explainable Energy and Latency Optimization

Panuganti Chirag Sai, Gandholi Sarat+3 more|arXiv (Cornell University)|Mar 18, 2026

Explainable Artificial Intelligence (XAI)被引用数 0

ひとこと要約

ReLMXEL は報酬分解を備えたマルチエージェント強化学習メモリコントローラを導入し、遅延とエネルギーの適応最適化を行い、決定の説明性も提供します。

ABSTRACT

Reducing latency and energy consumption is critical to improving the efficiency of memory systems in modern computing. This work introduces ReLMXEL (Reinforcement Learning for Memory Controller with Explainable Energy and Latency Optimization), a explainable multi-agent online reinforcement learning framework that dynamically optimizes memory controller parameters using reward decomposition. ReLMXEL operates within the memory controller, leveraging detailed memory behavior metrics to guide decision-making. Experimental evaluations across diverse workloads demonstrate consistent performance gains over baseline configurations, with refinements driven by workload-specific memory access behaviour. By incorporating explainability into the learning process, ReLMXEL not only enhances performance but also increases the transparency of control decisions, paving the way for more accountable and adaptive memory system designs.

研究の動機と目的

• Motivate reducing DRAM latency and energy in modern memory systems and the need for transparent, adaptable controllers.
• Propose ReLMXEL, an explainable multi-agent RL framework that tunes memory controller parameters online.
• Demonstrate performance gains across diverse workloads using DRAMSys simulations and explainability mechanisms.
• Show how reward decomposition enables interpretable decisions and identify potential future extensions in hardware-in-the-loop and security.

提案手法

• Model memory controller as a multi-agent RL environment where each agent tunes a configurable DRAM parameter via a Q-table.
• Decompose scalar RL rewards into per-metric components (energy, bandwidth, latency) and aggregate via a target-based reward function.
• Use SARSA with per-parameter Q-tables and an epsilon-greedy policy, with a warmup exploration period and trace-split feedback loops.
• Apply Minimal Sufficient Explanation (MSX) to justify action choices through reward component differences (RDX).
• Experiment with DDR4 DRAM, DRAMSys/DRAMPower, and traces from GEMM, STREAM, BFS, SPEC CPU 2017 workloads to evaluate metrics.
• Demonstrate adaptability to workload patterns and analyze trade-offs between energy, bandwidth, and latency.

実験結果

リサーチクエスチョン

• RQ1 Can an adaptive RL-based memory controller improve energy, bandwidth, and latency across diverse workloads?
• RQ2 Does reward decomposition enhance explainability without sacrificing performance?
• RQ3 How do workload characteristics influence optimal controller parameter settings and convergence?
• RQ4 What is the impact of explainability on trust and transparency of memory control decisions?

主な発見

Workload	Time Steps	Threshold w	Baseline Reward	ReLMXEL Reward	Average Energy (%)	Average Bandwidth (%)	Average Latency (%)
STREAM	20170	16000	15555.06	17597.07	3.84	8.39	0.23
GEMM	19468	17000	6572.88	7121.46	3.83	4.95	0.01
BFS	17995	14000	9673.14	10842.41	7.66	7.22	-0.03
fotonik_3d	20770	17000	4870.89	9165.52	7.66	2.90	0.07
xalancbmk	16494	14000	3092.9	3320.38	7.68	107.03	-0.02
gcc	17863	14000	9154.29	9556.25	7.66	1.70	-0.24
roms	17563	14000	8017.8	13554.84	7.67	35.63	0.08
mcf	17894	14000	6013.5	6075.53	7.67	40.19	-4.43
lbm	18473	15000	5496.77	14934.6	7.67	26.73	0.05
omnetpp	16682	14000	4743.99	6688.05	4.06	138.78	-0.09

• ReLMXEL consistently improves baseline metrics across workloads in energy, bandwidth, and often latency.
• Average energy and bandwidth gains are notable, with some workloads showing minimal latency impact.
• Explainability via reward decomposition and MSX provides interpretable rationales for actions (e.g., energy savings vs latency/bandwidth costs).
• The framework adapts controller parameters to workload patterns, achieving robust performance with a SARSA-based learning scheme.
• Experiments use DDR4 with realistic traces (GEMM, STREAM, BFS, SPEC 2017) and show practical viability for adaptive, explainable memory control.