[Paper Review] Explicit-Blurred Memory Network for Analyzing Patient Electronic Health Records

Prithwish Chakraborty, Fei Wang+2 more|arXiv (Cornell University)|Nov 15, 2019

Machine Learning in Healthcare15 references4 citations

TL;DR

This paper proposes EBmRNN, a novel memory network for electronic health record (EHR) analysis that partitions external memory into explicit and blurred components: the former stores exact recent events for interpretability, the latter accumulates abstracted past features to stabilize training. The model achieves state-of-the-art performance on ICU prediction tasks while enabling clear interpretation of model decisions through traceable memory access.

ABSTRACT

In recent years, we have witnessed an increased interest in temporal modeling of patient records from large scale Electronic Health Records (EHR). While simpler RNN models have been used for such problems, memory networks, which in other domains were found to generalize well, are underutilized. Traditional memory networks involve diffused and non-linear operations where influence of past events on outputs are not readily quantifiable. We posit that this lack of interpretability makes such networks not applicable for EHR analysis. While networks with explicit memory have been proposed recently, the discontinuities imposed by the discrete operations make such networks harder to train and require more supervision. The problem is further exacerbated in the limited data setting of EHR studies. In this paper, we propose a novel memory architecture that is more interpretable than traditional memory networks while being easier to train than explicit memory banks. Inspired by well-known models of human cognition, we propose partitioning the external memory space into (a) a primary explicit memory block to store exact replicas of recent events to support interpretations, followed by (b) a secondary blurred memory block that accumulates salient aspects of past events dropped from the explicit block as higher level abstractions and allow training with less supervision by stabilize the gradients. We apply the model for 3 learning problems on ICU records from the MIMIC III database spanning millions of data points. Our model performs comparably to the state-of the art while also, crucially, enabling ready interpretation of the results.

Motivation & Objective

• Address the lack of interpretability in traditional memory networks used for EHR analysis, where past event influence is diffused and non-transparent.
• Overcome the training instability and high supervision needs of explicit memory-only models by introducing a secondary blurred memory for gradient stability.
• Enable model interpretability through exact memory slot tracking while maintaining generalization and performance comparable to state-of-the-art architectures.
• Balance interpretability and trainability in low-data EHR settings by emulating human memory structures (sensory, explicit, long-term memory).
• Demonstrate that explicit memory access patterns can be used to interpret model decisions in clinical prediction tasks such as mortality and phenotyping.

Proposed method

• Partition the external memory into two distinct blocks: a primary explicit memory that stores exact, unmodified copies of recent input events for traceability.
• Introduce a secondary blurred memory that accumulates high-level abstractions of past events no longer in explicit memory, acting as a stable, continuous memory for training.
• Use a gated memory controller (GRU-based) to dynamically decide which memory blocks to read from at each time step, with separate attention gates for explicit and blurred memory.
• Apply a fixed temperature Gumbel-Softmax reparameterization to the read gate for the explicit memory, enabling differentiable training despite discrete-like behavior.
• Train the model using SGD with momentum and gradient clipping, with hyperparameters (memory size, controller size, number of reads) tuned via validation for each task.
• Leverage the Atkinson-Shiffrin cognitive model as inspiration, structuring the memory hierarchy to mirror human sensory, short-term, and long-term memory processes.

Experimental results

Research questions

• RQ1 Can a memory network architecture that combines explicit and blurred memory components achieve state-of-the-art performance on EHR classification tasks while enabling interpretable model decisions?
• RQ2 How does the use of explicit memory improve model interpretability in clinical prediction tasks such as in-hospital mortality and phenotyping?
• RQ3 What is the role of the blurred memory in stabilizing training and improving generalization, especially in low-data settings common in EHR research?
• RQ4 How do the read gate dynamics differ across tasks, and what do they reveal about the model’s reliance on explicit versus abstracted memory representations?
• RQ5 Can the proposed architecture outperform or match the performance of complex models like multi-layered transformers while remaining more interpretable?

Key findings

• EBmRNN achieves AUC-ROC scores comparable to state-of-the-art models, including multi-layered transformer architectures, on three ICU prediction tasks using only a single-layered GRU controller.
• The explicit memory enables full traceability of input events, with visualization showing that key events are preserved in explicit slots for extended durations before being moved to the blurred memory.
• The read gate dynamics reveal task-specific memory usage: higher reliance on explicit memory for phenotyping tasks, while both memory types are used more equally for in-hospital mortality prediction.
• The blurred memory significantly improves training stability; models without it (e.g., EmRNN) suffered from exploding gradients and required higher supervision, making them harder to train.
• Despite the benefits of explicit memory, the inclusion of the blurred memory is crucial for performance on complex tasks like in-hospital mortality, where abstracted long-term patterns are essential.
• The model’s interpretability is enhanced by the ability to analyze which specific past events (via explicit memory slots) contributed most to predictions, supporting post-hoc explanation in clinical contexts.