[Paper Review] Memory-Augmented Neural Networks for Predictive Process Analytics

Asjad Khan, Hung Lê+5 more|arXiv (Cornell University)|Feb 3, 2018

Business Process Modeling and Analysis12 references16 citations

TL;DR

This paper proposes a memory-augmented neural network (MANN) framework, enhanced with dual controllers and a write-protected memory policy, to improve predictive process analytics. It achieves state-of-the-art performance on next activity prediction, time-to-completion, and suffix prediction using real-world datasets, demonstrating strong potential as a prescriptive process recommendation engine.

ABSTRACT

Process analytics involves a sophisticated layer of data analytics built over the traditional notion of process mining. The flexible execution of business process instances involves multiple critical decisions including what task to perform next and what resources to allocate to a task. In this paper, we explore the application of deep learning techniques for solving various process analytics related problems. Based on recent advances in the field we specifically look at memory-augmented neural networks (MANN)s and adapt the latest model to date, namely the Differential Neural Computer. We introduce two modifications to account for a variety of tasks in predictive process analytics: (i) separating the encoding phase and decoding phase, resulting dual controllers, one for each phase; (ii) implementing a write-protected policy for the memory during the decoding phase. We demonstrate the feasibility and usefulness of our approach by solving a number of common process analytics tasks such as next activity prediction, time to completion and suffix prediction. We also introduce the notion of MANN based process analytics recommendation machinery that once deployed can serve as an effective business process recommendation engine enabling organizations to answer various prescriptive process analytics related questions.Using real-world datasets, we benchmark our results against those obtained from the state-of-art methods. We show that MANNs based process analytics methods can acheive state-of-the-art performance and have a lot of value to offer for enterprise specific process anlaytics applications.

Motivation & Objective

• Address the limitations of traditional process mining by integrating deep learning for more accurate predictive and prescriptive analytics.
• Overcome challenges in modeling complex, dynamic business processes with long-term dependencies and variable execution paths.
• Develop a flexible, scalable framework capable of handling diverse predictive process analytics tasks such as next activity prediction and time-to-completion.
• Introduce a memory-augmented architecture that supports reasoning over process instances with external memory for improved decision-making.
• Enable the deployment of an intelligent recommendation engine for enterprise process optimization through MANN-based inference.

Proposed method

• Adapt the Differential Neural Computer (DNC), a state-of-the-art MANN, for process analytics by introducing separate encoding and decoding controllers.
• Decouple the memory access mechanism into distinct phases: encoding phase for learning process representations and decoding phase for prediction.
• Implement a write-protected memory policy during the decoding phase to preserve learned representations and prevent interference during inference.
• Use memory read and write operations to store and retrieve process state information, enabling reasoning over sequences of activities and resources.
• Train the model end-to-end on event logs to optimize for multiple predictive tasks simultaneously.
• Leverage attention-based memory addressing to dynamically focus on relevant historical process states during prediction.

Experimental results

Research questions

• RQ1 Can a memory-augmented neural network effectively model long-term dependencies in business process execution logs?
• RQ2 How does separating encoding and decoding controllers improve performance in predictive process analytics tasks?
• RQ3 To what extent does a write-protected memory policy during decoding enhance prediction stability and accuracy?
• RQ4 Can the proposed MANN-based framework achieve state-of-the-art performance across multiple process analytics tasks?
• RQ5 How effective is the framework as a foundation for a real-time business process recommendation engine?

Key findings

• The proposed MANN-based method achieves state-of-the-art performance on next activity prediction, time-to-completion, and suffix prediction tasks.
• The separation of encoding and decoding controllers improves model stability and predictive accuracy by isolating representation learning from generation.
• The write-protected memory policy during decoding significantly reduces memory interference and enhances generalization across diverse process instances.
• The framework demonstrates strong generalization on real-world datasets, outperforming existing state-of-the-art methods in multiple evaluation scenarios.
• The architecture is effective as a foundation for a prescriptive process recommendation engine, capable of answering complex business process optimization questions.
• The model's ability to reason over long sequences and external memory enables robust handling of variable and complex process flows.