[Paper Review] Modeling the intensity function of point process via recurrent neural networkss

Shuai Xiao, Junchi Yan+3 more|arXiv (Cornell University)|Feb 4, 2017

Machine Learning in Materials Science39 references50 citations

TL;DR

This paper proposes a hybrid RNN architecture that models the intensity function of point processes by separately encoding background intensity via a time-indexed RNN and history-dependent effects via an event-indexed RNN, enabling end-to-end training for precise timestamp and event type prediction. The approach outperforms traditional parametric point process models on a real-world ATM maintenance log dataset.

ABSTRACT

Event sequence, asynchronously generated with random timestamp, is ubiquitous among applications. The precise and arbitrary timestamp can carry important clues about the underlying dynamics, and has lent the event data fundamentally different from the time-series whereby series is indexed with fixed and equal time interval. One expressive mathematical tool for modeling event is point process. The intensity functions of many point processes involve two components: the background and the effect by the history. Due to its inherent spontaneousness, the background can be treated as a time series while the other need to handle the history events. In this paper, we model the background by a Recurrent Neural Network (RNN) with its units aligned with time series indexes while the history effect is modeled by another RNN whose units are aligned with asynchronous events to capture the long-range dynamics. The whole model with event type and timestamp prediction output layers can be trained end-to-end. Our approach takes an RNN perspective to point process, and models its background and history effect. For utility, our method allows a black-box treatment for modeling the intensity which is often a pre-defined parametric form in point processes. Meanwhile end-to-end training opens the venue for reusing existing rich techniques in deep network for point process modeling. We apply our model to the predictive maintenance problem using a log dataset by more than 1000 ATMs from a global bank headquartered in North America.

Motivation & Objective

• To address the challenge of modeling complex, asynchronous event sequences with arbitrary timestamps in point processes.
• To decouple the background intensity (spontaneous events) from the history-dependent effects (reactive events) in point process modeling.
• To enable end-to-end training of intensity functions without relying on pre-specified parametric forms.
• To leverage deep learning techniques for modeling long-range temporal dependencies in event data.
• To evaluate the method on a real-world predictive maintenance task using a large-scale ATM log dataset.

Proposed method

• The background intensity is modeled using a time-indexed RNN, treating it as a standard time series.
• The history effect is modeled using an event-indexed RNN, where units are aligned with actual event timestamps to capture asynchronous dynamics.
• The two RNNs are combined to form a unified intensity function that integrates spontaneous and reactive components.
• The model predicts both event types and timestamps through dedicated output layers trained end-to-end.
• The architecture allows black-box modeling of intensity, avoiding the need for predefined parametric forms.
• End-to-end training enables the use of established deep learning optimization and regularization techniques.

Experimental results

Research questions

• RQ1 Can a dual-RNN architecture effectively model both background and history-dependent components of point process intensity?
• RQ2 Does end-to-end training of a neural intensity function improve predictive performance compared to traditional parametric models?
• RQ3 Can the model capture long-range temporal dependencies in asynchronous event sequences without relying on handcrafted intensity functions?
• RQ4 How well does the model generalize to real-world predictive maintenance tasks with complex event patterns?
• RQ5 To what extent does separating time-indexed and event-indexed RNNs improve modeling fidelity?

Key findings

• The proposed model achieves superior predictive performance on event type and timestamp prediction compared to conventional parametric point process models.
• The dual-RNN architecture effectively captures both spontaneous background activity and reactive history effects in event sequences.
• End-to-end training enables the model to learn complex intensity patterns without requiring explicit parametric assumptions.
• The model demonstrates strong generalization on a real-world log dataset from over 1000 ATMs, indicating practical utility in predictive maintenance.
• The separation of time-indexed and event-indexed RNNs allows for more expressive modeling of asynchronous event dynamics.
• The approach enables black-box modeling of intensity functions, opening the door to richer representations through deep learning.