[Paper Review] An Empirical Study: Extensive Deep Temporal Point Process

Haitao Lin, Cheng Tan+4 more|arXiv (Cornell University)|Oct 19, 2021

Complex Network Analysis Techniques59 references4 citations

TL;DR

This paper proposes an extensive deep temporal point process (EDTPP) framework that systematically evaluates and extends key components of deep temporal point processes: history encoding, intensity function formulation, and Granger causality discovery. By remodularizing models with a unified learning strategy and introducing a variational graph learning framework for latent causal structure discovery, EDTPP achieves improved predictive performance and enhanced interpretability on real-world datasets.

ABSTRACT

Temporal point process as the stochastic process on continuous domain of time is commonly used to model the asynchronous event sequence featuring with occurrence timestamps. Thanks to the strong expressivity of deep neural networks, they are emerging as a promising choice for capturing the patterns in asynchronous sequences, in the context of temporal point process. In this paper, we first review recent research emphasis and difficulties in modeling asynchronous event sequences with deep temporal point process, which can be concluded into four fields: encoding of history sequence, formulation of conditional intensity function, relational discovery of events and learning approaches for optimization. We introduce most of recently proposed models by dismantling them into the four parts, and conduct experiments by remodularizing the first three parts with the same learning strategy for a fair empirical evaluation. Besides, we extend the history encoders and conditional intensity function family, and propose a Granger causality discovery framework for exploiting the relations among multi-types of events. Because the Granger causality can be represented by the Granger causality graph, discrete graph structure learning in the framework of Variational Inference is employed to reveal latent structures of the graph. Further experiments show that the proposed framework with latent graph discovery can both capture the relations and achieve an improved fitting and predicting performance.

Motivation & Objective

• To identify the most critical components in deep temporal point processes that drive performance improvements.
• To address the lack of interpretability in deep temporal point processes by enabling relational discovery among multi-type events.
• To propose a fair empirical evaluation framework by decoupling and recombining model components (encoding, intensity function) under a unified optimization strategy.
• To develop a variational inference-based graph structure learning method for discovering Granger causalities in event sequences.
• To highlight limitations in current datasets and experimental practices, calling for improved data quality and standardized evaluation protocols.

Proposed method

• Dismantles existing deep temporal point process models into four core components: history encoding, intensity function formulation, relational discovery, and optimization.
• Extends history encoders with a modified FNet architecture to improve sequence representation.
• Introduces a family of mixture distributions supported on semi-infinite intervals for more flexible conditional intensity function modeling.
• Proposes a variational inference framework for discrete graph structure learning to discover latent Granger causality graphs among event types.
• Employs a unified training strategy across all experiments to ensure fair comparison across different combinations of encoders and intensity functions.
• Applies the framework to real-world datasets (e.g., MOOC, Stack Overflow) to evaluate both predictive performance and causal structure discovery.

Experimental results

Research questions

• RQ1 Which component—history encoding or intensity function formulation—has the most significant impact on model performance in deep temporal point processes?
• RQ2 Can a unified modular framework enable fair and systematic evaluation of existing deep temporal point process models?
• RQ3 To what extent can variational graph structure learning uncover meaningful Granger causal relationships in multi-type event sequences?
• RQ4 How do current experimental settings and data quality affect the reliability of model evaluation in temporal point process research?
• RQ5 What are the key limitations in current deep temporal point process models regarding interpretability, generalization, and computational efficiency?

Key findings

• The history encoder has a more substantial impact on model performance than the intensity function formulation, as shown by ablation studies across multiple datasets.
• The proposed variational Granger causality discovery framework successfully identifies latent causal structures in event sequences, improving both interpretability and predictive accuracy.
• Experiments reveal high variance in first-event timestamps across real-world datasets like MOOC, challenging the i.i.d. assumption and raising concerns about data quality and model generalization.
• The framework demonstrates competitive performance on real-world datasets while enabling the discovery of meaningful inter-event causal relationships.
• The study identifies critical gaps in current evaluation practices, including inconsistent experimental settings and insufficient ablation studies, urging standardized benchmarks.
• Computational complexity increases significantly with model depth and sequence length, highlighting the need for complexity analysis in future model design.