[Paper Review] Representations and Ensemble Methods for Dynamic Relational Classification

Ryan A. Rossi, Jennifer Neville|arXiv (Cornell University)|Nov 22, 2011

Bayesian Modeling and Causal Inference11 references3 citations

TL;DR

This paper proposes a framework for dynamic relational classification that leverages temporal representations of evolving relational data—focusing on time-varying links, attributes, and nodes—through temporal granularity and weighting strategies. By integrating these representations into relational classifiers and ensemble methods, the approach significantly improves classification accuracy over static models, especially when modeling evolving topics in communication networks.

ABSTRACT

Temporal networks are ubiquitous and evolve over time by the addition, deletion, and changing of links, nodes, and attributes. Although many relational datasets contain temporal information, the majority of existing techniques in relational learning focus on static snapshots and ignore the temporal dynamics. We propose a framework for discovering temporal representations of relational data to increase the accuracy of statistical relational learning algorithms. The temporal relational representations serve as a basis for classification, ensembles, and pattern mining in evolving domains. The framework includes (1) selecting the time-varying relational components (links, attributes, nodes), (2) selecting the temporal granularity, (3) predicting the temporal influence of each time-varying relational component, and (4) choosing the weighted relational classifier. Additionally, we propose temporal ensemble methods that exploit the temporal-dimension of relational data. These ensembles outperform traditional and more sophisticated relational ensembles while avoiding the issue of learning the most optimal representation. Finally, the space of temporal-relational models are evaluated using a sample of classifiers. In all cases, the proposed temporal-relational classifiers outperform competing models that ignore the temporal information. The results demonstrate the capability and necessity of the temporal-relational representations for classification, ensembles, and for mining temporal datasets.

Motivation & Objective

• Address the gap in relational learning by incorporating temporal dynamics into classification, as most existing methods focus on static snapshots.
• Develop a flexible framework to discover optimal temporal-relational representations by modeling time-varying components (links, attributes, nodes) with varying granularity and influence.
• Improve classification accuracy by leveraging temporal representations in both individual classifiers and ensemble methods.
• Explore the utility of temporal representations for mining temporal patterns and discovering evolutionary structures in relational data.
• Demonstrate that modeling temporal dynamics is essential for accurate prediction in evolving networks, especially in domains like communication and collaboration networks.

Proposed method

• Define a temporal-relational classification framework that selects time-varying components (links, attributes, nodes) and applies temporal granularity (timesteps, windows, unions).
• Apply temporal weighting—such as exponential decay—to model the influence of past relational components, reducing the impact of outdated data.
• Construct weighted relational classifiers by combining temporal representations, using models like Relational Bayes or other probabilistic classifiers.
• Design temporal ensemble methods that sample, randomize, or transform features across time, exploiting temporal variation to improve robustness and accuracy.
• Use LDA (Latent Dirichlet Allocation) to discover latent topics in communication networks and annotate links and individuals with topic memberships over time.
• Evaluate models using real-world datasets (email, bug, and combined communication networks) with annotated temporal topics and developer effectiveness metrics.

Experimental results

Research questions

• RQ1 How do temporal representations of evolving links, attributes, and nodes improve classification accuracy compared to static models?
• RQ2 What is the optimal combination of temporal granularity and weighting strategy for maximizing predictive performance in dynamic relational data?
• RQ3 Can temporal ensemble methods outperform traditional and more complex relational ensembles while avoiding the need to learn the best representation?
• RQ4 To what extent do latent topic models of communication evolve over time, and how do these patterns correlate with developer effectiveness?
• RQ5 How do temporal representations enhance the discovery of meaningful temporal patterns in relational datasets?

Key findings

• Temporal-relational classifiers consistently outperform static models across all evaluated classification tasks, demonstrating the necessity of modeling temporal dynamics.
• Models using only simple temporal representations (e.g., uniform weighting or basic windows) perform significantly worse than those with optimized temporal weighting and granularity.
• Temporal ensemble methods that exploit time-varying features outperform both traditional and sophisticated relational ensembles, achieving higher accuracy without requiring optimal representation learning.
• Latent topic modeling via LDA reveals meaningful evolutionary patterns in communication networks, with effective developers showing consistent topic engagement over time.
• The temporal evolution of topics is strongly correlated with developer effectiveness, indicating that communication structure and topic consistency are predictive of performance.
• Complex models with fine-grained temporal granularities improve performance but at a high computational cost, suggesting a trade-off between complexity and gain.