[Paper Review] Learning Transactions Representations for Information Management in Banks: Mastering Local, Global, and External Knowledge

Alexandra Bazarova, Maria Kovaleva+8 more|arXiv (Cornell University)|Apr 2, 2024

Business Strategy and Innovation4 citations

TL;DR

This paper proposes a unified representation learning framework for financial transactional data that jointly captures local, global, and dynamic properties of customer behavior, integrating external context from aggregated client representations. It introduces novel generative models and a benchmark for evaluating representations across local (e.g., next MCC prediction), global (entire history), and dynamic (evolution over time) dimensions, achieving up to 14% higher ROC-AUC than contrastive baselines and 20% improvement with external context.

ABSTRACT

In today's world, banks use artificial intelligence to optimize diverse business processes, aiming to improve customer experience. Most of the customer-related tasks can be categorized into two groups: 1) local ones, which focus on a client's current state, such as transaction forecasting, and 2) global ones, which consider the general customer behaviour, e.g., predicting successful loan repayment. Unfortunately, maintaining separate models for each task is costly. Therefore, to better facilitate information management, we compared eight state-of-the-art unsupervised methods on 11 tasks in search for a one-size-fits-all solution. Contrastive self-supervised learning methods were demonstrated to excel at global problems, while generative techniques were superior at local tasks. We also introduced a novel approach, which enriches the client's representation by incorporating external information gathered from other clients. Our method outperforms classical models, boosting accuracy by up to 20\%.

Motivation & Objective

• To address the gap in existing methods that focus on either local or global representations, but not both simultaneously.
• To develop a unified representation learning framework that captures temporal dynamics and evolving client behavior over time.
• To incorporate external context—such as aggregated client behaviors or macroeconomic trends—into individual customer representations without relying on expert-defined indicators.
• To design a comprehensive benchmark for evaluating representations across local, global, and dynamic properties in financial transaction data.
• To improve performance on downstream tasks like next event prediction and anomaly detection through better representation quality.

Proposed method

• The framework uses a hybrid approach combining generative models (e.g., autoencoders, transformers, and temporal point processes) with contrastive learning to learn universal embeddings from transaction sequences.
• It employs a dual-loss training strategy: a reconstruction loss for generative modeling and a contrastive loss to preserve structural similarity between sequences.
• External context is modeled by aggregating representations of all clients in the training set and injecting this global signal into individual client encoders via attention mechanisms.
• The method uses temporal encoding and normalized time intervals (in days) to handle irregularly spaced transaction events, avoiding numerical overflow from raw timestamps.
• A dedicated validation protocol evaluates local representations using sliding windows over sequences, with metrics like ROC-AUC for next event prediction.
• Models are trained with careful hyperparameter tuning, including sequence length clipping for rare MCC codes and batch size reduction when incorporating external context due to memory constraints.

Experimental results

Research questions

• RQ1 Can a single representation learning framework effectively capture both local (current state) and global (entire history) properties of financial transaction sequences?
• RQ2 How does integrating external context—derived from aggregated client representations—improve the quality of individual customer embeddings?
• RQ3 To what extent do generative models outperform contrastive baselines in local downstream tasks such as next MCC prediction?
• RQ4 How can representation dynamics over time be meaningfully assessed and benchmarked in transactional data?
• RQ5 What is the impact of data-specific characteristics (e.g., irregular timing, mixed categorical and continuous features) on representation learning performance?

Key findings

• The proposed generative framework achieves up to 14% higher ROC-AUC on the next MCC prediction task compared to state-of-the-art contrastive baselines.
• Incorporating external context improves performance by an additional 20% on downstream tasks, demonstrating the value of global behavioral signals.
• The model’s local representations significantly outperform existing methods in capturing momentary customer behavior, crucial for fraud detection and change point detection.
• The benchmark framework successfully evaluates representations across local, global, and dynamic dimensions, offering a holistic assessment of representation quality.
• The use of a random subset of client representations for external context modeling enables scalability despite high memory demands.
• The framework generalizes across diverse datasets, including public (Churn, Age, Default, HSBC) and private industrial data, showing robustness to domain shifts.