[Paper Review] Generative AI for Banks: Benchmarks and Algorithms for Synthetic Financial Transaction Data

Fabian Karst, Sook-Yee Chong+4 more|arXiv (Cornell University)|Dec 19, 2024

Stock Market Forecasting MethodsDecision Sciences3 citations

TL;DR

This paper evaluates five generative AI models—CTGAN, DoppelGAN, Wasserstein GAN, Financial Diffusion (FinDiff), and TVAE—for creating synthetic financial transaction data, benchmarking them across fidelity, privacy, efficiency, synthesis quality, and graph structure. CTGAN achieves the best overall balance, while FinDiff and TVAE excel in data replication and augmentation, and DoppelGAN is optimal for privacy-sensitive applications.

ABSTRACT

The banking sector faces challenges in using deep learning due to data sensitivity and regulatory constraints, but generative AI may offer a solution. Thus, this study identifies effective algorithms for generating synthetic financial transaction data and evaluates five leading models - Conditional Tabular Generative Adversarial Networks (CTGAN), DoppelGANger (DGAN), Wasserstein GAN, Financial Diffusion (FinDiff), and Tabular Variational AutoEncoders (TVAE) - across five criteria: fidelity, synthesis quality, efficiency, privacy, and graph structure. While none of the algorithms is able to replicate the real data's graph structure, each excels in specific areas: DGAN is ideal for privacy-sensitive tasks, FinDiff and TVAE excel in data replication and augmentation, and CTGAN achieves a balance across all five criteria, making it suitable for general applications with moderate privacy concerns. As a result, our findings offer valuable insights for choosing the most suitable algorithm.

Motivation & Objective

• Address the challenge of using deep learning in banking due to data sensitivity and regulatory constraints.
• Identify generative AI algorithms suitable for creating synthetic financial transaction data that preserve privacy and data quality.
• Evaluate multiple state-of-the-art models across key criteria relevant to financial institutions.
• Provide actionable guidance for selecting the most appropriate algorithm based on specific use-case requirements such as privacy, data fidelity, or efficiency.

Proposed method

• Benchmark five leading tabular generative models: CTGAN, DoppelGAN, Wasserstein GAN, Financial Diffusion (FinDiff), and Tabular Variational Autoencoders (TVAE).
• Evaluate models across five criteria: data fidelity, synthesis quality, computational efficiency, privacy preservation, and graph structure replication.
• Use real-world financial transaction data to train and validate synthetic data generation.
• Apply quantitative metrics such as statistical similarity, privacy leakage, and reconstruction accuracy to assess performance.
• Analyze graph structure preservation by measuring topological similarity between real and synthetic transaction networks.
• Conduct ablation studies to isolate performance differences across model architectures and training protocols.

Experimental results

Research questions

• RQ1 Which generative model produces synthetic financial transaction data with the highest fidelity to real-world data?
• RQ2 How well do different models preserve privacy while generating realistic transaction patterns?
• RQ3 What is the computational efficiency of each model in generating large-scale synthetic transaction datasets?
• RQ4 To what extent can each model replicate the underlying graph structure of real transaction networks?
• RQ5 Which model offers the best trade-off between data quality, privacy, and efficiency for general banking applications?

Key findings

• CTGAN achieves the best overall balance across all five evaluation criteria, making it suitable for general-purpose synthetic data generation with moderate privacy concerns.
• DoppelGAN is the most effective model for privacy-sensitive applications, minimizing privacy leakage while maintaining acceptable data quality.
• FinDiff and TVAE demonstrate superior performance in data replication and augmentation, particularly in preserving complex statistical patterns in transaction data.
• No model fully replicates the graph structure of real transaction networks, indicating a key limitation in current generative modeling for financial data.
• TVAE and FinDiff show strong performance in generating diverse and realistic transaction sequences, outperforming GAN-based models in certain fidelity metrics.
• Computational efficiency varies significantly, with TVAE and CTGAN being more scalable for large-scale deployment compared to DoppelGAN and Wasserstein GAN.