[Paper Review] TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series

Alexander Nikitin, Letizia Iannucci+1 more|arXiv (Cornell University)|May 19, 2023

Time Series Analysis and Forecasting4 citations

TL;DR

TSGM is an open-source, extensible framework for generative modeling of synthetic time series that supports data-driven and simulator-based methods, enabling evaluation across similarity, downstream effectiveness, consistency, diversity, and privacy. It streamlines research and industrial use through customizable training, built-in metrics, CLI tools, and integration with TensorFlow and TensorBoard, significantly reducing implementation time from weeks to days.

ABSTRACT

Temporally indexed data are essential in a wide range of fields and of interest to machine learning researchers. Time series data, however, are often scarce or highly sensitive, which precludes the sharing of data between researchers and industrial organizations and the application of existing and new data-intensive ML methods. A possible solution to this bottleneck is to generate synthetic data. In this work, we introduce Time Series Generative Modeling (TSGM), an open-source framework for the generative modeling of synthetic time series. TSGM includes a broad repertoire of machine learning methods: generative models, probabilistic, and simulator-based approaches. The framework enables users to evaluate the quality of the produced data from different angles: similarity, downstream effectiveness, predictive consistency, diversity, and privacy. The framework is extensible, which allows researchers to rapidly implement their own methods and compare them in a shareable environment. TSGM was tested on open datasets and in production and proved to be beneficial in both cases. Additionally to the library, the project allows users to employ command line interfaces for synthetic data generation which lowers the entry threshold for those without a programming background.

Motivation & Objective

• To address the scarcity and sensitivity of real-world time series data by enabling synthetic data generation.
• To reduce the barrier to entry for researchers and practitioners in time series generative modeling through a unified, extensible framework.
• To provide standardized, multi-faceted evaluation of synthetic time series across similarity, downstream task performance, predictive consistency, diversity, and privacy.
• To accelerate research and deployment by offering CLI access, pre-built architectures, and integration with existing ML tools like TensorFlow and TensorBoard.
• To promote fairness and privacy in synthetic data generation by embedding evaluation metrics for these critical concerns.

Proposed method

• TSGM implements a modular framework with generators for data-driven (e.g., GANs, VAEs) and simulator-based (e.g., Gaussian processes, surrogate models) time series generation.
• The framework includes a customizable architecture zoo, enabling reuse and extension of neural network components across different generative pipelines.
• It provides monitoring tools for tracking training convergence and intermediate results, including integration with TensorBoard for visualization.
• A comprehensive metrics suite evaluates synthetic data on similarity (e.g., statistical moments), downstream effectiveness (e.g., model accuracy), consistency, diversity, and privacy.
• The framework supports both unconditional and conditionally conditioned time series generation, enhancing applicability to diverse real-world scenarios.
• It includes utilities for data preprocessing, augmentation, visualization (e.g., t-SNE, line plots), and built-in datasets to streamline experimentation.

Experimental results

Research questions

• RQ1 How can a unified, extensible framework improve the reproducibility and comparability of synthetic time series generation methods?
• RQ2 To what extent can TSGM reduce the time required to implement and evaluate time series generative models in industrial and research settings?
• RQ3 How well do the proposed evaluation metrics—covering similarity, downstream performance, consistency, diversity, and privacy—capture the quality of synthetic time series?
• RQ4 Can TSGM effectively support both data-driven and simulator-based generative modeling approaches in a single, interoperable environment?
• RQ5 How does the inclusion of privacy and fairness metrics in the framework influence the design and evaluation of synthetic time series?

Key findings

• TSGM reduced the time to integrate and evaluate a single time series generation method from an estimated one week to one month (for open-source implementations) down to a few days when using the framework’s library.
• Using the CLI, users could generate synthetic time series in under 25 seconds on GPU and under 21 seconds on 2xGPU, demonstrating high computational efficiency.
• The framework achieved a 2x speedup on TPU compared to CPU, with training times dropping from ~25 minutes to ~16.5 seconds for the first two epochs.
• Experiments showed that models in TSGM could be trained and evaluated efficiently across multiple datasets, with consistent performance and reproducible results.
• The framework’s built-in metrics enabled comprehensive evaluation of synthetic data, with visualizations like t-SNE and loss curves aiding in model diagnostics and convergence monitoring.
• Industrial feedback confirmed that TSGM significantly lowered the barrier to entry, enabling rapid prototyping and deployment of synthetic data pipelines.