[Paper Review] ForecastPFN: Synthetically-Trained Zero-Shot Forecasting

Samuel Dooley, Gurnoor Singh Khurana+3 more|arXiv (Cornell University)|Nov 3, 2023

Forecasting Techniques and Applications7 citations

TL;DR

ForecastPFN trains on a novel synthetic data distribution to perform zero-shot time-series forecasting, delivering accurate predictions in one forward pass without using target-series data for training. It outperforms several state-of-the-art methods under low data/time budgets and is exceptionally fast.

ABSTRACT

The vast majority of time-series forecasting approaches require a substantial training dataset. However, many real-life forecasting applications have very little initial observations, sometimes just 40 or fewer. Thus, the applicability of most forecasting methods is restricted in data-sparse commercial applications. While there is recent work in the setting of very limited initial data (so-called `zero-shot' forecasting), its performance is inconsistent depending on the data used for pretraining. In this work, we take a different approach and devise ForecastPFN, the first zero-shot forecasting model trained purely on a novel synthetic data distribution. ForecastPFN is a prior-data fitted network, trained to approximate Bayesian inference, which can make predictions on a new time series dataset in a single forward pass. Through extensive experiments, we show that zero-shot predictions made by ForecastPFN are more accurate and faster compared to state-of-the-art forecasting methods, even when the other methods are allowed to train on hundreds of additional in-distribution data points.

Motivation & Objective

• Motivate forecasting in data-sparse real-world settings where few initial observations exist.
• Propose a zero-shot forecasting model trained purely on synthetic data that approximates Bayesian inference.
• Develop a flexible transformer-based PFN capable of arbitrary future-query predictions.
• Show that synthetic-prior training yields superior accuracy and speed compared to data-hungry baselines.
• Demonstrate robustness and speed advantages across diverse real-world datasets.

Proposed method

• Introduce ForecastPFN, a prior-data fitted network (PFN) trained offline on a novel synthetic time-series prior.
• Design a modular synthetic data distribution with multi-scale seasonality, linear/exponential global trends, and Weibull-based multiplicative noise.
• Use a transformer-based encoder that accepts (t, y_t) tokens and a future date query to produce a single future prediction.
• Apply robust scaling and a loss that removes ground-truth noise to stabilize training.
• Train on 300,000 synthetic series with a sliding window to create 101 prediction tasks per series, total 1,024,000 tasks per epoch across 600 epochs.
• ForecastPFN predicts arbitrary future timesteps in a single forward pass, enabling true zero-shot forecasting on unseen datasets.

Experimental results

Research questions

• RQ1 Can a zero-shot forecasting model trained solely on synthetic data achieve competitive accuracy on real-world time series without target-series training data?
• RQ2 How does a synthetic prior enable generalization across diverse time-series patterns (seasonality, trends, noise) compared to real-data priors?
• RQ3 What architectural and training choices best support zero-shot, variable-length future queries in forecasting?
• RQ4 How does ForecastPFN perform relative to existing zero-shot and data-informed forecasting methods under constrained data and time budgets?
• RQ5 What are the speed and robustness benefits of single-pass zero-shot inference in practical forecasting workflows?

Key findings

• ForecastPFN achieves strong zero-shot performance across multiple real-world datasets without observing target-series data for training.
• In low data and low time budgets, ForecastPFN outperforms state-of-the-art methods, and remains competitive with them as budgets grow.
• ForecastPFN is exceptionally fast, producing predictions in about 0.2 seconds per dataset (single forward pass).
• When compared to Meta-N-BEATS (another zero-shot method), ForecastPFN consistently outperforms across settings.
• The synthetic prior with multi-scale trends and noise enables broad generalization to diverse time-series patterns.
• ForecastPFN often achieves the lowest average MSE across datasets at small data budgets and remains strong at larger budgets.