[Paper Review] Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future

Harshavardhan Kamarthi, Alexander Rodríguez+1 more|arXiv (Cornell University)|Jun 8, 2021

Anomaly Detection Techniques and Applications44 references4 citations

TL;DR

Back2Future (B2F) is a novel deep learning framework that improves real-time forecasting by modeling multi-variate backfill dynamics—where both features and targets are revised over time—using graph convolutional and recurrent networks to refine predictions. It achieves 6.65% to 11.24% improvement in top COVID-19 forecasting models and 18% over baselines, while also enhancing model evaluation accuracy by aligning real-time predictions with stable, revised ground truth.

ABSTRACT

In real-time forecasting in public health, data collection is a non-trivial and demanding task. Often after initially released, it undergoes several revisions later (maybe due to human or technical constraints) - as a result, it may take weeks until the data reaches to a stable value. This so-called 'backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. In this paper, we introduce the multi-variate backfill problem using COVID-19 as the motivating example. We construct a detailed dataset composed of relevant signals over the past year of the pandemic. We then systematically characterize several patterns in backfill dynamics and leverage our observations for formulating a novel problem and neural framework Back2Future that aims to refines a given model's predictions in real-time. Our extensive experiments demonstrate that our method refines the performance of top models for COVID-19 forecasting, in contrast to non-trivial baselines, yielding 18% improvement over baselines, enabling us obtain a new SOTA performance. In addition, we show that our model improves model evaluation too; hence policy-makers can better understand the true accuracy of forecasting models in real-time.

Motivation & Objective

• To address the underexplored problem of multi-variate backfill in real-time forecasting, where both target variables and exogenous features are subject to retrospective revisions.
• To characterize backfill dynamics across diverse signals during the COVID-19 pandemic and identify patterns that affect model performance.
• To develop a generalizable neural framework, Back2Future (B2F), that refines real-time model predictions using revision patterns across features and regions.
• To improve model evaluation by rectifying performance metrics to reflect true accuracy on fully revised data.
• To extend the framework to macroeconomic forecasting, demonstrating generalizability beyond public health.

Proposed method

• B2F models backfill dynamics using a Graph Recurrent Neural Network (GRNN) that captures cross-feature and cross-regional similarities in revision patterns.
• The model learns latent representations of features by encoding their temporal backfill trajectories through bidirectional GRUs.
• It integrates these learned representations with historical model predictions to refine real-time forecasts toward revised ground truth values.
• The framework is applied as a wrapper to diverse forecasting models, enabling performance improvement without retraining the base models.
• Spatial and temporal dependencies in revision patterns are modeled jointly using graph convolutional layers over a regional signal graph.
• The method is trained end-to-end to minimize the mean absolute error (MAE) between refined predictions and final revised targets.

Experimental results

Research questions

• RQ1 How do backfill dynamics vary across different epidemiological and economic signals during real-time forecasting?
• RQ2 To what extent do unrevised feature values degrade the performance of real-time forecasting models?
• RQ3 Can cross-signal and cross-regional similarities in backfill patterns be leveraged to improve prediction accuracy?
• RQ4 How quickly can a backfill-aware model achieve stable performance with limited revision data?
• RQ5 Can the proposed framework generalize beyond public health to other domains with backfill, such as macroeconomic forecasting?

Key findings

• Back2Future improved top-performing COVID-19 forecasting models by 6.65% to 11.24% in mean absolute error (MAE) on revised data.
• The method achieved an 18% improvement over non-trivial baselines in reducing prediction error on fully revised targets.
• B2F’s performance stabilized within 6 weeks of data availability, demonstrating rapid adaptation even with limited revision history.
• During anomalous revisions—such as a 48% error spike in week 5—B2F reduced prediction error by up to 74.2% for ensemble models.
• The framework improved GDP forecasts by 6% to 15% across multiple candidate models, confirming its generalizability to macroeconomic forecasting.
• B2F refined model evaluations by aligning real-time performance metrics closer to stable, final MAE scores, enabling better assessment of model accuracy.