[Paper Review] Visual Forecasting of Time Series with Image-to-Image Regression.

Naftali Cohen, Srijan Sood+3 more|arXiv (Cornell University)|Nov 18, 2020

Data Visualization and Analytics24 references4 citations

TL;DR

This paper proposes visual forecasting for time series by transforming numerical data into visual representations and using a convolutional autoencoder to generate visual forecasts. The method outperforms numerical baselines by converting continuous regression into discrete pixel-space prediction, showing strong results on cyclic data but reduced performance on irregular data like stock prices.

ABSTRACT

Time series forecasting is essential for agents to make decisions in many domains. Existing models rely on classical statistical methods to predict future values based on previously observed numerical information. Yet, practitioners often rely on visualizations such as charts and plots to reason about their predictions. Inspired by the end-users, we re-imagine the topic by creating a framework to produce visual forecasts, similar to the way humans intuitively do. In this work, we take a novel approach by leveraging advances in deep learning to extend the field of time series forecasting to a visual setting. We do this by transforming the numerical analysis problem into the computer vision domain. Using visualizations of time series data as input, we train a convolutional autoencoder to produce corresponding visual forecasts. We examine various synthetic and real datasets with diverse degrees of complexity. Our experiments show that visual forecasting is effective for cyclic data but somewhat less for irregular data such as stock price. Importantly, we find the proposed visual forecasting method to outperform numerical baselines. We attribute the success of the visual forecasting approach to the fact that we convert the continuous numerical regression problem into a discrete domain with quantization of the continuous target signal into pixel space.

Motivation & Objective

• To address the gap between human intuition in visual reasoning and traditional numerical forecasting in time series analysis.
• To explore whether visual representations of time series can improve forecasting performance through deep learning.
• To investigate if converting continuous numerical regression into discrete pixel-space prediction enhances model generalization and accuracy.
• To evaluate the effectiveness of visual forecasting across diverse data types, including synthetic and real-world time series.

Proposed method

• Transform time series data into visualizations (e.g., line plots) to represent the input as images.
• Use a convolutional autoencoder to learn a latent representation of the input visualizations and reconstruct the forecasted visual output.
• Quantize the continuous target signal into discrete pixel values to convert the regression problem into a classification-like task in pixel space.
• Train the model end-to-end using image-to-image regression, minimizing reconstruction loss between predicted and ground-truth visual forecasts.
• Apply the model to both synthetic and real-world datasets with varying complexity and temporal patterns.

Experimental results

Research questions

• RQ1 Can visual forecasting via image-to-image regression outperform traditional numerical forecasting models?
• RQ2 How does the performance of visual forecasting vary across different time series patterns, such as cyclic versus irregular trends?
• RQ3 To what extent does quantizing the target signal into pixel space improve forecasting accuracy?
• RQ4 Does the human-like visual reasoning process in forecasting translate into measurable performance gains in machine learning models?

Key findings

• The visual forecasting approach outperforms numerical baselines across multiple datasets, demonstrating the effectiveness of the proposed method.
• The model achieves strong performance on cyclic time series data, where visual patterns are more predictable and consistent.
• Performance degrades on irregular data such as stock price series, indicating limitations in handling high-noise or non-periodic patterns.
• Quantization of the continuous target signal into pixel space is a key factor in the method's success, enabling better generalization through discrete representation.