[Paper Review] Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models

Vincent Le Guen, Nicolas Thome|arXiv (Cornell University)|Sep 19, 2019

Time Series Analysis and Forecasting105 citations

TL;DR

DILATE introduces a differentiable loss that separately optimizes shape accuracy and temporal localization for multi-step, non-stationary time series forecasting, improving over MSE and DTW-based losses across several datasets.

ABSTRACT

This paper addresses the problem of time series forecasting for non-stationary signals and multiple future steps prediction. To handle this challenging task, we introduce DILATE (DIstortion Loss including shApe and TimE), a new objective function for training deep neural networks. DILATE aims at accurately predicting sudden changes, and explicitly incorporates two terms supporting precise shape and temporal change detection. We introduce a differentiable loss function suitable for training deep neural nets, and provide a custom back-prop implementation for speeding up optimization. We also introduce a variant of DILATE, which provides a smooth generalization of temporally-constrained Dynamic Time Warping (DTW). Experiments carried out on various non-stationary datasets reveal the very good behaviour of DILATE compared to models trained with the standard Mean Squared Error (MSE) loss function, and also to DTW and variants. DILATE is also agnostic to the choice of the model, and we highlight its benefit for training fully connected networks as well as specialized recurrent architectures, showing its capacity to improve over state-of-the-art trajectory forecasting approaches.

Motivation & Objective

• Address multi-step forecasting of non-stationary time series with abrupt regime changes.
• Provide a differentiable loss that disentangles shape error from temporal misalignment.
• Enable efficient training of deep models (MLP and Seq2Seq) using DILATE.
• Demonstrate superior shape and temporal metrics over MSE and DTW variants across multiple datasets.

Proposed method

• Define a two-term differentiable loss: L_DILATE = alpha * L_shape + (1 - alpha) * L_temporal.
• L_shape is a differentiable DTW-based loss using a softmin over warping paths (DTW_gamma).
• L_temporal measures deviation from the optimal DTW path via a smoothed approximation A_gamma^* and a temporal penalty matrix Omega.
• Provide a custom backward implementation to reduce computational cost to O(k^2) per forward/backward pass.
• Extend with a tangential variant L_DILATE^t that blends DTW with temporal constraints (through Omega).
• Show that DILATE can be used with MLP and Seq2Seq architectures and is compatible with various datasets.

Experimental results

Research questions

• RQ1 Can a differentiable, two-term distortion loss improve multi-step forecasting of non-stationary time series over standard MSE?
• RQ2 Does separating shape and temporal components help capture sharp changes and their timing more accurately?
• RQ3 Is the DILATE loss agnostic to network architecture and effective with both simple and sequence-based models?
• RQ4 How does DILATE compare to DTW-based losses and other state-of-the-art forecasting models on real and synthetic data?

Key findings

• DILATE outperforms MSE on shape (DTW) and time (TDI) metrics across synthetic, ECG, and traffic datasets, with significant differences in several cases.
• Compared to DTW_gamma, DILATE yields better temporal accuracy (TDI) in all experiments and superior shape-time balance overall.
• Seq2Seq models trained with DILATE achieve better shape and time metrics than state-of-the-art models trained with MSE on the Traffic dataset.
• DILATE maintains competitive or superior performance on MSE when evaluated with MSE, showing robustness across evaluation metrics.
• A bespoke backward implementation reduces training time, with larger speedups as prediction horizon k grows.