[Paper Review] Autoregressive Convolutional Neural Networks for Asynchronous Time Series

Mikołaj Bińkowski, Gautier Marti+1 more|arXiv (Cornell University)|Mar 12, 2017

Stock Market Forecasting Methods54 citations

TL;DR

Introduces Significance-Offset Convolutional Neural Network (SOCNN), a CNN-based autoregressive model for multivariate asynchronous time series that learns data-dependent weights to combine past observations, outperforming several baselines on asynchronous financial and consumption data.

ABSTRACT

We propose Significance-Offset Convolutional Neural Network, a deep convolutional network architecture for regression of multivariate asynchronous time series. The model is inspired by standard autoregressive (AR) models and gating mechanisms used in recurrent neural networks. It involves an AR-like weighting system, where the final predictor is obtained as a weighted sum of adjusted regressors, while the weights are datadependent functions learnt through a convolutional network. The architecture was designed for applications on asynchronous time series and is evaluated on such datasets: a hedge fund proprietary dataset of over 2 million quotes for a credit derivative index, an artificially generated noisy autoregressive series and UCI household electricity consumption dataset. The proposed architecture achieves promising results as compared to convolutional and recurrent neural networks.

Motivation & Objective

• Motivate forecasting of multivariate, irregularly sampled time series, especially in finance and electricity consumption.
• Develop a neural architecture combining autoregressive ideas with CNN-based weighting mechanisms.
• Provide interpretable components (offset and significance networks) to handle asynchronous inputs.
• Empirically compare SOCNN against CNN, ResNet, LSTM variants, and VAR baselines across diverse datasets.
• Assess robustness and training dynamics of SOCNN including auxiliary loss and offset depth.

Proposed method

• Propose SOCNN where y_hat_n is a weighted sum of adjusted regressors via F(x_n^-M) and a data-dependent weighting function sigma(S(x_n^-M)).
• Decompose F into a linear projection plus a 1-site convolutional structure with an offset network off(x_n-m).
• Use a significance network S with fully convolutional layers to capture local temporal dependencies and compute data-dependent weights.
• Impose an auxiliary loss L_aux that encourages the offset predictors to approximate the actual target changes, controlled by alpha in L_tot = L^2 + alpha L_aux.
• Train with Adam, batch normalization, dropout, early stopping, and gradient clipping where needed; compare against CNN, ResNet, LSTM, Phased LSTM, and VAR.

Experimental results

Research questions

• RQ1 Can SOCNN effectively forecast asynchronously sampled multivariate time series with noisy observations?
• RQ2 Do data-dependent significance weights learned by a convolutional network improve predictive accuracy over standard autoregressive CNNs or LSTMs?
• RQ3 What is the impact of the offset depth and auxiliary loss on SOCNN performance and stability?
• RQ4 How does SOCNN perform on real-world asynchronous financial data, electricity consumption data, and synthetic asynchronous series compared to strong baselines?
• RQ5 Is the SOCNN architecture robust to added input noise and data irregularities?

Key findings

• SOCNN and its variants outperform benchmark networks on asynchronous, electricity, and quotes datasets.
• On synchronous data, SOCNN roughly matches benchmark models, suggesting its advantages are most pronounced with asynchrony and noise.
• The depth of the offset sub-network has little to negative impact, whereas the significance network drives performance gains.
• Auxiliary loss can stabilize training and, in some datasets, improve test error, especially for asynchronous data.
• Phased LSTM performs well in some tasks but can be unstable; SOCNN often achieves lower variance and better generalization in asynchronous settings.
• SOCNN shows robustness to added noise and tends to better utilize noisy observations instead of discarding them.