[论文解读] Autoregressive Convolutional Neural Networks for Asynchronous Time Series

Mikołaj Bińkowski, Gautier Marti+1 more|arXiv (Cornell University)|Mar 12, 2017

Stock Market Forecasting Methods参考文献 30被引用 50

一句话总结

SOCNN 是一种 Significance-Offset Convolutional Neural Network 用于回归的多变量异步时间序列，在异步数据上优于 CNN 和 LSTM，同时在其他数据集上表现与之媲美或更好。

ABSTRACT

We propose Significance-Offset Convolutional Neural Network, a deep convolutional network architecture for regression of multivariate asynchronous time series. The model is inspired by standard autoregressive (AR) models and gating mechanisms used in recurrent neural networks. It involves an AR-like weighting system, where the final predictor is obtained as a weighted sum of adjusted regressors, while the weights are datadependent functions learnt through a convolutional network. The architecture was designed for applications on asynchronous time series and is evaluated on such datasets: a hedge fund proprietary dataset of over 2 million quotes for a credit derivative index, an artificially generated noisy autoregressive series and UCI household electricity consumption dataset. The proposed architecture achieves promising results as compared to convolutional and recurrent neural networks.

研究动机与目标

• Motivate and address regression on multivariate asynchronous time series with nonlinear dependencies.
• Propose a neural architecture that learns data-dependent significance weights and offset predictors for past observations.
• Assess SOCNN against CNN, ResNet, LSTM, Phased LSTM, and VAR on synthetic, electricity, and financial quote datasets.

提出的方法

• Introduce SOCNN architecture with an offset sub-network and a significance sub-network.
• Represent past observations with a weighted autoregressive formulation where weights are produced by a convolutional network.
• Use an auxiliary loss to regularize individual past predictors (offsets) alongside the main L2 loss.
• Train using Adam with batch normalization, dropout, and early stopping; compare against multiple baselines on diverse datasets.
• Provide interpretable components by separating temporal dependence (weights), local significance (S network), and predictors (off network).

实验结果

研究问题

• RQ1 Can SOCNN improve regression performance for asynchronous multivariate time series compared to standard CNNs and RNN-based models?
• RQ2 How do data-dependent significance weights and per-past-offset predictors impact forecasting accuracy in irregular time series?
• RQ3 What is the effect of auxiliary loss and offset depth on model performance and training stability?
• RQ4 How does SOCNN perform across synthetic asynchronous data, electricity consumption, and real-world quotes data?
• RQ5 Is SOCNN robust to added noise relative to other architectures?

主要发现

模型	VAR	CNN	ResNet	LSTM	Phased LSTM	SOCNN1	SOCNN1+
Synchronous 16	0.841 (0.000)	0.154 (0.003)	0.152 (0.001)	0.151 (0.001)	0.166 (0.026)	0.152 (0.001)	0.172 (0.001)
Synchronous 64	0.364 (0.000)	0.029 (0.001)	0.029 (0.001)	0.028 (0.000)	0.038 (0.004)	0.030 (0.001)	0.032 (0.001)
Asynchronous 16	0.577 (0.000)	0.080 (0.032)	0.059 (0.017)	0.038 (0.008)	1.021 (0.090)	0.019 (0.003)	0.026 (0.004)
Asynchronous 64	0.318 (0.000)	0.078 (0.029)	0.087 (0.014)	0.065 (0.020)	0.924 (0.119)	0.035 (0.006)	0.044 (0.118)
Electricity	0.729 (0.005)	0.371 (0.005)	0.394 (0.013)	0.461 (0.011)	0.744 (0.015)	0.163 (0.010)	0.165 (0.012)
Quotes	1.000 (0.019)	0.897 (0.019)	2.245 (0.179)	0.827 (0.024)	0.945 (0.034)	0.387 (0.016)	–

• SOCNN and its variant SOCNN+ outperform CNN, ResNet, LSTM, Phased LSTM, and VAR on asynchronous and electricity/quotes datasets.
• On asynchronous data (16 and 64), SOCNN achieves lower mean squared error than benchmark networks, notably surpassing Phased LSTM and ResNet in several tasks.
• Auxiliary loss generally stabilizes training and can improve test error on asynchronous datasets, while offset-depth has limited impact.
• SOCNN is more robust to additional noise than the benchmark models, maintaining lower error under noisy inputs.
• On synchronous data, SOCNN matches but does not consistently exceed strong baselines, suggesting its advantages are most pronounced with asynchronous signals.