[论文解读] SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learning

Cheng Tan, Zhangyang Gao+2 more|arXiv (Cornell University)|Nov 22, 2022

Domain Adaptation and Few-Shot Learning被引用 34

一句话总结

SimVPv2 表明一个简单的基于 CNN 的基线在时空预测性能上达到最先进水平，包含将门控时空注意力（gSTA）和 Inception 风格的时序模块的变体。它在数据集上展现出强大的效率与泛化能力。

ABSTRACT

Recent years have witnessed remarkable advances in spatiotemporal predictive learning, with methods incorporating auxiliary inputs, complex neural architectures, and sophisticated training strategies. While SimVP has introduced a simpler, CNN-based baseline for this task, it still relies on heavy Unet-like architectures for spatial and temporal modeling, which still suffers from high complexity and computational overhead. In this paper, we propose SimVPv2, a streamlined model that eliminates the need for Unet architectures and demonstrates that plain stacks of convolutional layers, enhanced with an efficient Gated Spatiotemporal Attention mechanism, can deliver state-of-the-art performance. SimVPv2 not only simplifies the model architecture but also improves both performance and computational efficiency. On the standard Moving MNIST benchmark, SimVPv2 achieves superior performance compared to SimVP, with fewer FLOPs, about half the training time, and 60% faster inference efficiency. Extensive experiments across eight diverse datasets, including real-world tasks such as traffic forecasting and climate prediction, further demonstrate that SimVPv2 offers a powerful yet straightforward solution, achieving robust generalization across various spatiotemporal learning scenarios. We believe the proposed SimVPv2 can serve as a solid baseline to benefit the spatiotemporal predictive learning community.

研究动机与目标

• 在不使用循环或变换器的情况下，倡导一个简单的、完全卷积的时空预测基线。
• 展示一种轻量级的自编码器风格架构能够有效地对过去帧进行编码并转译为未来帧。
• 引入变体（gSTA 与 Inception-Unet）以提升性能，同时保持训练和推理的高效性。
• 在多个数据集上提供对比一致、公平的评估，与循环网络和基于 CNN 的基线进行比较。

提出的方法

• 采用纯卷积的编码-翻译-解码架构，将过去帧映射到未来帧。
• 鼓励按帧的时空编码，使用一个在多帧特征堆叠上运行的共享时序翻译器。
• 引入两种时空翻译器变体：（i）Inception-Unet 翻译器，具有多分支、大核时序处理；（ii）使用分解的大核来模拟注意力的门控时空注意力（gSTA）翻译器。
• 端到端训练，使用标准均方误差损失，且不使用额外技巧或对抗策略。
• 在 Moving MNIST、TaxiBJ、WeatherBench、Caltech Pedestrian 以及基于 KITTI 的场景上进行评估，以将效率与准确性与最先进方法进行对比。

实验结果

研究问题

• RQ1 Can a simple CNN-based CNN-CNN-CNN framework achieve competitive spatiotemporal predictive performance without recurrence or attention tricks?
• RQ2 Do variants with Inception-style temporal modules or gated spatiotemporal attention offer meaningful accuracy and efficiency gains over the baseline?
• RQ3 How does SimVPv2 generalize across diverse datasets and prediction horizons compared to recurrent and transformer-based methods?
• RQ4 What is the trade-off between training time, inference speed, and predictive quality for SimVPv2 and its variants?

主要发现

方法	Flops (G) ↓	训练时间 ≈ (s) ↓	推理效率 ↑	MSE ↓	MAE ↓	SSIM ↑
ConvLSTM-S	14.45	190	7.50	46.26 ± 0.26	142.18 ± 0.61	0.878 ± 0.001
PhyDNet	15.33	452	4.62	35.68 ± 0.40	96.70 ± 0.29	0.917 ± 0.000
MAU	17.79	535	3.08	30.64 ± 0.10	88.17 ± 0.35	0.928 ± 0.001
SimVP+IncepU	19.43	261	27.15	32.22 ± 0.02	89.19 ± 0.33	0.927 ± 0.000
SimVP+gSTA-S	16.53	156	44.09	26.60 ± 0.02	77.32 ± 0.22	0.940 ± 0.000
ConvLSTM-L	127.01	879	6.24	29.88 ± 0.17	95.05 ± 0.25	0.925 ± 0.000
PredRNN	115.95	869	3.97	25.04 ± 0.08	76.26 ± 0.29	0.944 ± 0.000
PredRNN++	171.73	1280	3.71	22.45 ± 0.36	69.70 ± 0.25	0.950 ± 0.000
MIM	179.18	1388	3.08	23.66 ± 0.20	74.37 ± 0.46	0.946 ± 0.000
E3D-LSTM	298.87	2693	3.73	36.19 ± 0.20	78.64 ± 0.35	0.932 ± 0.000
CrevNet	270.68	1166	1.01	30.15 ± 1.61	86.28 ± 2.65	0.935 ± 0.003
PredRNNv2	116.59	899	3.49	27.73 ± 0.08	82.17 ± 0.33	0.937 ± 0.000
SimVP+gSTA-S × 10	16.53	1560	44.09	15.05 ± 0.03	49.80 ± 0.10	0.967 ± 0.000
SimVP+gSTA-S × 5	16.53	780	44.09	16.47 ± 0.02	53.24 ± 0.04	0.964 ± 0.000
SimVP+gSTA-S × 3	16.53	468	44.09	22.37 ± 0.06	67.52 ± 0.03	0.951 ± 0.000
SimVP+gSTA-L	152.20	796	21.23	21.81 ± 0.03	66.43 ± 0.04	0.952 ± 0.000

• SimVP variants achieve competitive or superior MSE/MAE/SSIM compared to state-of-the-art recurrent models on Moving MNIST.
• gSTA variants provide strong gains in both prediction quality and inference efficiency, often reaching higher SSIM with lower MSE/MAE than baselines.
• On TaxiBJ, SimVP+gSTA substantially improves over IncepU and other baselines, demonstrating effectiveness on traffic forecasting tasks.
• Across standard benchmarks, SimVP variants show favorable training-time and inference-speed trade-offs, often with significantly faster inference than recurrent models.
• Trained with modest epochs or scaled epochs, SimVP variants still reach competitive performance with reduced computational cost.
• The approach emphasizes simplicity and generalizability, suggesting SimVPv2 as a strong, easy-to-use baseline for spatiotemporal predictive learning.