[论文解读] Flipped Classroom: Effective Teaching for Time Series Forecasting

Philipp Teutsch, Patrick Mäder|arXiv (Cornell University)|Oct 17, 2022

Time Series Analysis and Forecasting被引用 4

一句话总结

本文提出了一种新颖的课程学习（CL）框架，用于在时间序列预测中训练序列到序列的RNN，通过引入逐步增加的训练规模和概率性迭代规模课程，克服教师强制（TF）和自由运行（FR）训练中的暴露偏差与不稳定性。该方法在六个混沌动力系统上实现了最高81%的NRMSE降低，并提升了预测稳定性，始终优于TF和FR基线方法。

ABSTRACT

Sequence-to-sequence models based on LSTM and GRU are a most popular choice for forecasting time series data reaching state-of-the-art performance. Training such models can be delicate though. The two most common training strategies within this context are teacher forcing (TF) and free running (FR). TF can be used to help the model to converge faster but may provoke an exposure bias issue due to a discrepancy between training and inference phase. FR helps to avoid this but does not necessarily lead to better results, since it tends to make the training slow and unstable instead. Scheduled sampling was the first approach tackling these issues by picking the best from both worlds and combining it into a curriculum learning (CL) strategy. Although scheduled sampling seems to be a convincing alternative to FR and TF, we found that, even if parametrized carefully, scheduled sampling may lead to premature termination of the training when applied for time series forecasting. To mitigate the problems of the above approaches we formalize CL strategies along the training as well as the training iteration scale. We propose several new curricula, and systematically evaluate their performance in two experimental sets. For our experiments, we utilize six datasets generated from prominent chaotic systems. We found that the newly proposed increasing training scale curricula with a probabilistic iteration scale curriculum consistently outperforms previous training strategies yielding an NRMSE improvement of up to 81% over FR or TF training. For some datasets we additionally observe a reduced number of training iterations. We observed that all models trained with the new curricula yield higher prediction stability allowing for longer prediction horizons.

研究动机与目标

• 解决在时间序列预测中，基于RNN的序列到序列模型在教师强制（TF）训练中的暴露偏差问题，以及自由运行（FR）训练中的不稳定性问题。
• 开发课程学习（CL）策略，通过缩放训练和迭代调度，系统性地平衡训练稳定性与泛化能力。
• 在来自混沌动力系统的挑战性连续时间序列数据上，评估新课程策略的有效性。
• 识别在NRMSE和预测时域稳定性方面，CL策略优于标准TF和FR基线的条件。
• 探索课程参数化与概率性调度对收敛性和鲁棒性的影响。

提出的方法

• 提出两种新型课程学习策略：逐步增加的训练规模课程和概率性迭代规模课程。
• 引入CL-ITF-P和CL-ITF-D，分别通过几何分布和确定性衰减动态调整教师强制间隙。
• 采用课程函数随时间逐步减小TF间隙，以受控且渐进的方式从TF过渡到FR。
• 使用六个源自混沌系统（如Lorenz、Rössler）的基准数据集，评估不同训练制度下的模型性能。
• 仅引入一个额外超参数以控制课程节奏，实现一致的性能提升。
• 在多步预测任务中，以NRMSE和预测时域稳定性作为主要评估指标。

实验结果

研究问题

• RQ1 课程学习策略是否能在不牺牲训练稳定性的情况下，有效缓解基于RNN的时间序列预测中的暴露偏差？
• RQ2 与教师强制和自由运行基线相比，逐步增加的训练规模和概率性迭代规模课程在NRMSE和预测稳定性方面表现如何？
• RQ3 课程参数化在实现混沌时间序列上稳健收敛与提升泛化能力方面起到何种作用？
• RQ4 为何某些数据集更受益于TF，而另一些则更偏好FR？统一的课程策略能否同时超越两者？
• RQ5 新课程在多大程度上实现了更长预测时域的更高精度与稳定性？

主要发现

• 所提出的具有概率性迭代规模调度的逐步增加训练规模课程，在所有测试中均显著优于教师强制和自由运行训练，平均实现高达81%的NRMSE降低。
• 采用新课程训练的模型表现出显著更高的预测稳定性，支持更长时域的可靠预测。
• 在某些数据集中，新课程可减少所需训练迭代次数，表明收敛速度更快。
• 概率性变体（CL-ITF-P）由于TF间隙的几何分布，在早期训练中表现出更大方差，但整体性能更优。
• 新课程在多种混沌系统中均表现稳健，在全部六个基准数据集中均观察到性能提升。
• 该方法仅需一个额外超参数控制课程节奏，展现出良好的可集成性与调优便捷性。