[论文解读] Unbiasing Truncated Backpropagation Through Time

Corentin Tallec, Yann Ollivier|arXiv (Cornell University)|May 23, 2017

Topic Modeling参考文献 3被引用 53

一句话总结

ARTBP 引入随机、可变长度的截断并附带补偿因子，以在截断的 BPTT 中提供无偏梯度估计，同时保持在线适用性并改进相较于标准截断 BPTT 的收敛。在 Penn Treebank 字符级建模上，ARTBP 相较于截断 BPTT 在验证/测试性能上略有提升。

ABSTRACT

Truncated Backpropagation Through Time (truncated BPTT) is a widespread method for learning recurrent computational graphs. Truncated BPTT keeps the computational benefits of Backpropagation Through Time (BPTT) while relieving the need for a complete backtrack through the whole data sequence at every step. However, truncation favors short-term dependencies: the gradient estimate of truncated BPTT is biased, so that it does not benefit from the convergence guarantees from stochastic gradient theory. We introduce Anticipated Reweighted Truncated Backpropagation (ARTBP), an algorithm that keeps the computational benefits of truncated BPTT, while providing unbiasedness. ARTBP works by using variable truncation lengths together with carefully chosen compensation factors in the backpropagation equation. We check the viability of ARTBP on two tasks. First, a simple synthetic task where careful balancing of temporal dependencies at different scales is needed: truncated BPTT displays unreliable performance, and in worst case scenarios, divergence, while ARTBP converges reliably. Second, on Penn Treebank character-level language modelling, ARTBP slightly outperforms truncated BPTT.

研究动机与目标

• 动机：解释截断 BPTT 中的偏差问题，以及在训练 RNN 时需要无偏梯度估计。
• 提出 ARTBP 作为一种在保留截断带来的计算优势的同时实现无偏性的方法。
• 推导对在反向传播中随机截断进行补偿的重权重方案。
• 给出 ARTBP 下无偏梯度估计的理论保证。
• 在合成任务和 Penn Treebank 字符级语言建模上对 ARTBP 进行实证验证。

提出的方法

• 将训练序列分割为长度可变、从概率分布中采样的子序列。
• 在反向传播方程中加入补偿因子 1/(1 - c_t) 以确保无偏性（方程 11）。
• 证明 ARTBP 梯度估计是无偏的（命题 1，方程 12-13）。
• 讨论如何选择截断概率 c_t 以在内存和方差之间取得平衡（方程 14）。
• 描述在线实现，在每个子序列后进行更新（第 5 节）。
• 在合成任务和 Penn Treebank 上将 ARTBP 与截断 BPTT 进行比较（第 6 节）。

实验结果

研究问题

• RQ1 Can stochastic, variable-length truncation with appropriate compensation yield unbiased gradient estimates for BPTT?
• RQ2 How does ARTBP trade memory usage and gradient variance relative to fixed-length truncated BPTT?
• RQ3 Do ARTBP and truncated BPTT differ in performance on synthetic tasks requiring multi-timescale dependency learning and real-world language modelling?
• RQ4 What practical guidelines (e.g., choice of c_t) optimize ARTBP’s bias-variance tradeoff for online learning?
• RQ5 Is ARTBP applicable online without backtracking across the entire sequence?

主要发现

• ARTBP provides unbiased gradient estimates despite using variable truncation lengths.
• In synthetic tests, truncated BPTT can diverge due to gradient bias, while ARTBP converges reliably.
• On Penn Treebank character-level language modelling, ARTBP slightly outperforms truncated BPTT in validation and test error.
• ARTBP introduces gradient variance due to stochastic truncations but reduces memory demands, enabling longer effective traces.
• A fixed memory-equivalent truncation (L) is comparable to ARTBP with c_t chosen to yield similar average subsequence length, yet ARTBP often yields better convergence properties in biased scenarios.