[论文解读] Learning Generative ConvNet with Continuous Latent Factors by Alternating Back-Propagation.

Tian Han, Yang Lu+2 more|arXiv (Cornell University)|Jun 28, 2016

Generative Adversarial Networks and Image Synthesis参考文献 53被引用 4

一句话总结

本文提出一种用于训练具有连续潜在因子的生成性卷积神经网络（ConvNets）的交替反向传播算法，其中潜在变量的推断与参数更新均通过反向传播实现。该方法通过迭代地执行推断与学习的反向传播步骤，联合优化潜在因子与模型参数，从而实现逼真的图像与声音生成。

ABSTRACT

The supervised learning of the discriminative convolutional neural network (ConvNet) is powered by back-propagation on the parameters. In this paper, we show that the unsupervised learning of a popular top-down generative ConvNet model with latent continuous factors can be accomplished by a learning algorithm that consists of alternatively performing back-propagation on both the latent factors and the parameters. The model is a non-linear generalization of factor analysis, where the high-dimensional observed data, is assumed to be the noisy version of a vector generated by a non-linear transformation of a low-dimensional vector of continuous latent factors. Furthermore, it is assumed that these latent factors follow known independent distributions, such as standard normal distributions, and the non-linear transformation is assumed to be parametrized by a top-down ConvNet, which is capable of approximating the highly non-linear mapping from the latent factors to the image. We explore a simple and natural learning algorithm for this model that alternates between the following two steps: (1) inferring the latent factors by Langevin dynamics or gradient descent, and (2) updating the parameters of the ConvNet by gradient descent. Step (1) is based on the gradient of the reconstruction error with respect to the latent factors, which is available by back-propagation. We call this step inferential back-propagation. Step (2) is based on the gradient of the reconstruction error with respect to the parameters, and is also obtained by back-propagation. We refer to this step as learning back-propagation. The code for inferential back-propagation is actually part of the code for learning back-propagation, and thus the inferential back-propagation is actually a by-product of the learning back-propagation. We show that our algorithm can learn realistic generative models of images and sounds.

研究动机与目标

• 开发一种用于生成性卷积神经网络的无监督学习算法，将高维数据建模为低维连续潜在因子的非线性变换。
• 解决从潜在空间到观测数据的复杂非线性映射在自顶向下生成模型中训练的挑战。
• 通过交替使用反向传播进行潜在因子推断与参数优化，实现端到端学习。
• 证明可将反向传播框架统一应用于参数学习与潜在变量推断，而无需单独实现。

提出的方法

• 该模型假设观测数据是连续潜在因子非线性变换的噪声版本，且该变换由一个自顶向下的卷积神经网络参数化。
• 潜在因子通过拉夫林动力学（Langevin dynamics）或基于潜在变量重构误差梯度的梯度下降法进行推断，该过程称为推断反向传播。
• 模型参数通过在重构误差上使用标准反向传播进行梯度下降更新，该过程称为学习反向传播。
• 算法在推断反向传播（更新潜在因子）与学习反向传播（更新模型参数）之间交替进行，利用共享的计算图。
• 推断反向传播作为学习反向传播计算图的副产品实现，从而实现高效的联合优化。
• 该方法通过使用深层卷积神经网络对潜在因子到数据的非线性映射进行建模，将概率主成分分析（probabilistic PCA）推广至非线性场景。

实验结果

研究问题

• RQ1 反向传播能否有效扩展至模型参数之外，涵盖生成性卷积神经网络中潜在变量的推断？
• RQ2 与现有方法相比，交替反向传播算法在学习逼真图像与声音生成模型方面的表现如何？
• RQ3 连续潜在因子在非线性生成模型中对实现高保真重建与生成起什么作用？
• RQ4 能否在不单独实现的情况下，使用相同的反向传播框架同时完成参数学习与潜在因子推断？
• RQ5 在重构质量方面，该交替算法与标准变分推断或其他深度生成模型相比表现如何？

主要发现

• 所提出的交替反向传播算法成功训练了具有连续潜在因子的生成性卷积神经网络模型，实现了高质量的图像与声音生成。
• 源自与学习反向传播相同计算图的推断反向传播，实现了高效且可微的潜在因子推断。
• 通过迭代优化潜在因子与模型参数，该方法实现了逼真样本的生成。
• 该算法表明，反向传播可自然地扩展至潜在变量推断，简化了实现并提升了训练效率。
• 由自顶向下卷积神经网络支持的非线性映射能力，使模型能够捕捉超越线性因子分析的复杂数据分布。
• 该框架支持图像与音频数据的生成，展现出在多种模态间的广泛适用性。