[论文解读] Conditional Generative Adversarial Networks for Regression

Karan Aggarwal, Matthieu Kirchmeyer+3 more|arXiv (Cornell University)|May 30, 2019

Gaussian Processes and Bayesian Inference参考文献 8被引用 8

一句话总结

本文探讨了条件生成对抗网络（CGANs）在低维输出回归任务中的应用，提出了一种通过引入输入噪声来更好地反映现实世界不确定性的隐式概率模型。尽管CGANs展现出潜力，但在似然度上表现不如混合密度网络（MDNs），在平均绝对误差上也逊于XGBoost，表明CGANs在实际回归应用中仍需进一步创新。

ABSTRACT

In recent years, impressive progress has been made in the design of implicit probabilistic models via Generative Adversarial Networks (GAN) and its extension, the Conditional GAN (CGAN). Excellent solutions have been demonstrated mostly in image processing applications which involve large, continuous output spaces. There is almost no application of these powerful tools to problems having small dimensional output spaces. Regression problems involving the inductive learning of a map, $y=f(x,z)$, $z$ denoting noise, $f:\mathbb{R}^n imes \mathbb{R}^k ightarrow \mathbb{R}^m$, with $m$ small (e.g., $m=1$ or just a few) is one good case in point. The standard approach to solve regression problems is to probabilistically model the output $y$ as the sum of a mean function $m(x)$ and a noise term $z$; it is also usual to take the noise to be a Gaussian. These are done for convenience sake so that the likelihood of observed data is expressible in closed form. In the real world, on the other hand, stochasticity of the output is usually caused by missing or noisy input variables. Such a real world situation is best represented using an implicit model in which an extra noise vector, $z$ is included with $x$ as input. CGAN is naturally suited to design such implicit models. This paper makes the first step in this direction and compares the existing regression methods with CGAN. We notice however, that the existing methods like mixture density networks (MDN) and XGBoost do quite well compared to CGAN in terms of likelihood and mean absolute error, respectively. Both these methods are comparatively easier to train than CGANs. CGANs need more innovation to have a comparable modeling and ease-of-training with respect to the existing regression solvers. In summary, for modeling uncertainty MDNs are better while XGBoost is better for the cases where accurate prediction is more important.

研究动机与目标

• 研究条件生成对抗网络（CGANs）在输出维度较小的回归问题中的应用，例如m=1或少数几个值。
• 评估CGANs是否能通过使用输入噪声z的隐式生成方法，有效建模回归中的不确定性。
• 从似然度和预测准确性的角度，将CGANs与成熟的回归方法（如混合密度网络MDNs）和XGBoost进行比较。
• 识别CGANs在回归任务中训练时面临的挑战，并评估其相对于现有更简单模型的可行性。

提出的方法

• 将回归建模为条件生成模型 y = f(x, z)，其中x为输入，z为噪声，f映射 R^n × R^k → R^m，且m较小。
• 采用CGAN架构隐式学习条件分布 p(y|x)，避免显式计算似然度。
• 使用生成器网络生成基于x和随机噪声z的输出，同时使用判别器区分真实样本与生成样本。
• 通过对抗方式联合训练生成器与判别器，优化生成结果的保真度与分布一致性。
• 将模型应用于输出空间低维的回归任务，与标准参数化模型进行对比。
• 采用标准评估指标：对数似然度用于不确定性建模，平均绝对误差（MAE）用于预测准确性评估。

实验结果

研究问题

• RQ1 CGANs能否有效建模小输出维度（如m=1）的回归任务？
• RQ2 CGANs在通过似然度建模输出不确定性方面，与MDNs相比表现如何？
• RQ3 CGANs在预测准确性（MAE）方面，与XGBoost相比表现如何？
• RQ4 与更简单的模型相比，使用CGANs进行回归时面临哪些训练挑战和实际限制？
• RQ5 对于存在噪声或缺失输入的真实世界回归任务，CGANs的隐式建模方法是否比显式似然模型更具优势？

主要发现

• CGANs在理论上非常适合通过引入输入噪声z来建模回归不确定性，能够反映现实世界中的随机性。
• 尽管具有理论优势，CGANs在对数似然度上仍劣于MDNs，表明MDNs在不确定性量化方面更为有效。
• XGBoost在平均绝对误差（MAE）上优于CGANs，说明XGBoost在点预测任务中更具准确性。
• 与MDNs和XGBoost相比，CGANs需要更复杂的训练过程和超参数调优，后者更易于训练。
• 研究结论认为，CGANs在建模质量与训练便捷性方面，目前仍无法与现有回归求解器相媲美。
• 在不确定性建模方面，MDNs更优；在高精度预测方面，XGBoost优于CGANs。