[论文解读] Deep Neural Networks and Tabular Data: A Survey

Vadim Borisov, Tobias Leemann+4 more|arXiv (Cornell University)|Oct 5, 2021

Explainable Artificial Intelligence (XAI)被引用 29

一句话总结

深度学习在异构表格数据上的全面综述，提出一个分类法（数据转换、专用架构、正则化），综述生成与可解释性，并且一个经验基准显示梯度提升树在监督任务上往往优于深度模型。

ABSTRACT

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous data sets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data, and we also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas, while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with eleven deep learning approaches across five popular real-world tabular data sets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

研究动机与目标

• 对表格数据的深度学习在监督、无监督、数据生成和可解释性任务等方面的现有文献进行全面综述。
• 提出一个统一的分类法（数据转换、专用架构、正则化）以组织异构表格数据的方法。
• 总结表格数据生成技术以及对表格数据的深度模型的可解释性方法。
• 在真实世界数据集上提供传统机器学习方法与深度学习方法的广泛实证比较，并提供开放基准以便复制。

提出的方法

• 引入一个面向表格数据的深度学习方法的统一分类法（数据转换、专用架构、正则化）。
• 对分类特征和数值特征的数据转换技术进行综述，包括单维和多维编码。
• 描述专用架构，包括混合模型和基于变换器的模型，用于表格数据。
• 回顾用于提升表格数据上深度模型的正则化策略。
• 讨论表格数据生成方法及对生成质量的评估。
• 提出对表格数据上深度模型的解释机制以及一个经验基准框架。

实验结果

研究问题

• RQ1 将深度学习应用于异构表格数据的主要研究方向和方法论有哪些？
• RQ2 在真实世界数据集上，表格数据的深度学习方法与传统方法相比如何？
• RQ3 在表格数据中使用深度学习进行推理、生成和可解释性方面存在哪些未知挑战与未来方向？
• RQ4 统一的分类法能否帮助从业者为表格数据任务选择合适的方法？

主要发现

• 梯度提升树集成在对经过测试的数据集上的监督表格任务中仍然在很大程度上超过深度学习模型。
• 相对于强大的基线树模型，表格数据的深度学习进展似乎停滞，表明仍有方法学进步的空间。
• 本论文提供了一个开放的基准和代码，以复现和扩展比较结果。
• 一项广泛的调查确定了表格数据的核心挑战：数据质量、特征依赖不规则、预处理依赖以及逐特征敏感性。