[Paper Review] TabGSL: Graph Structure Learning for Tabular Data Prediction

Jay Chiehen Liao, Cheng–Te Li|arXiv (Cornell University)|May 25, 2023

Data Quality and Management4 citations

TL;DR

TabGSL proposes a novel graph structure learning framework for tabular data prediction that jointly learns instance correlations and feature interactions via a contrastive learning module, a transformer-based feature extractor, and a graph neural network. It achieves state-of-the-art performance on 30 benchmark datasets, outperforming both tree-based and recent deep learning models for tabular data.

ABSTRACT

This work presents a novel approach to tabular data prediction leveraging graph structure learning and graph neural networks. Despite the prevalence of tabular data in real-world applications, traditional deep learning methods often overlook the potentially valuable associations between data instances. Such associations can offer beneficial insights for classification tasks, as instances may exhibit similar patterns of correlations among features and target labels. This information can be exploited by graph neural networks, necessitating robust graph structures. However, existing studies primarily focus on improving graph structure from noisy data, largely neglecting the possibility of deriving graph structures from tabular data. We present a novel solution, Tabular Graph Structure Learning (TabGSL), to enhance tabular data prediction by simultaneously learning instance correlation and feature interaction within a unified framework. This is achieved through a proposed graph contrastive learning module, along with transformer-based feature extractor and graph neural network. Comprehensive experiments conducted on 30 benchmark tabular datasets demonstrate that TabGSL markedly outperforms both tree-based models and recent deep learning-based tabular models. Visualizations of the learned instance embeddings further substantiate the effectiveness of TabGSL.

Motivation & Objective

• To address the lack of graph structure learning methods that derive graph topology directly from tabular data without relying on pre-existing graphs.
• To jointly model feature interactions and instance-level correlations in a unified deep learning framework for tabular data.
• To improve tabular data prediction by learning a noise-robust graph structure that enhances representation learning via graph contrastive learning.
• To demonstrate that graph structure learning from tabular data can make deep learning models competitive with or superior to gradient boosted decision trees.

Proposed method

• Proposes a graph contrastive learning module that uses a teacher-student framework to guide the learning of instance-level graph structures from tabular data.
• Employs a transformer-based feature extractor to generate contextualized embeddings for tabular features, capturing inter-feature interactions.
• Integrates a graph neural network (GNN) with the learned graph structure to aggregate neighborhood information and refine instance-level representations.
• Uses end-to-end training to jointly optimize the graph structure learning, feature representation, and prediction head.
• Applies data masking during contrastive learning to create positive and negative views for graph structure refinement.
• Leverages t-SNE visualization to analyze and compare the quality of learned instance embeddings across raw features, SUBLIME, and TabGSL.

Experimental results

Research questions

• RQ1 Can graph structure learning from tabular data improve prediction performance compared to existing deep learning and tree-based models?
• RQ2 How effective is the proposed contrastive learning mechanism in learning meaningful instance-level correlations without prior graph structure?
• RQ3 To what extent do the learned graph structures enhance class separability in the embedding space compared to raw features or existing GSL methods?
• RQ4 Can joint modeling of feature interactions and instance associations lead to better generalization in tabular data prediction?

Key findings

• TabGSL achieves state-of-the-art performance on 30 benchmark tabular datasets, outperforming both gradient boosted decision trees and recent deep learning-based tabular models.
• t-SNE visualizations show that TabGSL embeddings exhibit significantly better class separation than raw features and SUBLIME embeddings, with same-class instances more closely clustered.
• The contrastive learning module effectively guides the graph structure to reflect true instance-level correlations, as evidenced by improved clustering and prediction accuracy.
• The integration of transformer-based feature extraction with graph structure learning enhances representation quality, enabling the GNN to learn more robust and informative node embeddings.
• The method demonstrates that learning graph structures from tabular data alone can yield performance gains comparable to or exceeding that of tree-based models.
• The learned graph structure helps mitigate the negative impact of irrelevant or noisy features by adjusting edge weights based on feature utility and similarity.