[Paper Review] Tabular Few-Shot Generalization Across Heterogeneous Feature Spaces

Max Zhu, Katarzyna Kobalczyk+5 more|arXiv (Cornell University)|Nov 16, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

FLAT proposes a novel few-shot learning framework for tabular data that generalizes across datasets with heterogeneous feature spaces by using a meta-learning approach with dataset and column embeddings. It leverages a Graph Attention Network (GAT) decoder to enable permutation-invariant, dynamic feature relationship modeling, achieving state-of-the-art performance on 118 UCI datasets, especially in low-shot and imbalanced settings.

ABSTRACT

Despite the prevalence of tabular datasets, few-shot learning remains under-explored within this domain. Existing few-shot methods are not directly applicable to tabular datasets due to varying column relationships, meanings, and permutational invariance. To address these challenges, we propose FLAT-a novel approach to tabular few-shot learning, encompassing knowledge sharing between datasets with heterogeneous feature spaces. Utilizing an encoder inspired by Dataset2Vec, FLAT learns low-dimensional embeddings of datasets and their individual columns, which facilitate knowledge transfer and generalization to previously unseen datasets. A decoder network parametrizes the predictive target network, implemented as a Graph Attention Network, to accommodate the heterogeneous nature of tabular datasets. Experiments on a diverse collection of 118 UCI datasets demonstrate FLAT's successful generalization to new tabular datasets and a considerable improvement over the baselines.

Motivation & Objective

• To address the lack of effective few-shot learning methods for tabular data, which often have limited labeled examples and varying feature spaces.
• To overcome challenges in tabular few-shot learning, including permutational invariance, lack of shared feature semantics, and heterogeneous column relationships.
• To enable knowledge transfer across datasets with different sets of features by learning shared low-dimensional embeddings of datasets and their columns.
• To design a meta-learning framework that generalizes to unseen datasets with minimal labeled examples, even when feature spaces differ significantly.
• To improve performance on imbalanced datasets, a common real-world challenge in healthcare and finance.

Proposed method

• FLAT employs a meta-learning framework where a meta network generates weights for a target network based on a few-shot support set.
• A dataset encoder inspired by Dataset2Vec learns low-dimensional embeddings of entire datasets and individual columns, enabling knowledge sharing across heterogeneous feature spaces.
• The target network is a Graph Attention Network (GAT) that models inter-column relationships dynamically, ensuring permutation invariance and adaptability to varying column sets.
• The decoder network generates task-specific weights for the GAT using the embedded meta-data, allowing dynamic adaptation to new tasks.
• The model is trained end-to-end on a diverse set of 118 UCI datasets, with the meta network learning to generalize across tasks with different feature compositions.
• FLATadapt, an improved variant, fine-tunes the meta network using a perturbed grid synthetic task, improving decision boundary alignment and performance on low-shot tasks.

Figure 1: Overview of the FLAT architecture, highlighting its three key components: (1—dataset encoder $\mathcal{F}$ with the column encoder $\mathcal{G}$ , (2)—weight generating decoder network $\mathcal{H}$ and (3)—the target GAT network $\Phi$ . (1) and (2) together form the meta network.

Experimental results

Research questions

• RQ1 Can a few-shot learning model generalize effectively across tabular datasets with entirely different feature spaces?
• RQ2 How does the model perform when the meta-dataset is highly imbalanced, a common real-world scenario?
• RQ3 To what extent does the model benefit from structural similarity between meta and target tasks?
• RQ4 Can the model make accurate predictions with only a single labeled example per class?
• RQ5 How does the use of graph attention and shared embeddings improve generalization compared to standard few-shot baselines?

Key findings

• FLAT significantly outperforms existing baselines across 118 UCI datasets at few-shot settings with 3, 5, and 10 labeled examples per class.
• FLAT achieves up to 1.25 percentage point improvement over its baseline via the FLATadapt fine-tuning strategy, especially in low-shot regimes.
• FLAT demonstrates strong robustness on imbalanced meta-datasets, outperforming baselines even when the positive and negative classes are highly skewed.
• The model successfully learns meaningful decision boundaries with only one labeled example per class, showing effective prior knowledge acquisition.
• FLAT achieves the highest average accuracy among all baselines at N^{meta}=3, 5, 10, and remains competitive at N^{meta}=15.
• The performance gain is most pronounced when meta-tasks share structural relationships with downstream tasks, confirming the importance of task similarity in few-shot generalization.

Figure 2: Median model ranks based on accuracy over 29 medical datasets (Left) and all 118 UCI datasets (Right)