[Paper Review] Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction

Benjamin Hilprecht, Carsten Binnig|arXiv (Cornell University)|Jan 3, 2022

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper introduces zero-shot cost models that enable accurate learned cost estimation for unseen databases without requiring query execution on the target database. By leveraging a pre-trained model with a transferable query representation and a novel architecture that decouples database-specific statistics, the approach achieves state-of-the-art accuracy out-of-the-box and further improves with few-shot fine-tuning using minimal additional queries.

ABSTRACT

In this paper, we introduce zero-shot cost models which enable learned cost estimation that generalizes to unseen databases. In contrast to state-of-the-art workload-driven approaches which require to execute a large set of training queries on every new database, zero-shot cost models thus allow to instantiate a learned cost model out-of-the-box without expensive training data collection. To enable such zero-shot cost models, we suggest a new learning paradigm based on pre-trained cost models. As core contributions to support the transfer of such a pre-trained cost model to unseen databases, we introduce a new model architecture and representation technique for encoding query workloads as input to those models. As we will show in our evaluation, zero-shot cost estimation can provide more accurate cost estimates than state-of-the-art models for a wide range of (real-world) databases without requiring any query executions on unseen databases. Furthermore, we show that zero-shot cost models can be used in a few-shot mode that further improves their quality by retraining them just with a small number of additional training queries on the unseen database.

Motivation & Objective

• To address the high cost of collecting training data via query execution for every new database in workload-driven learned cost models.
• To enable accurate cost estimation for unseen databases without executing any queries on them.
• To design a model architecture and query representation that generalize across diverse databases and data characteristics.
• To demonstrate that zero-shot cost models can outperform existing workload-driven models even without any database-specific training.
• To enable few-shot fine-tuning with minimal additional query executions to further improve accuracy.

Proposed method

• Propose a new model architecture that separates database-specific statistics (e.g., table cardinalities) from query plan features, enabling transferability.
• Introduce a transferable query representation that encodes query plans using database-agnostic features such as tuple width and operator selectivity.
• Pre-train the model on a diverse corpus of databases and workloads to learn general cost estimation patterns.
• Use the pre-trained model as a zero-shot estimator for any new, unseen database by feeding it the transferable query representation and database statistics.
• Fine-tune the zero-shot model with a small number of query executions on the new database to improve accuracy in few-shot mode.
• Decouple model training from database-specific data by treating data statistics as input features rather than learned components.

Experimental results

Research questions

• RQ1 Can a learned cost model generalize to entirely unseen databases without executing any queries on them?
• RQ2 Can a pre-trained model with a transferable query representation achieve high accuracy on diverse real-world databases without database-specific training?
• RQ3 How does the performance of zero-shot cost estimation compare to state-of-the-art workload-driven models that require extensive query execution?
• RQ4 To what extent can few-shot fine-tuning further improve the accuracy of zero-shot cost models?
• RQ5 What features and representation techniques are essential for enabling zero-shot generalization across databases?

Key findings

• Zero-shot cost models achieve higher accuracy than state-of-the-art workload-driven models on unseen databases such as IMDB without any query execution on the target database.
• The zero-shot model outperforms workload-driven models even when the latter are trained on over 5 hours of query execution for the IMDB database.
• Few-shot fine-tuning with just a small number of additional queries on the unseen database further improves estimation accuracy beyond the zero-shot baseline.
• The proposed transferable query representation, based on features like tuple width and operator selectivity, enables generalization across diverse databases and data characteristics.
• The model architecture successfully decouples database-specific statistics from learned patterns, allowing transferability without retraining from scratch.
• The approach eliminates the need for repeated, expensive training data collection for every new database or data update, enabling scalable deployment of learned cost models.