[Paper Review] Towards a more efficient representation of imputation operators in TPOT

Unai Garciarena, Alexander Mendiburu+1 more|arXiv (Cornell University)|Jan 13, 2018

Evolutionary Algorithms and Applications16 references3 citations

TL;DR

This paper proposes a strongly typed genetic programming (GP) approach to integrate imputation methods (IMs) more efficiently into TPOT, a tool for automated machine learning pipeline optimization. By redefining IMs as constrained operators placed only at the pipeline start, the method drastically reduces unfeasible pipelines, improving search efficiency and accuracy while slightly increasing runtime due to a larger but more effective search space.

ABSTRACT

Automated Machine Learning encompasses a set of meta-algorithms intended to design and apply machine learning techniques (e.g., model selection, hyperparameter tuning, model assessment, etc.). TPOT, a software for optimizing machine learning pipelines based on genetic programming (GP), is a novel example of this kind of applications. Recently we have proposed a way to introduce imputation methods as part of TPOT. While our approach was able to deal with problems with missing data, it can produce a high number of unfeasible pipelines. In this paper we propose a strongly-typed-GP based approach that enforces constraint satisfaction by GP solutions. The enhancement we introduce is based on the redefinition of the operators and implicit enforcement of constraints in the generation of the GP trees. We evaluate the method to introduce imputation methods as part of TPOT. We show that the method can notably increase the efficiency of the GP search for optimal pipelines.

Motivation & Objective

• To address the inefficiency of previous TPOT methods that generated a high number of unfeasible pipelines when incorporating imputation methods (IMs).
• To improve the search efficiency of genetic programming (GP) in TPOT by enforcing structural constraints on IM integration.
• To enable IMs to be evolved as part of the pipeline optimization process, rather than being fixed at the start.
• To maintain compatibility with TPOT’s existing grammar-based GP framework while enhancing its expressiveness for handling incomplete data.
• To lay the groundwork for extending the framework to other constrained ML operators, such as dimensionality reduction or feature scaling.

Proposed method

• Redefine imputation methods as special, constrained operators within the GP grammar, ensuring they are only placed at the beginning of the pipeline.
• Introduce a conceptual constraint that enforces exactly one IM at the start of every pipeline, preventing invalid placements later in the pipeline.
• Use strongly typed GP to enforce type consistency and prevent invalid pipeline structures during tree generation.
• Modify the grammar to embed constraints that disallow multiple or misplaced IMs, ensuring only valid, imputed data flows to downstream components.
• Leverage TPOT’s existing GP framework with minimal changes to support the new grammar structure.
• Ensure that downstream components always receive imputed data by preventing IMs from being inserted after the initial imputation step.

Experimental results

Research questions

• RQ1 How can imputation methods be efficiently integrated into TPOT’s genetic programming-based pipeline search without producing a high number of unfeasible pipelines?
• RQ2 What impact does enforcing a strict placement constraint for imputation methods (only at the pipeline start) have on search efficiency and solution quality?
• RQ3 Can a strongly typed GP grammar with embedded constraints improve the convergence and performance of TPOT in the presence of missing data?
• RQ4 How does the proposed method compare to the previous approach of treating IMs as simple preprocessing units in terms of accuracy and computational cost?
• RQ5 To what extent can this constraint-based grammar extension be generalized to other ML operators in automated pipeline optimization?

Key findings

• The enhanced TPOT with the new grammar consistently achieved higher mean accuracy across all 9 datasets compared to the previous approach, especially in early evaluation stages.
• The number of unfeasible pipelines was drastically reduced due to the enforced constraint that IMs must be placed only at the beginning of the pipeline.
• Despite a slight increase in runtime due to a larger number of evaluated pipelines, the improved search efficiency led to faster convergence to better solutions.
• The method reduced the bias toward randomly generated valid pipelines, allowing the evolutionary process to explore more diverse and effective pipeline structures.
• The results show that integrating IMs into the evolutionary process leads to better model performance than fixed, pre-processing imputation.
• The approach is extensible and can be adapted to incorporate other constrained ML operators, such as feature scaling or dimensionality reduction, into the pipeline evolution process.