[Paper Review] AutonoML: Towards an Integrated Framework for Autonomous Machine Learning

David Jacob Kedziora, Katarzyna Musiał+1 more|arXiv (Cornell University)|Dec 23, 2020

Machine Learning and Data Classification10 citations

TL;DR

This paper proposes AutonoML, an integrated framework for autonomous machine learning that unifies hyperparameter optimization, neural architecture search, automated feature engineering, meta-learning, dynamic adaptation, and multi-objective evaluation into a cohesive system. The key contribution is a conceptual architecture demonstrating how these components can be synthesized into a self-adapting, minimally supervised ML lifecycle system.

ABSTRACT

Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence, stimulated by advances in optimisation techniques and their impact on selecting ML models/algorithms. Central to this drive is the appeal of engineering a computational system that both discovers and deploys high-performance solutions to arbitrary ML problems with minimal human interaction. Beyond this, an even loftier goal is the pursuit of autonomy, which describes the capability of the system to independently adjust an ML solution over a lifetime of changing contexts. However, these ambitions are unlikely to be achieved in a robust manner without the broader synthesis of various mechanisms and theoretical frameworks, which, at the present time, remain scattered across numerous research threads. Accordingly, this review seeks to motivate a more expansive perspective on what constitutes an automated/autonomous ML system, alongside consideration of how best to consolidate those elements. In doing so, we survey developments in the following research areas: hyperparameter optimisation, multi-component models, neural architecture search, automated feature engineering, meta-learning, multi-level ensembling, dynamic adaptation, multi-objective evaluation, resource constraints, flexible user involvement, and the principles of generalisation. We also develop a conceptual framework throughout the review, augmented by each topic, to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. Ultimately, we conclude that the notion of architectural integration deserves more discussion, without which the field of automated ML risks stifling both its technical advantages and general uptake.

Motivation & Objective

• To address the fragmented state of AutoML research by proposing a holistic integration of diverse mechanisms into a single autonomous framework.
• To identify architectural integration as a critical but underexplored challenge in advancing from AutoML to true AutonoML.
• To synthesize existing research in hyperparameter optimization, neural architecture search, automated feature engineering, meta-learning, and dynamic adaptation into a unified conceptual model.
• To emphasize the need for system-level design that supports lifelong learning, resource constraints, and flexible human-AI interaction.
• To stimulate discussion on building complex adaptive systems capable of autonomous, evolving machine learning deployment.

Proposed method

• Develops a conceptual framework that evolves from individual ML components to ML pipelines, then to pipeline bundles and adaptive systems.
• Integrates sequential model-based optimization (SMBO) and Bayesian optimization for hyperparameter and neural architecture search.
• Incorporates meta-learning to transfer knowledge across past ML experiments and improve search efficiency.
• Enables dynamic adaptation through mechanisms for concept drift detection and continuous learning in streaming data environments.
• Supports multi-objective evaluation and resource-aware operations to balance performance, latency, and computational cost.
• Facilitates human-AI collaboration through flexible user involvement, including visualizations for model transparency and user control.

Experimental results

Research questions

• RQ1 How can disparate AutoML components—such as hyperparameter optimization, neural architecture search, and automated feature engineering—be coherently integrated into a unified system architecture?
• RQ2 What architectural principles are necessary to enable lifelong, autonomous learning in machine learning systems under changing data and environment conditions?
• RQ3 How can meta-knowledge from prior ML experiments be effectively leveraged to accelerate and improve the performance of new learning tasks?
• RQ4 What role does multi-objective optimization play in balancing model performance, resource usage, and deployment constraints in autonomous systems?
• RQ5 How can human expertise and user control be meaningfully embedded within an autonomous ML pipeline without undermining system autonomy?

Key findings

• The integration of multiple AutoML components into a single, cohesive framework is technically feasible and represents a necessary step toward true AutonoML.
• Architectural integration is identified as the central challenge preventing the field from advancing beyond automated model selection to fully autonomous, self-adapting systems.
• Dynamic adaptation and concept drift handling are essential for enabling continuous learning in real-world, evolving environments.
• Meta-learning significantly improves search efficiency by reusing knowledge from previous experiments, reducing the need for exhaustive retraining.
• Multi-objective evaluation and resource-aware optimization are critical for deploying models in real-world systems with hardware and latency constraints.
• User involvement, particularly through visualizations of data, feature engineering, and model behavior, is essential for trust and system transparency, even in autonomous systems.