[Paper Review] Multi-Fidelity Active Learning with GFlowNets

Álex Hernández-García, Nikita Saxena+3 more|arXiv (Cornell University)|Jun 20, 2023

Machine Learning and Algorithms4 citations

TL;DR

This paper proposes MF-GFN, a novel multi-fidelity active learning framework that leverages GFlowNets to jointly sample both molecular candidates and their optimal evaluation fidelity across multiple oracles with varying costs and accuracies. By training a stochastic policy to explore high-dimensional search spaces efficiently, MF-GFN achieves superior cost-effectiveness and diversity in molecular discovery tasks compared to single-fidelity and existing multi-fidelity baselines.

ABSTRACT

In the last decades, the capacity to generate large amounts of data in science and engineering applications has been growing steadily. Meanwhile, machine learning has progressed to become a suitable tool to process and utilise the available data. Nonetheless, many relevant scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in scientific discovery, we are often faced with the problem of exploring very large, structured and high-dimensional spaces. Moreover, the high fidelity, black-box objective function is often very expensive to evaluate. Progress in machine learning methods that can efficiently tackle such challenges would help accelerate currently crucial areas such as drug and materials discovery. In this paper, we propose a multi-fidelity active learning algorithm with GFlowNets as a sampler, to efficiently discover diverse, high-scoring candidates where multiple approximations of the black-box function are available at lower fidelity and cost. Our evaluation on molecular discovery tasks shows that multi-fidelity active learning with GFlowNets can discover high-scoring candidates at a fraction of the budget of its single-fidelity counterpart while maintaining diversity, unlike RL-based alternatives. These results open new avenues for multi-fidelity active learning to accelerate scientific discovery and engineering design.

Motivation & Objective

• To address the challenge of efficiently exploring large, high-dimensional chemical spaces where high-fidelity evaluations are extremely costly.
• To develop a method that adaptively allocates computational resources across multiple oracles of varying fidelity and cost in scientific discovery.
• To improve upon single-fidelity active learning by enabling joint sampling of both molecular candidates and their optimal evaluation fidelity.
• To enhance diversity and cost-effectiveness in molecular generation tasks such as DNA aptamer design and antimicrobial peptide discovery.
• To demonstrate the advantages of GFlowNets in multi-fidelity settings through empirical evaluation on synthetic and real-world scientific tasks.

Proposed method

• The method employs a multi-fidelity surrogate model, such as a Gaussian Process, to estimate the posterior distribution $ p(f_m(x) \mid x, m, \mathcal{D}) $ over function values at different fidelities.
• An acquisition function—specifically max-value entropy search (MES)—is computed using the surrogate model to guide exploration.
• A GFlowNet is trained to sample both candidate molecules $ x $ and their corresponding fidelity levels $ m $, with the acquisition function values used as a reward signal.
• The GFlowNet learns a stochastic policy that generates diverse, high-scoring candidates by modeling the probability of trajectories in a flow network.
• Samples are generated from the trained GFlowNet, and the top $ B $ tuples $ (x, m) $ are selected for evaluation on the corresponding oracle.
• The process iteratively updates the dataset $ \mathcal{D} $ with new evaluations and retrains the surrogate and GFlowNet models.

Experimental results

Research questions

• RQ1 Can GFlowNets be effectively extended to multi-fidelity active learning for scientific discovery?
• RQ2 How does joint sampling of candidates and fidelities improve cost-effectiveness and diversity compared to single-fidelity methods?
• RQ3 What is the impact of varying oracle cost ratios on the performance of multi-fidelity active learning with GFlowNets?
• RQ4 How does MF-GFN compare to existing multi-fidelity Bayesian optimization and reinforcement learning baselines in terms of sample efficiency and diversity?
• RQ5 Can MF-GFN maintain high performance even when low-fidelity oracles are relatively expensive, approaching the cost of high-fidelity ones?

Key findings

• MF-GFN outperforms single-fidelity GFlowNet (SF-GFN) and multi-fidelity PPO (MF-PPO) in terms of cost-effectiveness across all tested tasks, including DNA aptamer generation and molecular design.
• Even with a low-fidelity oracle cost ratio as high as 1:2 (i.e., $ \lambda_1 = 1 $, $ \lambda_M = 20 $), MF-GFN maintains a significant performance advantage over SF-GFN and MF-PPO.
• The method achieves higher diversity in generated candidates without compromising on the quality of top-scoring molecules, as measured by standard diversity metrics in molecular generation tasks.
• On the DNA aptamer task, MF-GFN consistently discovers higher-scoring sequences with fewer high-fidelity evaluations than baseline methods.
• Empirical results show that the performance gap between MF-GFN and baselines increases as the cost difference between oracles grows, confirming the method’s strength in low-cost, high-fidelity regimes.
• The ablation study confirms that the joint sampling of candidates and fidelities via GFlowNet is a key contributor to the method’s success, not just the use of multi-fidelity surrogates.