[Paper Review] Diffusion Models for Black-Box Optimization

Siddarth Krishnamoorthy, Satvik Mehul Mashkaria+1 more|arXiv (Cornell University)|Jun 12, 2023

Neural Networks and Applications4 citations

TL;DR

This paper proposes Denoising Diffusion Optimization Models (DDOM), a novel inverse approach for offline black-box optimization that uses conditional diffusion models to map function values to input points. By reweighting the dataset to prioritize high-value points and applying classifier-free guidance at test time, DDOM achieves state-of-the-art performance on the Design-Bench benchmark, outperforming both forward and inverse baselines on average.

ABSTRACT

The goal of offline black-box optimization (BBO) is to optimize an expensive black-box function using a fixed dataset of function evaluations. Prior works consider forward approaches that learn surrogates to the black-box function and inverse approaches that directly map function values to corresponding points in the input domain of the black-box function. These approaches are limited by the quality of the offline dataset and the difficulty in learning one-to-many mappings in high dimensions, respectively. We propose Denoising Diffusion Optimization Models (DDOM), a new inverse approach for offline black-box optimization based on diffusion models. Given an offline dataset, DDOM learns a conditional generative model over the domain of the black-box function conditioned on the function values. We investigate several design choices in DDOM, such as re-weighting the dataset to focus on high function values and the use of classifier-free guidance at test-time to enable generalization to function values that can even exceed the dataset maxima. Empirically, we conduct experiments on the Design-Bench benchmark and show that DDOM achieves results competitive with state-of-the-art baselines.

Motivation & Objective

• To address the limitations of existing offline black-box optimization (BBO) methods, particularly the instability and mode collapse in GAN-based inverse models and the poor generalization of surrogate-based forward models.
• To develop a stable, scalable inverse approach that directly learns the mapping from function values to input points in high-dimensional spaces.
• To enable generalization beyond the maximum observed function value in the dataset using test-time conditioning techniques.
• To improve sample quality and diversity in inverse modeling by resolving the diversity-quality tradeoff inherent in conditional diffusion models.

Proposed method

• DDOM trains a conditional diffusion model to learn a generative distribution over the input domain given a function value, modeling the inverse mapping from output to input.
• A reweighted objective function is used during training to downweight low-value datapoints, focusing the model on high-performing regions of the input space.
• Classifier-free guidance is applied at test time, allowing dynamic control over the strength of conditioning on the target function value to generate high-quality candidates.
• The model decomposes the score function into unconditional and conditional components, enabling disentangled control over generation quality and adherence to conditioning.
• The method is evaluated on the Design-Bench suite, a diverse benchmark of real-world offline BBO tasks with varying dimensions, data sizes, and domains.
• Sampling is performed by conditioning the diffusion model on high function values, including values exceeding the dataset maximum, via classifier-free guidance.

Figure 2 : Branin function

Experimental results

Research questions

• RQ1 Can conditional diffusion models effectively learn the inverse mapping from function values to inputs in high-dimensional offline BBO settings?
• RQ2 How can dataset reweighting improve optimization performance by focusing on high-value regions without sacrificing model stability?
• RQ3 To what extent can classifier-free guidance enhance generalization in offline BBO, especially for function values outside the observed range?
• RQ4 How does DDOM compare to both forward surrogate-based and inverse GAN-based methods in terms of sample quality and optimization performance?

Key findings

• DDOM achieves state-of-the-art performance on the Design-Bench benchmark, ranking first on average with a mean rank of 2.8 across all tasks.
• The model demonstrates high training stability and robustness, outperforming GAN-based inverse models like MINs, which suffer from mode collapse and training instability.
• Classifier-free guidance enables effective generalization to function values exceeding the maximum in the dataset, a key advantage over prior inverse methods.
• Dataset reweighting significantly improves optimization performance by prioritizing high-value points during training, enhancing the model’s ability to discover optimal inputs.

(a) $\mathcal{D}_{\text{GMM}}$