[Paper Review] Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting

Aditya Grover, Jiaming Song+5 more|arXiv (Cornell University)|Jun 23, 2019

Generative Adversarial Networks and Image Synthesis62 references47 citations

TL;DR

The paper introduces likelihood-free importance weighting (LFIW) to debias expectations from learned generative models by training a probabilistic classifier to estimate density ratios, improving bias-related metrics and aiding applications like data augmentation and model-based off-policy evaluation.

ABSTRACT

A learned generative model often produces biased statistics relative to the underlying data distribution. A standard technique to correct this bias is importance sampling, where samples from the model are weighted by the likelihood ratio under model and true distributions. When the likelihood ratio is unknown, it can be estimated by training a probabilistic classifier to distinguish samples from the two distributions. We employ this likelihood-free importance weighting method to correct for the bias in generative models. We find that this technique consistently improves standard goodness-of-fit metrics for evaluating the sample quality of state-of-the-art deep generative models, suggesting reduced bias. Finally, we demonstrate its utility on representative applications in a) data augmentation for classification using generative adversarial networks, and b) model-based policy evaluation using off-policy data.

Motivation & Objective

• Motivate and formalize bias in expectations computed from learned generative models.
• Propose a likelihood-free importance weighting (LFIW) framework to debias Monte Carlo estimates.
• Develop practical variants (self-normalization, flattening, clipping) to stabilize weights.
• Demonstrate improvements in goodness-of-fit metrics and downstream tasks (data augmentation, MBOPE).
• Provide theoretical conditions under which the induced weighted distribution improves fit to data distribution.

Proposed method

• Formulate bias in Monte Carlo expectations when p_theta differs from p_data and propose LFIW to estimate density ratios via a binary classifier.
• Train a calibrated probabilistic classifier to distinguish samples from p_theta and p_data; use its output to compute weights w_phi(x) = gamma * c_phi(x) / (1 - c_phi(x)).
• Define a biased-to-unbiased estimator for expectations using the estimated weights; apply self-normalization, weight flattening, or clipping as variants.
• Construct an importance-resampled generative model p_{theta,phi} proportional to p_theta(x) * w_phi(x) and implement sampling via Sampling-Importance-Resampling (SIR).
• Derive a KL-divergence based condition for improvement from p_theta to p_{theta,phi} and discuss practical estimation and variance considerations.
• Apply to three setups: goodness-of-fit of CIFAR-10 models (PixelCNN++, SNGAN), Omniglot data augmentation for multi-class classification, and model-based off-policy policy evaluation (MuJoCo).

Experimental results

Research questions

• RQ1 Can likelihood-free importance weighting reduce bias in expectations derived from learned generative models?
• RQ2 How does LFIW perform across standard generative-model evaluation metrics and downstream tasks like data augmentation and off-policy policy evaluation?
• RQ3 What practical variants (self-normalization, flattening, clipping) best stabilize LFIW in real-world models?
• RQ4 Under what conditions does the induced weighted distribution actually improve fit to the data distribution?
• RQ5 How close can the debiased estimates come to the true (unknown) model performance in challenging settings?

Key findings

• LFIW consistently reduces bias in goodness-of-fit metrics for state-of-the-art deep generative models on CIFAR-10 (PixelCNN++ and SNGAN), showing substantial improvements in bias-corrected metrics.
• Data augmentation with LFIW improves multi-class classification accuracy on Omniglot from 66.03% to 68.18% when weighting generated data points.
• In model-based off-policy policy evaluation on MuJoCo tasks, LFIW reduces RMSE of value estimates and provides more accurate policy evaluation as the horizon increases.
• Self-normalized LFIW generally provided the best empirical performance across experiments, balancing bias reduction with variance control.
• The approach acts as a flexible black-box wrapper around generative models, improving downstream inferences but not fully eliminating bias, highlighting the need for powerful classifiers and sufficient data.