[Paper Review] On the Equivalence between Herding and Conditional Gradient Algorithms

Francis Bach, Simon Lacoste-Julien+1 more|arXiv (Cornell University)|Mar 20, 2012

Stochastic Gradient Optimization TechniquesComputer Science15 references67 citations

TL;DR

This paper establishes that the herding algorithm is mathematically equivalent to a conditional gradient (Frank-Wolfe) method for minimizing quadratic moment discrepancy in a reproducing kernel Hilbert space. The equivalence enables faster convergence via advanced variants like line-search and active-set methods, yet experiments show these outperform herding in mean estimation but fail to approximate the maximum entropy distribution as well as original herding, suggesting a trade-off between efficiency and entropy preservation.

ABSTRACT

We show that the herding procedure of Welling (2009) takes exactly the form of a standard convex optimization algorithm--namely a conditional gradient algorithm minimizing a quadratic moment discrepancy. This link enables us to invoke convergence results from convex optimization and to consider faster alternatives for the task of approximating integrals in a reproducing kernel Hilbert space. We study the behavior of the different variants through numerical simulations. The experiments indicate that while we can improve over herding on the task of approximating integrals, the original herding algorithm tends to approach more often the maximum entropy distribution, shedding more light on the learning bias behind herding.

Motivation & Objective

• To clarify the theoretical foundation of the herding algorithm by linking it to convex optimization.
• To improve mean estimation performance by adapting faster conditional gradient variants to the herding framework.
• To investigate whether faster convergence in moment estimation correlates with better approximation of the maximum entropy distribution.
• To analyze the learning bias of herding, particularly its tendency to approach maximum entropy under certain conditions.

Proposed method

• The herding algorithm is reinterpreted as a conditional gradient method minimizing the quadratic error between the empirical moment and target mean vector.
• The algorithm updates the current iterate by solving a linear maximization over the feature space: $ x_{t+1} = \arg\max_{x\in\mathcal{X}} \langle w_t, \Phi(x) \rangle $.
• A line-search variant is introduced to adaptively choose step sizes, improving convergence speed over standard herding.
• An active-set variant is proposed to accelerate convergence by focusing on active constraints in the marginal polytope.
• Theoretical convergence rates are derived, including linear convergence for the line-search variant in finite-dimensional settings.
• Numerical experiments compare standard herding, line-search, and active-set variants on mean estimation and maximum entropy approximation tasks.

Experimental results

Research questions

• RQ1 Is herding equivalent to a known convex optimization algorithm, and if so, which one?
• RQ2 Can faster variants of the conditional gradient algorithm improve the rate of convergence in moment estimation compared to standard herding?
• RQ3 Does improved convergence in moment estimation lead to better approximation of the maximum entropy distribution?
• RQ4 Under what conditions does herding converge to the maximum entropy distribution?
• RQ5 What is the relationship between convergence speed and entropy preservation in herding-based sampling?

Key findings

• Herding is formally equivalent to a conditional gradient algorithm minimizing the quadratic moment discrepancy, providing a new optimization interpretation.
• The line-search variant achieves linear convergence rate in finite-dimensional settings, outperforming the standard $ O(1/t) $ rate of herding.
• The active-set variant also improves convergence speed, though with less theoretical guarantee than the line-search version.
• Despite faster convergence, the line-search and active-set variants fail to approximate the maximum entropy distribution as effectively as standard herding.
• Experiments show that standard herding converges to the maximum entropy distribution for almost all random mean vectors, particularly when mean ratios are irrational.
• In contrast, faster variants tend to converge to low-entropy solutions with sparse support, suggesting a trade-off between estimation speed and entropy preservation.