[Paper Review] Generalized Polya urns via stochastic approximation

Henrik Renlund|arXiv (Cornell University)|Feb 19, 2010

Markov Chains and Monte Carlo Methods16 references19 citations

TL;DR

This paper applies stochastic approximation theory to analyze generalized Pólya urn models with one- or two-ball draws, deriving conditions under which the fraction of white balls converges almost surely. The key contribution is a characterization of the limiting distribution's support in terms of equilibrium points of a drift function, showing convergence to stable zeros of the drift function under general conditions, with exceptions only in degenerate parameter cases.

ABSTRACT

We collect, survey and develop methods of (one-dimensional) stochastic approximation in a framework that seems suitable to handle fairly broad generalizations of Polya urns. To show the applicability of the results we determine the limiting fraction of balls in an urn with balls of two colors. We consider two models generalizing the Polya urn, in the first one ball is drawn and replaced with balls of (possibly) both colors according to which color was drawn. In the second, two balls are drawn simultaneously and replaced along with balls of (possibly) both colors according to what combination of colors were drawn.

Motivation & Objective

• To develop a stochastic approximation framework for analyzing generalized Pólya urns with complex reinforcement rules.
• To determine the limiting fraction of white balls in two-color urn models where balls are added based on drawn colors.
• To characterize the almost sure convergence of the fraction of white balls in generalized urn schemes.
• To identify conditions under which the limiting distribution has support only on stable equilibrium points of the drift function.
• To extend existing results on urn convergence by incorporating two-draw mechanisms and non-linear drifts.

Proposed method

• Uses stochastic approximation to model the evolution of the fraction of white balls in generalized Pólya urns.
• Defines a normalized process $ \hat{Z}_n $ representing the fraction of white balls at step $ n $.
• Derives a drift function $ g(x) $ that governs the expected change in $ \hat{Z}_n $, derived from replacement rules.
• Introduces an error function $ \mathcal{E}(\hat{Z}_n) $ to quantify the deviation of the process from the drift.
• Compares the behavior of the normalized process $ \hat{Z}_n $ to the original urn process $ Z_n $, showing equivalence in equilibrium structure.
• Applies conditions from stochastic approximation theory, including stability and convergence criteria at equilibrium points.

Experimental results

Research questions

• RQ1 Under what conditions does the fraction of white balls in a generalized Pólya urn converge almost surely?
• RQ2 How do the equilibrium points of the drift function relate to the limiting distribution of the urn composition?
• RQ3 What happens to the limiting distribution when the drift function has unstable or touchpoint equilibria?
• RQ4 How does the two-draw mechanism affect the convergence behavior compared to one-draw schemes?
• RQ5 Can the stochastic approximation framework handle non-linear drift functions arising from complex reinforcement rules?

Key findings

• The fraction of white balls in the urn converges almost surely to a limit under general conditions, provided $ w_0, b_0 > 1 $ or $ w_0, b_0 > 0 $ in the with-replacement case.
• The limiting distribution has support only on the zeros of the drift function $ g(x) $, where $ g'(x) \leq 0 $, indicating stable equilibria.
• In the exceptional case where $ a = 2d, f = 2c, b = e = 0 $, the limiting variable has no point masses in $ (0,1) $, but the support is still restricted to equilibrium points.
• The drift function $ g(x) $ and its normalized version $ \hat{g}(x) $ share the same equilibrium points, and their stability properties (stable, unstable, touchpoint) are identical.
• The convergence behavior is preserved under transformation: $ \hat{g}(x) $ and $ g(x) $ have matching stability classifications at equilibrium points due to shared sign of derivatives.
• The framework extends prior results, including a central limit theorem from Mahfouzi (2008), by providing a broader convergence characterization beyond linear drifts.