[论文解读] α-divergence derived as the generalized rate function in a power-law system

Hiroki Suyari, A.M. Scarfone|arXiv (Cornell University)|Dec 11, 2014

Statistical Mechanics and Entropy参考文献 22被引用 2

一句话总结

本文通过精确的 q-Stirling 公式严格推导出 Tsallis 统计中的 α-散度（q-散度），将其作为大偏差原理中广义率函数的推广。结果表明，当 α → −1（q → 1）时，α-散度退化为 Kullback-Leibler 散度，证实其与经典信息论的一致性，同时将理论扩展至无需预先假设 q-高斯分布的幂律系统。

ABSTRACT

Abstract—The generalized binomial distribution in Tsallisstatistics (power-law system) is explicitly formulated from theprecise q-Stirling’s formula. Theα-divergence (orq-divergence)is uniquely derived from the generalized binomial distributionin the sense that when α → −1 (i.e., q → 1) it recovers KLdivergence obtained from the standard binomial distribution.Based on these combinatorial considerations, it is shown thatα-divergence (or q-divergence) is appeared as the generalizedrate function in the large deviation estimate in Tsallis statistics. I. I NTRODUCTION The large deviation principle (LDP for short) has mathemat-ically presented and quantified the asymptotic behavior of t heprobabilities of rare events in many stochastic phenomena. Ithas brought about deep significant insights for understandi ngof each phenomena [1][2][3]. The LDP covers quite broadareas ranging from the fundamentals in probability theoryand statistics to its applications such as statistical physics[4][5], risk management [6], information theory [7] and soon. In most of theoretical results in LDP, the assumption of“i.i.d. (independent and identically distributed)” for ra ndomvariables is used. This assumption leads to the discussion onthe exponential decay of rare events in stochastic phenomenawith great help of many well-established theoretical resultsbased on “i.i.d.” assumption. This strong “i.i.d.” assumpt ionhas been often tried to be weakened in many studies. Oneof the reasons is that actual observations generally do notsatisfy i.i.d. assumptions. A typical and well-known exampleis power-law behavior often observed in strongly correlatedsystems. In these cases we take Tsallis statistics as one ofsuch power-law systems because its mathematical foundationshas been widely explored [8].Along similar studies on LDP related with Tsallis statistics,there are a few papers such as [9] and [10]. The paper [9]discusses the possibility of LDP for the strongly correlatedrandom variables in Tsallis statistics. They consider the corre-lated coin tossing model based on the q-Gaussian distributionand numerically evaluate the possibility of a q-generalizationof LDP for a given q-divergence. On the other hand, ourpresent paper does not require the q-Gaussian distribution andthe q-divergence in advance for the large deviation estimate.Our approach is completely analytical starting from the funda-mental nonlinear differential equation dy/dx =y

研究动机与目标

• 通过精确的 q-Stirling 公式，为 Tsallis 统计中的广义二项分布建立严格的组合基础。
• 在幂律系统背景下，从第一原理推导出 α-散度（q-散度），避免依赖于如 q-高斯分布等预先假设的分布。
• 证明 α-散度自然地作为 Tsallis 统计中大偏差估计的广义率函数出现。
• 通过展示在 q → 1 时与 Kullback-Leibler 散度的一致性，将经典大偏差理论与非广延统计力学统一起来。

提出的方法

• 利用精确的 q-Stirling 公式推导广义二项分布，实现对非广延系统中组合系数的精确渐近分析。
• 将广义二项分布应用于大偏差理论，推导出 α-散度作为幂律系统中率函数的形式。
• 利用广义二项分布框架，分析在非 i.i.d. 幂律相关结构下，i.i.d.-类似序列中罕见事件的渐近行为。
• 求解基本的非线性微分方程 dy/dx = y，以建模系统的潜在动力学，将其与 q-指数和 q-对数结构联系起来。
• 证明 α-散度可自然地从广义二项分布的组合结构中导出，而无需假设 q-高斯性。
• 通过证明当 q → 1（α → −1）时，所导出的 α-散度收敛于 Kullback-Leibler 散度，验证其一致性。

实验结果

研究问题

• RQ1 如何通过精确的 q-Stirling 公式严格推导出 Tsallis 统计中广义二项分布？
• RQ2 α-散度在幂律系统的大型偏差估计中以何种方式作为率函数出现？
• RQ3 能否在不预先假设 q-高斯分布的前提下推导出 α-散度？
• RQ4 在经典极限 q → 1 下，α-散度如何退化为 Kullback-Leibler 散度？
• RQ5 非线性微分方程 dy/dx = y 在建模系统潜在统计力学中的作用是什么？

主要发现

• 通过精确的 q-Stirling 公式，严格推导出 Tsallis 统计中的广义二项分布，为非广延系统提供了坚实的组合基础。
• α-散度被唯一地从广义二项分布中推导为幂律系统大偏差理论中的率函数。
• 当 α → −1（q → 1）时，所推导的 α-散度收敛于 Kullback-Leibler 散度，证实其与经典信息论的一致性。
• 该推导无需预先引入 q-高斯分布，因此比以往方法更具根本性和普适性。
• 非线性微分方程 dy/dx = y 被证明是系统动力学的内在基础，将其与 Tsallis 统计中的 q-指数和 q-对数结构联系起来。
• α-散度被确立为强相关、幂律系统中大偏差原理率函数的自然推广。