[Paper Review] Spectra of winner-take-all stochastic neural networks

Tomasz Schreiber|arXiv (Cornell University)|Oct 17, 2008

Neural dynamics and brain function14 references6 citations

TL;DR

This paper rigorously analyzes the spectral properties of winner-take-all (WTA) stochastic neural networks, deriving a limit theorem for the spectra of spike-flow graphs induced by WTA dynamics. It identifies the limiting spectral measure explicitly in terms of the zeros of Bessel's J-function, establishing a mathematical foundation for scale-free behavior in mesoscale brain networks observed in fMRI data.

ABSTRACT

During the recent few years, in response to empirical findings suggesting scale-free self-organisation phenomena emerging in complex nervous systems at a mesoscale level, there has been significant search for suitable models and theoretical explanations in neuroscientific literature, see the recent survey by Bullmore and Sporns (2009). In Piekniewski and Schreiber (2008) we have developed a simple and tractable mathematical model shedding some light on a particular class of the afore-mentioned phenomena, namely on mesoscopic level self-organisation of functional brain networks under fMRI imaging, where we have achieved a high degree of agreement with existing empirical reports. Being addressed to the neuroscientific community, our work Piekniewski and Schreiber (2008) relied on semi-rigorous study of information flow structure in a class of recurrent neural networks exhibiting asymptotic scale-free behaviour and admitting a description in terms of the so-called winner-take-all dynamics. The purpose of the present paper is to define and study these winner-take-all networks with full mathematical rigour in context of their asymptotic spectral properties, well known to be of interest for neuroscientific community. Our main result is a limit theorem for spectra of the spike-flow graphs induced by the winner-take-all dynamics. We provide an explicit characterisation of the limit spectral measure expressed in terms of zeros of Bessel's J-function.

Motivation & Objective

• To provide a mathematically rigorous framework for understanding the asymptotic spectral properties of winner-take-all (WTA) stochastic neural networks.
• To characterize the limiting spectral measure of spike-flow graphs generated by WTA dynamics in large-N asymptotics.
• To establish a connection between the spectral distribution of these neural networks and the zeros of Bessel's J-function.
• To justify the emergence of scale-free topology in mesoscale functional brain networks through a tractable stochastic model.
• To extend prior semi-rigorous findings in Piekniewski & Schreiber (2008) with full mathematical proof using operator theory and extreme value methods.

Proposed method

• Formal definition of a winner-take-all (WTA) stochastic neural network model with N neurons and i.i.d. Gaussian weights.
• Construction of spike-flow graphs where edge multiplicities represent the number of charge transfers between neurons.
• Application of trace-class operator theory and spectral analysis to study the limiting spectral distribution of the normalized adjacency matrix.
• Use of the method of moments and Carleman’s criterion to prove weak convergence of spectral measures.
• Derivation of an integral eigenvalue equation for the limiting kernel operator, leading to a differential equation involving Bessel functions.
• Solution of the differential equation using Bessel functions of order 1 and 0, with boundary conditions enforcing L2-integrability and normalization.

Experimental results

Research questions

• RQ1 What is the limiting spectral distribution of the spike-flow graph in large-N winner-take-all stochastic neural networks?
• RQ2 How are the eigenvalues of the limiting kernel operator related to special functions such as Bessel functions?
• RQ3 Can the emergence of scale-free topology in mesoscale brain networks be rigorously explained via WTA dynamics?
• RQ4 What conditions on the scaling of system parameters ensure convergence of the spectral measure to a deterministic limit?
• RQ5 What is the role of extreme value theory and trace-class operators in characterizing the spectral properties of these stochastic neural systems?

Key findings

• The limiting spectral measure of the spike-flow graph is characterized by the zeros of Bessel's J-function of order 1.
• All eigenvalues of the limiting kernel operator are positive and simple, with eigenvalues λ satisfying J₁(2√2 / √λ) = 0.
• The spectral measure arises as the weak limit of the empirical spectral distribution of the normalized adjacency matrix under appropriate scaling.
• The convergence of the spectral measure is established via the method of moments and Carleman’s criterion, relying on trace-class properties of the limiting operator.
• The model confirms that WTA dynamics leads to asymptotically scale-free networks with a power-law exponent of 2, consistent with empirical fMRI observations.
• The analysis justifies the heuristic semi-rigorous results of Piekniewski & Schreiber (2008) with full mathematical rigor using functional analysis and special functions.