[Paper Review] Variational limits of k-NN graph based functionals on data clouds

Nicolás García Trillos|arXiv (Cornell University)|Jul 3, 2016

Advanced Clustering Algorithms Research28 references3 citations

TL;DR

This paper establishes the rigorous large-sample consistency of k-NN graph-based functionals, such as Cheeger cuts, by proving their Gamma-convergence to continuum variational problems under the scaling condition $\log n \ll k_n \ll n$. It demonstrates that solutions to discrete graph optimization problems converge almost surely to solutions of corresponding continuous variational problems, providing theoretical justification for k-NN-based clustering and learning methods.

ABSTRACT

This paper studies the large sample asymptotics of data analysis procedures based on the optimization of functionals defined on $k$-NN graphs on point clouds. The paper is framed in the context of minimization of balanced cut functionals, but our techniques, ideas and results can be adapted to other functionals of relevance. We rigorously show that provided the number of neighbors in the graph $k:=k_n$ scales with the number of points in the cloud as $n \gg k_n \gg \log(n)$, then with probability one, the solution to the graph cut optimization problem converges towards the solution of an analogue variational problem at the continuum level.

Motivation & Objective

• To rigorously analyze the large-sample asymptotics of graph-based data analysis procedures using k-NN graphs.
• To establish the convergence of discrete graph cut optimization solutions to their continuum analogues.
• To provide a theoretical foundation for the statistical consistency of k-NN-based clustering and regularization methods.
• To address the lack of theoretical results for k-NN graphs compared to the more commonly studied ε-graphs.
• To show that the required scaling condition on $k_n$ is dimension-free, enhancing applicability to high-dimensional data.

Proposed method

• Uses Gamma-convergence to analyze the limit of rescaled graph total variation functionals defined on k-NN graphs.
• Defines the graph total variation (GTV) functional as $\sum_{\mathbf{x}_i \sim_k \mathbf{x}_j} |u(\mathbf{x}_i) - u(\mathbf{x}_j)|$ for functions $u$ on data points.
• Establishes the $\Gamma$-limit of GTV as a weighted local total variation functional at the continuum level.
• Analyzes the bias of the random functional GTV by constructing a kernel with inhomogeneous bandwidth and relating it to the underlying data density.
• Applies compactness and recovery sequence arguments using $C_c^\infty$ functions and indicator functions to prove convergence.
• Employs a diagonal argument over compact subsets of the reduced boundary to handle sets of finite perimeter.

Experimental results

Research questions

• RQ1 Under what conditions does the solution to a k-NN graph-based discrete optimization problem converge to a continuum variational problem?
• RQ2 How does the scaling of $k_n$ affect the consistency of graph-based clustering methods like spectral clustering or Cheeger cuts?
• RQ3 Can the $\Gamma$-limit of graph total variation on k-NN graphs be characterized as a weighted total variation functional at the continuum level?
• RQ4 Why is the $k$-NN construction theoretically preferable to $\varepsilon$-graphs in the context of large-sample consistency?
• RQ5 Is the required scaling condition on $k_n$ dependent on the data dimension?

Key findings

• With probability one, the minimizer of the discrete Cheeger cut on a k-NN graph converges to the minimizer of the continuum Cheeger cut functional as $n \to \infty$.
• The $\Gamma$-limit of the rescaled graph total variation functional $GTV_{n,k_n}$ is a weighted local total variation functional of the form $\frac{\sigma_\eta}{\alpha_d^{1+1/d}} TV(u; \rho^{1-1/d})$.
• The convergence holds under the scaling condition $\log n \ll k_n \ll n$, which is dimension-free and robust to data dimensionality.
• The recovery sequence for indicator functions $u = \mathbf{1}_A$ can be constructed from discrete indicator functions, ensuring consistency for partition-based problems.
• Compactness of the sequence of functions is established via comparison with $\varepsilon$-graph total variation, leveraging a lower bound on the local neighborhood size.
• The proof relies on a diagonal construction over compact subsets of the reduced boundary of sets of finite perimeter, ensuring convergence of Hausdorff measure approximations.