[Paper Review] Representation by Integrating Reproducing Kernels

Thomas Hotz, Fabian J. E. Telschow|arXiv (Cornell University)|Feb 20, 2012

Neural Networks and Applications16 references3 citations

TL;DR

This paper develops a general framework for integrating reproducing kernels over a parameter space using direct integrals, showing that pointwise integration yields a new reproducing kernel Hilbert space as the image of the direct integral under the summation operator. The key contribution is a unified treatment of diverse kernel constructions—such as Mercer kernels, scale-mixtures of radial basis functions, and integral transforms—under a single theoretical umbrella, enabling new applications in inverse problems and sampling theory.

ABSTRACT

Based on direct integrals, a framework allowing to integrate a parametrised family of reproducing kernels with respect to some measure on the parameter space is developed. By pointwise integration, one obtains again a reproducing kernel whose corresponding Hilbert space is given as the image of the direct integral of the individual Hilbert spaces under the summation operator. This generalises the well-known results for finite sums of reproducing kernels; however, many more special cases are subsumed under this approach: so-called Mercer kernels obtained through series expansions; kernels generated by integral transforms; mixtures of positive definite functions; and in particular scale-mixtures of radial basis functions. This opens new vistas into known results, e.g. generalising the Kramer sampling theorem; it also offers interesting connections between measurements and integral transforms, e.g. allowing to apply the representer theorem in certain inverse problems, or bounding the pointwise error in the image domain when observing the pre-image under an integral transform.

Motivation & Objective

• To develop a general mathematical framework for integrating parametrized families of reproducing kernels over a measure space.
• To characterize the resulting reproducing kernel Hilbert space as the image of a direct integral of individual Hilbert spaces under the summation operator.
• To unify and generalize known constructions of positive definite functions, including Mercer kernels, scale-mixtures of radial basis functions, and integral transforms.
• To enable new applications in inverse problems and sampling theory by connecting kernel methods with integral transforms and the representer theorem.
• To provide bounds on pointwise error in the image domain when observing pre-images under integral transforms.

Proposed method

• Uses direct integrals of Hilbert spaces over a measure space $(\Omega, \mathcal{A}, \mu)$ to generalize the direct sum of reproducing kernel Hilbert spaces.
• Defines the integrated kernel via pointwise integration: $K(x,y) = \int_\Omega K_\omega(x,y) \, d\mu(\omega)$, under conditions ensuring the result is a valid reproducing kernel.
• Characterizes the resulting Hilbert space as the image of the direct integral $\int_\Omega^\oplus \mathcal{H}_\omega \, d\mu$ under the summation operator $S$.
• Applies the framework to special cases: integral transforms, Mercer expansions, mixtures of positive definite functions, and radial basis functions.
• Uses the representer theorem in $\mathrm{L}_2(\Omega, \mu)$ to solve inverse problems by minimizing a regularized loss functional.
• Derives error bounds for interpolation in the pre-image space, relating the pointwise error in the image domain to the power function and kernel norms.

Experimental results

Research questions

• RQ1 How can a family of reproducing kernels indexed over a measure space be integrated to yield a new reproducing kernel?
• RQ2 What is the structure of the reproducing kernel Hilbert space generated by such an integration process?
• RQ3 How do classical constructions—such as Mercer kernels, scale-mixtures of radial basis functions, and integral transforms—fit into this general framework?
• RQ4 Can the representer theorem be applied to solve inverse problems in $\mathrm{L}_2(\Omega, \mu)$ when the forward map is an integral transform?
• RQ5 What bounds can be placed on the pointwise error in the image domain when approximating a function via interpolation in the pre-image space?

Key findings

• The integrated kernel $K(x,y) = \int_\Omega K_\omega(x,y) \, d\mu(\omega)$ is a valid reproducing kernel if the family $\{K_\omega\}$ satisfies measurability and integrability conditions.
• The resulting reproducing kernel Hilbert space is isometrically isomorphic to the image of the direct integral $\int_\Omega^\oplus \mathcal{H}_\omega \, d\mu$ under the summation operator $S$.
• The framework generalizes finite sums of kernels and subsumes all known constructions of positive definite functions via integral transforms, series expansions, and mixtures.
• The Kramer sampling theorem is generalized by viewing sampling equations as arising from integration of kernels over a parameter space.
• For inverse problems, the representer theorem applies to $\mathrm{L}_2(\Omega, \mu)$, allowing the solution of regularized minimization problems via finite-dimensional optimization.
• A pointwise error bound is derived: $|S(a)(x) - S(a_W)(x)| \leq \|a\|_{\mathcal{G}} \|P_W\|_{\mathrm{L}_2(\Omega,\mu)} \|k(x,\cdot)\|_{\mathrm{L}_2(\Omega,\mu)}$, where $P_W$ is the power function of the interpolant.