[Paper Review] Continuous-Time Functional Diffusion Processes

Giulio Franzese, Corallo, Giulio+4 more|arXiv (Cornell University)|Mar 1, 2023

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper introduces Functional Diffusion Processes (fDPs), a continuous-time framework that generalizes score-based diffusion models to infinite-dimensional function spaces using advanced stochastic calculus. By extending Girsanov's theorem and the sampling theorem to Hilbert spaces, fDPs enable high-fidelity generation of images, audio, and other continuous data using simple MLPs or Transformers, achieving state-of-the-art results with orders-of-magnitude fewer parameters than prior diffusion models.

ABSTRACT

We introduce Functional Diffusion Processes (FDPs), which generalize score-based diffusion models to infinite-dimensional function spaces. FDPs require a new mathematical framework to describe the forward and backward dynamics, and several extensions to derive practical training objectives. These include infinite-dimensional versions of Girsanov theorem, in order to be able to compute an ELBO, and of the sampling theorem, in order to guarantee that functional evaluations in a countable set of points are equivalent to infinite-dimensional functions. We use FDPs to build a new breed of generative models in function spaces, which do not require specialized network architectures, and that can work with any kind of continuous data. Our results on real data show that FDPs achieve high-quality image generation, using a simple MLP architecture with orders of magnitude fewer parameters than existing diffusion models.

Motivation & Objective

• To develop a theoretically grounded framework for diffusion models operating directly on continuous functions, avoiding discretization artifacts.
• To extend key tools like Girsanov's theorem and sampling theorems to infinite-dimensional Hilbert spaces for rigorous variational inference.
• To enable end-to-end generative modeling of diverse continuous data modalities—images, audio, video—without specialized architectures.
• To demonstrate that simple models like MLPs or Transformers can achieve high-quality generation by learning functional scores in continuous space.
• To provide a unified, scalable approach to generative modeling that supports arbitrary resolution and resolution-free generation.

Proposed method

• Proposes Functional Diffusion Processes (fDPs) as continuous-time stochastic processes in Hilbert space, generalizing finite-dimensional diffusion models to function spaces.
• Derives an evidence lower bound (ELBO) via an infinite-dimensional extension of Girsanov's theorem, enabling variational training of the score function.
• Introduces an infinite-dimensional sampling theorem, proving that functional evaluations at a countable set of points are sufficient to reconstruct the full function.
• Uses implicit neural representations (INRs) and Transformers to jointly model function values and their scores in a parameter-efficient manner.
• Employs a predictor-corrector numerical scheme adapted to the stochastic differential equations (SDEs) of fDPs for sampling and training.
• Applies the framework to image and audio generation without intermediate discretization, using raw continuous data as input.

Figure 1 : Qualitative results with MLP.

Experimental results

Research questions

• RQ1 Can score-based diffusion models be rigorously extended to infinite-dimensional function spaces using continuous-time stochastic processes?
• RQ2 How can the evidence lower bound (ELBO) be derived in infinite-dimensional Hilbert spaces to enable variational training of functional scores?
• RQ3 What mathematical conditions ensure that a function can be perfectly reconstructed from its evaluations at a countable set of points in the context of functional diffusion?
• RQ4 Can simple, non-specialized neural architectures like MLPs or Transformers achieve high-quality generation in continuous function spaces?
• RQ5 How does the proposed framework perform across diverse data modalities such as images, audio, and video, without architectural or data-specific tuning?

Key findings

• fDPs achieve high-fidelity image generation using a simple MLP with orders of magnitude fewer parameters than existing diffusion models, demonstrating strong parameter efficiency.
• The framework enables resolution-free generation, as shown in super-resolution experiments on MNIST, where models generate images at arbitrary resolutions from a single training run.
• Conditional generation tasks such as in-painting, de-blurring, and colorization produce qualitatively coherent results, with the model reconstructing missing or corrupted regions while preserving semantic structure.
• Preliminary audio generation experiments on the Spoken Digit dataset show that the model can generate intelligible spoken digits directly from raw waveforms, with perceptually accurate waveforms matching real samples.
• The use of implicit neural representations (INRs) and Transformers enables effective joint modeling of function values and scores, supporting flexible and scalable functional representation.
• The theoretical framework, grounded in extended Girsanov and sampling theorems, ensures that the training objective is a valid ELBO, providing a principled alternative to heuristic score matching in infinite dimensions.

Figure 2 : Qualitative results with UViT.