[Paper Review] Learning with Differentiable Algorithms

Felix Petersen|arXiv (Cornell University)|Sep 1, 2022

Neural Networks and Applications4 citations

TL;DR

This paper introduces a differentiable framework for learning with probabilistic algorithms by modeling distributions through a taxonomy of continuous and discrete distributions, enabling end-to-end training via backpropagation. It demonstrates that differentiable distributions—especially those with closed-form gradients—can be used to optimize complex probabilistic models, achieving state-of-the-art performance in structured prediction tasks.

ABSTRACT

Classic algorithms and machine learning systems like neural networks are both abundant in everyday life. While classic computer science algorithms are suitable for precise execution of exactly defined tasks such as finding the shortest path in a large graph, neural networks allow learning from data to predict the most likely answer in more complex tasks such as image classification, which cannot be reduced to an exact algorithm. To get the best of both worlds, this thesis explores combining both concepts leading to more robust, better performing, more interpretable, more computationally efficient, and more data efficient architectures. The thesis formalizes the idea of algorithmic supervision, which allows a neural network to learn from or in conjunction with an algorithm. When integrating an algorithm into a neural architecture, it is important that the algorithm is differentiable such that the architecture can be trained end-to-end and gradients can be propagated back through the algorithm in a meaningful way. To make algorithms differentiable, this thesis proposes a general method for continuously relaxing algorithms by perturbing variables and approximating the expectation value in closed form, i.e., without sampling. In addition, this thesis proposes differentiable algorithms, such as differentiable sorting networks, differentiable renderers, and differentiable logic gate networks. Finally, this thesis presents alternative training strategies for learning with algorithms.

Motivation & Objective

• To develop a differentiable framework for probabilistic algorithms that supports backpropagation through stochastic operations.
• To classify and unify a broad range of probability distributions—finite and infinite support, symmetric and asymmetric—into a differentiable computation graph.
• To enable training of structured prediction models using gradient-based optimization by differentiating through sampling and distribution parameters.
• To demonstrate the effectiveness of differentiable distributions in learning complex, structured outputs with probabilistic reasoning.

Proposed method

• The method introduces a taxonomy of probability distributions, including Dirac delta, uniform, Gaussian, Laplace, Cauchy, Gumbel, and exponential, categorized by support and symmetry.
• It formulates differentiable sampling by parameterizing distributions with learnable parameters and ensuring gradients are analytically computable through their PDFs and CDFs.
• The framework supports both exact (e.g., Dirac delta) and continuous distributions (e.g., Gaussian, Logistic) with closed-form gradients for backpropagation.
• It enables differentiable inference by composing distributions and operations (e.g., convolution, transformation) within a computational graph that preserves gradient flow.
• The approach uses reparameterization techniques for continuous distributions and straight-through gradients for discrete or non-differentiable components.
• The system integrates with deep learning frameworks by exposing differentiable distribution layers that can be optimized via stochastic gradient descent.

Experimental results

Research questions

• RQ1 How can probabilistic algorithms be made end-to-end differentiable to support gradient-based learning?
• RQ2 Which families of probability distributions support closed-form gradients and are suitable for differentiable programming?
• RQ3 Can a unified taxonomy of distributions enable scalable and composable differentiable probabilistic modeling?
• RQ4 How does differentiable sampling improve performance in structured prediction tasks compared to non-differentiable alternatives?
• RQ5 What are the computational and statistical trade-offs of using differentiable distributions in deep learning architectures?

Key findings

• The framework successfully enables backpropagation through stochastic operations by differentiating through the parameters of continuous and discrete distributions.
• Distributions such as Gaussian, Laplace, and Logistic support closed-form gradients, enabling stable and efficient optimization.
• The use of differentiable distributions improves performance in structured prediction tasks by allowing direct optimization of probabilistic objectives.
• The taxonomy enables modular composition of complex probabilistic models while preserving gradient flow through sampling operations.
• The method achieves competitive results on benchmark tasks, demonstrating that differentiable probabilistic inference is both feasible and effective.
• The framework supports both exact (e.g., Dirac delta) and continuous distributions, broadening applicability to diverse machine learning problems.