[Paper Review] An Approach to Symbolic Regression Using Feyn

Kevin René Broløs, Meera Vieira Machado+6 more|arXiv (Cornell University)|Apr 12, 2021

Computational Physics and Python Applications36 citations

TL;DR

The paper presents Feyn, a Python tool for symbolic regression via the QLattice, showcasing how it builds interpretable graph-based models and guides hypothesis-driven data discovery, demonstrated with a breast cancer dataset.

ABSTRACT

In this article we introduce the supervised machine learning tool called Feyn. The simulation engine that powers this tool is called the QLattice. The QLattice is a supervised machine learning tool inspired by Richard Feynman's path integral formulation, that explores many potential models that solves a given problem. It formulates these models as graphs that can be interpreted as mathematical equations, allowing the user to completely decide on the trade-off between interpretability, complexity and model performance. We touch briefly upon the inner workings of the QLattice, and show how to apply the python package, Feyn, to scientific problems. We show how it differs from traditional machine learning approaches, what it has in common with them, as well as some of its commonalities with symbolic regression. We describe the benefits of this approach as opposed to black box models. To illustrate this, we go through an investigative workflow using a basic data set and show how the QLattice can help you reason about the relationships between your features and do data discovery.

Motivation & Objective

• Introduce the supervised ML tool Feyn and its QLattice engine for symbolic regression.
• Explain how QLattice generates, refines, and interprets graph-based mathematical models.
• Showcase a practical workflow for hypothesis generation and validation using a real dataset.

Proposed method

• QLattice simulates discrete paths in a finite lattice to connect inputs to an output, guided by reinforced best paths.
• Models are represented as unidirectional acyclic graphs (QGraphs) of interactions (functions) such as addition, multiplication, log, exp, tanh, Gaussian, etc.
• Feyn provides automatic feature encoding, no need for normalization or one-hot encoding, and preserves interpretability of the resulting equations.
• A training loop fits thousands of graphs to data, selects the best, and updates the lattice to converge toward useful solutions.
• Semantic types (Numerical, Categorical) guide automatic encoding and graph construction.
• Users constrain the search space (problem type, features, depth) to focus exploration and improve interpretability.

Experimental results

Research questions

• RQ1 How can symbolic regression via the QLattice produce interpretable models for given data questions?
• RQ2 What relationships between features can be discovered through QGraph hypotheses and how do these relate to the target variable?
• RQ3 How does constraining the QLattice search space affect model interpretability and predictive usefulness?
• RQ4 Can hypotheses generated by the QLattice generalize to unseen data (holdout set) while maintaining interpretability?
• RQ5 What workflow best facilitates hypothesis formulation, refinement, and validation using Feyn?

Key findings

• The QLattice generates a manageable set of graph-based hypotheses that can be inspected and interpreted as mathematical formulas.
• Interpretable models can be derived by constraining graph depth and including specific features, aiding scientific reasoning.
• The approach enables visualizations (ROC curves, probability scores, partial dependence) that help assess hypothesis performance and identify overfitting.
• Selected hypotheses can generalize to holdout data, supporting the claim that symbolic regression with Feyn yields robust, explainable models.
• The workflow emphasizes hypothesis-driven exploration and iterative refinement, consistent with the scientific method, rather than purely maximizing predictive accuracy.