[Paper Review] Contemporary Symbolic Regression Methods and their Relative Performance

William La Cava, Patryk Orzechowski+6 more|PubMed|Jul 29, 2021

Evolutionary Algorithms and Applications77 references144 citations

TL;DR

The paper presents SRBench, an open-source benchmarking platform that compares 14 symbolic regression (SR) methods and 7 ML methods across 252 regression problems, finding GP-based SR methods often outperform others on real-world data while certain methods excel in recovering exact ground-truth equations under low-noise conditions.

ABSTRACT

Many promising approaches to symbolic regression have been presented in recent years, yet progress in the field continues to suffer from a lack of uniform, robust, and transparent benchmarking standards. We address this shortcoming by introducing an open-source, reproducible benchmarking platform for symbolic regression. We assess 14 symbolic regression methods and 7 machine learning methods on a set of 252 diverse regression problems. Our assessment includes both real-world datasets with no known model form as well as ground-truth benchmark problems. For the real-world datasets, we benchmark the ability of each method to learn models with low error and low complexity relative to state-of-the-art machine learning methods. For the synthetic problems, we assess each method's ability to find exact solutions in the presence of varying levels of noise. Under these controlled experiments, we conclude that the best performing methods for real-world regression combine genetic algorithms with parameter estimation and/or semantic search drivers. When tasked with recovering exact equations in the presence of noise, we find that several approaches perform similarly. We provide a detailed guide to reproducing this experiment and contributing new methods, and encourage other researchers to collaborate with us on a common and living symbolic regression benchmark.

Motivation & Objective

• Promote standardized, transparent benchmarking for symbolic regression to enable fair comparisons and reproducibility.
• Assess a wide range of contemporary SR methods alongside conventional ML baselines on diverse real-world and synthetic problems.
• Characterize accuracy–complexity trade-offs and the ability to recover known ground-truth equations under varying noise levels.
• Provide a reproducible workflow and repository to encourage community contributions to a living SR benchmark.

Proposed method

• Develop SRBench as an open-source, reproducible benchmarking platform for SR.
• Extend the Penn ML Benchmark (PMLB) with 130 SR datasets and standardize a Python API for methods.
• Benchmark 14 SR methods and 7 ML methods with fixed environments, CI, and reproducible experiment settings.
• Evaluate on black-box regression problems (unknown ground truth) and ground-truth problems (synthetic physics/ODEs).
• Use metrics for accuracy (R2), model complexity, and symbolic solution recovery, plus simplifications via sympy.
• Provide a detailed, community-driven protocol for reproducing experiments and adding new methods.

Experimental results

Research questions

• RQ1 Which SR methods best balance accuracy and simplicity on real-world, black-box regression problems?
• RQ2 Can SR methods recover ground-truth symbolic equations from noisy data, and how does performance compare to deep learning approaches?
• RQ3 How do GP-based SR methods compare to non-GP approaches (e.g., gradient boosting, neural models) on diverse benchmarks?
• RQ4 What is the impact of noise on the ability to find exact symbolic solutions for ground-truth problems?

Key findings

• Operon achieves the highest test-set R2 accuracy on black-box problems, significantly outperforming other methods.
• SBP-GP and FEAT rank near the top in accuracy, with FEAT producing substantially smaller models.
• Many of the top-performing SR methods are GP-based, and they outperform ensemble tree methods like LightGBM and XGBoost in simplicity and/or accuracy.
• Among non-GP SR methods, FFX and DSR perform similarly and better than BSR and AIFeynman; DSR produces some of the smallest solutions.
• On ground-truth problems with low noise, AIFeynman recovers exact solutions most often (53%), but with higher noise, DSR, gplearn, AFP_FE, and AFP perform better.
• When considering accuracy and simplicity together, Operon, GP-GOMEA, and DSR form the Pareto front across black-box problems.