[论文解读] Contemporary Symbolic Regression Methods and their Relative Performance

William La Cava, Patryk Orzechowski+6 more|PubMed|Jul 29, 2021

Evolutionary Algorithms and Applications参考文献 77被引用 144

一句话总结

论文提出了 SRBench，一个开源基准平台，比较了 14 种符号回归（SR）方法和 7 种 ML 方法，在 252 个回归问题上进行评估，发现基于 GP 的 SR 方法在现实世界数据上常常优于其他方法，而在噪声较低的条件下，某些方法在恢复精确的 ground-truth 方程方面表现出色。

ABSTRACT

Many promising approaches to symbolic regression have been presented in recent years, yet progress in the field continues to suffer from a lack of uniform, robust, and transparent benchmarking standards. We address this shortcoming by introducing an open-source, reproducible benchmarking platform for symbolic regression. We assess 14 symbolic regression methods and 7 machine learning methods on a set of 252 diverse regression problems. Our assessment includes both real-world datasets with no known model form as well as ground-truth benchmark problems. For the real-world datasets, we benchmark the ability of each method to learn models with low error and low complexity relative to state-of-the-art machine learning methods. For the synthetic problems, we assess each method's ability to find exact solutions in the presence of varying levels of noise. Under these controlled experiments, we conclude that the best performing methods for real-world regression combine genetic algorithms with parameter estimation and/or semantic search drivers. When tasked with recovering exact equations in the presence of noise, we find that several approaches perform similarly. We provide a detailed guide to reproducing this experiment and contributing new methods, and encourage other researchers to collaborate with us on a common and living symbolic regression benchmark.

研究动机与目标

• Promote standardized, transparent benchmarking for symbolic regression to enable fair comparisons and reproducibility.
• Assess a wide range of contemporary SR methods alongside conventional ML baselines on diverse real-world and synthetic problems.
• Characterize accuracy–complexity trade-offs and the ability to recover known ground-truth equations under varying noise levels.
• Provide a reproducible workflow and repository to encourage community contributions to a living SR benchmark.

提出的方法

• Develop SRBench as an open-source, reproducible benchmarking platform for SR.
• Extend the Penn ML Benchmark (PMLB) with 130 SR datasets and standardize a Python API for methods.
• Benchmark 14 SR methods and 7 ML methods with fixed environments, CI, and reproducible experiment settings.
• Evaluate on black-box regression problems (unknown ground truth) and ground-truth problems (synthetic physics/ODEs).
• Use metrics for accuracy (R2), model complexity, and symbolic solution recovery, plus simplifications via sympy.
• Provide a detailed, community-driven protocol for reproducing experiments and adding new methods.

实验结果

研究问题

• RQ1 Which SR methods best balance accuracy and simplicity on real-world, black-box regression problems?
• RQ2 Can SR methods recover ground-truth symbolic equations from noisy data, and how does performance compare to deep learning approaches?
• RQ3 How do GP-based SR methods compare to non-GP approaches (e.g., gradient boosting, neural models) on diverse benchmarks?
• RQ4 What is the impact of noise on the ability to find exact symbolic solutions for ground-truth problems?

主要发现

• Operon achieves the highest test-set R2 accuracy on black-box problems, significantly outperforming other methods.
• SBP-GP and FEAT rank near the top in accuracy, with FEAT producing substantially smaller models.
• Many of the top-performing SR methods are GP-based, and they outperform ensemble tree methods like LightGBM and XGBoost in simplicity and/or accuracy.
• Among non-GP SR methods, FFX and DSR perform similarly and better than BSR and AIFeynman; DSR produces some of the smallest solutions.
• On ground-truth problems with low noise, AIFeynman recovers exact solutions most often (53%), but with higher noise, DSR, gplearn, AFP_FE, and AFP perform better.
• When considering accuracy and simplicity together, Operon, GP-GOMEA, and DSR form the Pareto front across black-box problems.