[Paper Review] Symbolic Regression for Space Applications: Differentiable Cartesian Genetic Programming Powered by Multi-objective Memetic Algorithms

Marcus Märtens, Dario Izzo|arXiv (Cornell University)|Jun 13, 2022

Evolutionary Algorithms and Applications4 citations

TL;DR

This paper proposes a multi-objective memetic algorithm using differentiable Cartesian Genetic Programming (dCGP) to evolve interpretable symbolic regression models that simultaneously learn numerical constants during evolution. The method outperforms or matches state-of-the-art machine learning models on two space applications—Mars Express thermal power estimation and stellar age prediction—while producing mathematically interpretable expressions that generalize well despite sparse data.

ABSTRACT

Interpretable regression models are important for many application domains, as they allow experts to understand relations between variables from sparse data. Symbolic regression addresses this issue by searching the space of all possible free form equations that can be constructed from elementary algebraic functions. While explicit mathematical functions can be rediscovered this way, the determination of unknown numerical constants during search has been an often neglected issue. We propose a new multi-objective memetic algorithm that exploits a differentiable Cartesian Genetic Programming encoding to learn constants during evolutionary loops. We show that this approach is competitive or outperforms machine learned black box regression models or hand-engineered fits for two applications from space: the Mars express thermal power estimation and the determination of the age of stars by gyrochronology.

Motivation & Objective

• To address the lack of interpretability in black-box machine learning models for critical space applications.
• To overcome the limitation of symbolic regression in learning numerical constants during evolution.
• To develop a method that automatically discovers accurate, simple, and interpretable mathematical expressions from sparse data.
• To enable generalization across domain gaps in space science by leveraging physically meaningful algebraic structures.
• To provide a decision-support tool that balances model accuracy and complexity through multi-objective optimization.

Proposed method

• The method employs differentiable Cartesian Genetic Programming (dCGP), which encodes mathematical expressions as directed acyclic graphs with differentiable nodes.
• Numerical constants within expressions are learned via gradient-based optimization during evolutionary loops, enabling precise tuning of coefficients.
• A multi-objective memetic algorithm (MOMES) combines evolutionary search with local search heuristics to optimize for both model accuracy and expression complexity.
• The algorithm maintains a non-dominated front of solutions, allowing trade-offs between interpretability and performance to be visualized and selected.
• Evolutionary operators such as mutation and crossover are applied to the dCGP genotype, while gradient descent is used to refine constants in each generation.
• The final expressions are evaluated on test sets, with performance compared against standard ML models like neural networks, Gaussian processes, and random forests.

Experimental results

Research questions

• RQ1 Can differentiable CGP learn numerical constants effectively during symbolic regression, improving model accuracy?
• RQ2 How does the proposed multi-objective memetic algorithm balance accuracy and interpretability in symbolic expressions?
• RQ3 Can dCGP-based models generalize across domain gaps in space science applications, such as in stellar age estimation?
• RQ4 Do evolved symbolic expressions reveal insights into data structure or potential overfitting that black-box models miss?
• RQ5 How do dCGP models compare in performance to state-of-the-art machine learning models on real-world space datasets?

Key findings

• For the Mars Express thermal power estimation benchmark, dCGP achieved a mean absolute error (MAE) of 0.41, outperforming linear regression and matching the best-performing models.
• In the star dating benchmark B1, dCGP achieved an MAE of 1.41, outperforming most ML models and demonstrating strong generalization across domain gaps.
• On benchmark B2, a simple expression $x_6 - \sin(x_1 - x_4) + 6.987$ achieved an MAE of 1.39, outperforming all other models, including neural networks and Gaussian processes.
• The Pareto front analysis revealed that dCGP expressions were consistently more interpretable and less prone to overfitting than black-box models, especially in data-scarce settings.
• The correlation between training and test error was low for B2, suggesting that the simple expression's success may reflect true generalization rather than overfitting.
• In benchmark C, the best dCGP expression achieved an MAE of 15.63, ranking second only to Gaussian processes, demonstrating strong performance on a challenging dataset.