[Paper Review] Accelerating Understanding of Scientific Experiments with End to End Symbolic Regression

Nikos Aréchiga, Francine Chen+5 more|arXiv (Cornell University)|Dec 7, 2021

Evolutionary Algorithms and Applications5 citations

TL;DR

This paper introduces MACSYMA, a deep neural network that performs end-to-end symbolic regression by directly mapping raw experimental data tables to interpretable symbolic equations. Trained on synthetic datasets with variable length and noise, MACSYMA achieves high accuracy in recovering known scientific models—such as the subjective utility equation from behavioral science—using only a feedforward pass, bypassing computationally expensive methods like genetic programming.

ABSTRACT

We consider the problem of learning free-form symbolic expressions from raw data, such as that produced by an experiment in any scientific domain. Accurate and interpretable models of scientific phenomena are the cornerstone of scientific research. Simple yet interpretable models, such as linear or logistic regression and decision trees often lack predictive accuracy. Alternatively, accurate blackbox models such as deep neural networks provide high predictive accuracy, but do not readily admit human understanding in a way that would enrich the scientific theory of the phenomenon. Many great breakthroughs in science revolve around the development of parsimonious equational models with high predictive accuracy, such as Newton's laws, universal gravitation, and Maxwell's equations. Previous work on automating the search of equational models from data combine domain-specific heuristics as well as computationally expensive techniques, such as genetic programming and Monte-Carlo search. We develop a deep neural network (MACSYMA) to address the symbolic regression problem as an end-to-end supervised learning problem. MACSYMA can generate symbolic expressions that describe a dataset. The computational complexity of the task is reduced to the feedforward computation of a neural network. We train our neural network on a synthetic dataset consisting of data tables of varying length and varying levels of noise, for which the neural network must learn to produce the correct symbolic expression token by token. Finally, we validate our technique by running on a public dataset from behavioral science.

Motivation & Objective

• To develop a scalable, differentiable method for learning free-form symbolic expressions directly from raw experimental data.
• To overcome the computational intractability of traditional symbolic regression techniques such as genetic programming and Monte Carlo search.
• To enable interpretable, high-accuracy scientific models that support theory development and human understanding.
• To train a neural network to generate grammatically correct symbolic expressions without relying on explicit grammar constraints or post-hoc validation.
• To validate the method on real-world scientific data, including a behavioral science dataset modeling prospect theory.

Proposed method

• MACSYMA frames symbolic regression as an end-to-end supervised learning problem, mapping data tables directly to symbolic expressions.
• The model is trained on a synthetic dataset of varying lengths and noise levels, learning to generate equations token-by-token.
• A feedforward neural network architecture is used, with no recurrent or autoregressive components in the current version.
• The network is trained to predict symbolic expressions that minimize prediction error on the input data tables.
• The method does not enforce grammatical correctness via external rules, relying instead on the network's ability to learn valid syntax implicitly.
• Post-training inference is performed via a single forward pass, avoiding iterative or search-based decoding.

Experimental results

Research questions

• RQ1 Can a deep neural network learn to generate accurate and interpretable symbolic equations directly from raw data tables?
• RQ2 To what extent can a neural network learn the grammatical structure of symbolic expressions without explicit grammar constraints?
• RQ3 How does the performance of end-to-end neural symbolic regression compare to traditional heuristic or search-based methods in terms of speed and accuracy?
• RQ4 Can the model generalize to real-world scientific datasets and recover known theoretical equations?
• RQ5 Can the model be extended to handle variable-length outputs and support interactive human-in-the-loop formula selection?

Key findings

• MACSYMA successfully recovers the well-known subjective utility equation (SV_i = p_i * V_i^α) from the SEU behavioral science dataset, demonstrating its ability to discover theory-based models.
• The model achieves a parsing accuracy of approximately 80% for generated formulas during training, indicating that the network learns to produce grammatically valid symbolic expressions.
• The inference process is highly efficient, requiring only a single feedforward pass, unlike methods that rely on iterative search or optimization.
• The approach outperforms traditional symbolic regression techniques in speed by eliminating computationally expensive search procedures such as genetic programming or Monte Carlo sampling.
• The method generalizes to datasets of varying lengths and noise levels, showing robustness to data variability in the training distribution.
• The authors identify the limitation of fixed maximum output length and plan to extend the model with recurrent architectures and beam search for future work.