[Paper Review] Interpreting Multi-band Galaxy Observations with Large Language Model-Based Agents

Zechang Sun, Yuan-Sen Ting+4 more|arXiv (Cornell University)|Sep 23, 2024

demographic modeling and climate adaptationDecision Sciences3 citations

TL;DR

This paper introduces mephisto, a multi-agent LLM-based framework that emulates human expert reasoning to interpret multi-band galaxy observations from the James Webb Space Telescope. By iteratively exploring SED models through self-play, tree search, and knowledge distillation, mephisto achieves near-human proficiency in identifying physically plausible solutions—such as distinguishing between dusty star-forming galaxies and AGN-dominated systems—for challenging 'Little Red Dot' galaxies, marking the first agentic research framework in astronomy.

ABSTRACT

Astronomical research traditionally relies on extensive domain knowledge to interpret observations and narrow down hypotheses. We demonstrate that this process can be emulated using large language model-based agents to accelerate research workflows. We propose mephisto, a multi-agent collaboration framework that mimics human reasoning to interpret multi-band galaxy observations. mephisto interacts with the CIGALE codebase, which includes spectral energy distribution (SED) models to explain observations. In this open-world setting, mephisto learns from its self-play experience, performs tree search, and accumulates knowledge in a dynamically updated base. As a proof of concept, we apply mephisto to the latest data from the James Webb Space Telescope. mephisto attains near-human proficiency in reasoning about galaxies' physical scenarios, even when dealing with a recently discovered population of "Little Red Dot" galaxies. This represents the first demonstration of agentic research in astronomy, advancing towards end-to-end research via LLM agents and potentially expediting astronomical discoveries.

Motivation & Objective

• To accelerate astronomical research by automating the complex, knowledge-intensive process of interpreting multi-band galaxy observations.
• To address the challenge of fitting spectral energy distributions (SEDs) to noisy, low-resolution photometric data, especially for high-redshift galaxies with degenerate physical interpretations.
• To demonstrate that LLM-based agents can perform human-like reasoning—such as hypothesis generation, reflection, and iterative model refinement—without relying on explicit reward functions.
• To explore the feasibility of agentic research in astronomy by applying a self-improving, multi-agent system to real JWST data, including the newly discovered 'Little Red Dots'.

Proposed method

• mephisto employs a multi-agent collaboration framework that mimics human expert reasoning in SED fitting, interacting with the CIGALE codebase to build and refine physical models.
• The framework uses a self-play mechanism where agents explore hypotheses, reflect on model-data discrepancies, and iteratively improve SED models through tree search and action planning.
• It maintains a dynamically updated external knowledge base via knowledge distillation and validation, enabling long-term learning and model diversity.
• Input state is represented as a JSON structure containing observed fluxes (d), current SED model (m), and fitting results (r), including χ² and number of well-fitted bands.
• The evaluation process uses fuzzy, human-like logic to assess model quality, prioritizing physical plausibility over optimal χ² alone, thus avoiding unphysical solutions.
• Agents perform reasoning by analyzing discrepancies, proposing changes to model parameters (e.g., star formation history, dust attenuation, AGN fraction), and validating outcomes through repeated CIGALE calls.

Experimental results

Research questions

• RQ1 Can LLM-based agents autonomously perform complex, multi-step reasoning in astronomical SED fitting without explicit reward functions?
• RQ2 To what extent can agentic systems emulate human expert reasoning in interpreting ambiguous, high-redshift galaxy observations such as 'Little Red Dots'?
• RQ3 Can multi-agent collaboration with self-reflection and knowledge distillation improve model diversity and accuracy in SED fitting compared to standard optimization?
• RQ4 How effective is the framework in identifying physically plausible, degenerate solutions (e.g., AGN vs. dusty star-forming galaxies) for challenging observational cases?
• RQ5 Can the system discover non-obvious physical scenarios—'unknown unknowns'—through hypothesis exploration rather than hyperparameter tuning?

Key findings

• mephisto achieved near-human proficiency in reasoning about the physical nature of galaxies, particularly in interpreting complex, multimodal SED data from the James Webb Space Telescope.
• For five representative 'Little Red Dot' galaxies, mephisto identified two distinct, physically plausible solutions: one with a dominant AGN and low dust (Av ≈ 0.07–1.35 mag), and another with high dust attenuation (Av ≈ 0.36–10.1 mag) and no AGN.
• In the case of JADES LRD 79803 (z = 5.4007), mephisto proposed a solution with Av = 0.03 ± 0.13 mag and fracAGN = 0.99, and an alternative with Av = 0.92 ± 0.46 mag and no AGN, both fitting the data well.
• For JADES LRD 90354 (z = 7.6641), mephisto generated a solution with Av = 10.1 ± 2.2 mag and fracAGN = 0.99, indicating extreme dust obscuration, and a less dusty alternative with Av = 3.01 ± 0.53 mag and no AGN.
• The framework demonstrated the ability to explore physically degenerate solutions, such as in JADES LRD 99915, where both low-dust solutions with and without AGN provided good fits, highlighting the importance of hypothesis diversity.
• mephisto successfully navigated the 'unknown unknown' space by generating and evaluating non-traditional physical scenarios, such as low-mass, AGN-dominated galaxies (e.g., log M∗ = 6.68 ± 1.12 for LRD 120484), which may be overlooked by conventional fitting methods.