[Paper Review] Large Language Model-Guided Prediction Toward Quantum Materials Synthesis

Ryotaro Okabe, Zack West+6 more|arXiv (Cornell University)|Oct 28, 2024

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper presents a large language model (LLM)-based framework that predicts synthesis pathways for inorganic and quantum materials by fine-tuning on a text-mined synthesis database. The approach achieves up to 90% accuracy using a generalized Tanimoto similarity metric, significantly outperforming baseline models and demonstrating robustness across materials with varying quantum character.

ABSTRACT

The synthesis of inorganic crystalline materials is essential for modern technology, especially in quantum materials development. However, designing efficient synthesis workflows remains a significant challenge due to the precise experimental conditions and extensive trial and error. Here, we present a framework using large language models (LLMs) to predict synthesis pathways for inorganic materials, including quantum materials. Our framework contains three models: LHS2RHS, predicting products from reactants; RHS2LHS, predicting reactants from products; and TGT2CEQ, generating full chemical equations for target compounds. Fine-tuned on a text-mined synthesis database, our model raises accuracy from under 40% with pretrained models, to under 80% using conventional fine-tuning, and further to around 90% with our proposed generalized Tanimoto similarity, while maintaining robust to additional synthesis steps. Our model further demonstrates comparable performance across materials with varying degrees of quantumness quantified using quantum weight, indicating that LLMs offer a powerful tool to predict balanced chemical equations for quantum materials discovery.

Motivation & Objective

• To address the challenge of designing efficient synthesis workflows for inorganic and quantum materials, which are often hindered by trial-and-error experimentation.
• To develop an LLM-based framework capable of predicting complete chemical equations from reactants, products, or target compounds.
• To improve prediction accuracy beyond pretrained models and conventional fine-tuning, especially for complex materials with high quantum character.
• To ensure robustness to additional synthesis operation prompts (e.g., heating, quenching) without accuracy degradation.
• To introduce a novel similarity metric, generalized Tanimoto similarity (GTS), that allows for atomic permutation in formulas, enhancing flexibility and accuracy in chemical equation evaluation.

Proposed method

• Fine-tune three distinct LLMs: LHS2RHS (reactants to products), RHS2LHS (products to reactants), and TGT2CEQ (target compound to full chemical equation).
• Train models on a text-mined synthesis database using a 9:1 training-to-test split, with 100 epochs and 10-fold cross-validation, repeated for 200 epochs to ensure convergence.
• Use beam-search multinomial sampling with two beams, enabled sampling, and fixed-length decoding to balance diversity and accuracy while preventing redundant generation.
• Introduce a generalized Tanimoto similarity (GTS) metric that permits permutation of atoms within chemical formulas, improving robustness over standard Jaccard similarity (JS).
• Incorporate prompts describing synthesis operations (e.g., heating, mixing) to test model robustness, maintaining high accuracy under such perturbations.
• Evaluate model performance using both Jaccard similarity and the proposed GTS, with quantum weight $K_{xx}$ used to quantify the quantum character of materials in evaluation.

Figure 1: Overview of large language model prediction of the synthesis process. Synthesis protocols are pre-processed into structured data ( ? ) (the lower left figure is taken from the same reference), which specify target compounds, precursors, operations, and conditions. The synthesis pathway of

Experimental results

Research questions

• RQ1 Can LLMs be effectively fine-tuned to predict complete, balanced chemical equations for inorganic materials synthesis?
• RQ2 How does the generalized Tanimoto similarity (GTS) metric improve prediction accuracy compared to standard Jaccard similarity in chemical equation matching?
• RQ3 To what extent can LLM-based models generalize across materials with varying degrees of quantum character, as quantified by quantum weight $K_{xx}$?
• RQ4 Does the inclusion of synthesis operation prompts (e.g., heating, quenching) degrade model performance, and can robustness be maintained?
• RQ5 Can the proposed framework reduce reliance on trial-and-error in quantum materials synthesis by accurately predicting viable reaction pathways?

Key findings

• The LLM-based framework achieves a prediction accuracy of approximately 90% using the generalized Tanimoto similarity (GTS) metric, significantly improving upon the under 40% accuracy of pretrained models and under 80% with conventional fine-tuning.
• The generalized Tanimoto similarity (GTS) metric outperforms standard Jaccard similarity by allowing for atomic permutation within formulas, enabling more flexible and accurate evaluation of chemical equation predictions.
• The models maintain high accuracy and consistency even when prompted with additional synthesis operations (e.g., heating, mixing), demonstrating robustness to perturbations in input descriptions.
• The framework shows comparable performance across materials with varying quantum weight $K_{xx}$, indicating its effectiveness for predicting synthesis pathways in quantum materials regardless of their degree of quantum character.
• The TGT2CEQ model successfully generates full chemical equations from target compounds alone, enabling forward prediction of viable synthesis routes without prior reactant input.
• The source code for the framework is publicly available at https://github.com/RyotaroOKabe/llm4syn, supporting reproducibility and further development.

Figure 2: Workflow to compute the generalized Tanimoto similarity (GTS). For any pair of predicted and ground truth chemical equations, each chemical formula, represented by A, B, … J, from both equations is vectorized into vector of element counts $\mathbf{v}_{\text{A}},\mathbf{v}_{\text{B}},...\ma