[Paper Review] A deep learning approach to search for superconductors from electronic bands

Jun Li, Wenqi Fang+6 more|arXiv (Cornell University)|Sep 12, 2024

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper proposes BNAS, a 3D Vision Transformer-based deep learning model that directly links electronic band structures to superconducting transition temperatures ($T_c$), identifying key band regions correlated with superconductivity via attention mechanisms. The model predicts 14,956 potential superconductors from a high-throughput screening of 46,442 materials, including promising candidates based on Mn, Co, Li, and Mg, offering new insights into unconventional superconductivity mechanisms.

ABSTRACT

Energy band theory is a foundational framework in condensed matter physics. In this work, we employ a deep learning method, BNAS, to find a direct correlation between electronic band structure and superconducting transition temperature. Our findings suggest that electronic band structures can act as primary indicators of superconductivity. To avoid overfitting, we utilize a relatively simple deep learning neural network model, which, despite its simplicity, demonstrates predictive capabilities for superconducting properties. By leveraging the attention mechanism within deep learning, we are able to identify specific regions of the electronic band structure most correlated with superconductivity. This novel approach provides new insights into the mechanisms driving superconductivity from an alternative perspective. Moreover, we predict several potential superconductors that may serve as candidates for future experimental synthesis.

Motivation & Objective

• To establish a direct, data-driven correlation between electronic band structures and superconducting transition temperatures ($T_c$) using deep learning.
• To overcome limitations of traditional methods that rely on chemical composition or simplified physical parameters by using raw band structure data as input.
• To identify specific regions within electronic bands most predictive of superconductivity using attention mechanisms in a 3D-ViT model.
• To enable high-throughput discovery of novel superconductors by screening large material databases using learned band-based patterns.
• To provide theoretical and predictive guidance for future experimental synthesis of high-temperature superconductors.

Proposed method

• A 3D Vision Transformer (3D-ViT) model is trained on a curated dataset of electronic band structures derived from DFT calculations, with input dimensions of 18×32×32×32.
• The model uses a set of optimized hyperparameters including $L_d$ = 534, $D_e$ = 0.127, $H_d$ = 64, $D_m$ = 0.197, $M_d$ = 1038, and $L_t$ = 3.
• Stochastic gradient descent (SGD) with a learning rate of 0.001, momentum 0.9, weight decay $10^{-5}$, and batch size 64 is used for training over 700 epochs.
• Attention Rollout is applied to visualize how different parts of the band structure influence the model’s $T_c$ predictions, revealing critical regions across the Brillouin zone.
• The model is trained on a cleaned dataset from the SuperCon, Materials Project (MP), and OQMD databases, with disordered structures filtered using an order transformation method.
• High-throughput screening of 46,442 materials with band gaps ≤ 0.2 eV is performed using the trained model to identify potential superconductors.

Experimental results

Research questions

• RQ1 Can electronic band structures alone serve as reliable predictors of superconducting transition temperature ($T_c$) without relying on chemical composition or empirical parameters?
• RQ2 Which specific regions within the electronic band structure are most predictive of superconductivity, and can they be identified using attention mechanisms?
• RQ3 Can a deep learning model trained on DFT-derived band structures discover novel superconductor candidates beyond known elemental or compound categories?
• RQ4 Do materials with similar band structures but different chemical compositions exhibit comparable superconducting potential, and can the model detect such patterns?
• RQ5 Can the model distinguish between conventional and unconventional superconducting behavior based on band topology and symmetry?

Key findings

• The BNAS model achieves a validation loss of approximately 0.06 after 700 training epochs, indicating stable and effective learning on the band structure data.
• The attention mechanism successfully identifies crystalline symmetry-related features in band structures, with attention maps highlighting regions near the Fermi surface and near high-symmetry points.
• The model predicts 14,956 potential superconductors from a screening of 46,442 materials with band gaps ≤ 0.2 eV, significantly expanding the candidate pool.
• Manganese (Mn), cobalt (Co), iron (Fe), nickel (Ni), and copper (Cu) appear frequently among predicted candidates, suggesting potential for new unconventional superconductors.
• Lithium- and magnesium-based compounds are also identified as strong candidates, with one lithium-based superconductor predicted, consistent with known conventional superconductors like MgB₂.
• The model demonstrates that electronic band structures can serve as primary indicators of superconductivity, offering a new theoretical and predictive pathway beyond traditional composition-based or parameter-based models.