[Paper Review] Bridging microscopy with molecular dynamics and quantum simulations: An AtomAI based pipeline

Ayana Ghosh, Maxim Ziatdinov+3 more|arXiv (Cornell University)|Sep 9, 2021

Machine Learning in Materials Science85 references19 citations

TL;DR

This paper presents an AtomAI-based machine learning pipeline that bridges experimental transmission electron microscopy (TEM) data directly to molecular dynamics (MD) and quantum simulations, using pre-trained deep neural networks to convert imaging data into physical descriptors. The workflow enables real-time, physics-informed simulation of atomic-scale dynamics—demonstrated on graphene with Cr ad-atom adsorption and self-healing—by overcoming time-scale and data-latency disparities between experiment and simulation.

ABSTRACT

Recent advances in (scanning) transmission electron microscopy have enabled routine generation of large volumes of high-veracity structural data on 2D and 3D materials, naturally offering the challenge of using these as starting inputs for atomistic simulations. In this fashion, theory will address experimentally emerging structures, as opposed to the full range of theoretically possible atomic configurations. However, this challenge is highly non-trivial due to the extreme disparity between intrinsic time scales accessible to modern simulations and microscopy, as well as latencies of microscopy and simulations per se. Addressing this issue requires as a first step bridging the instrumental data flow and physics-based simulation environment, to enable the selection of regions of interest and exploring them using physical simulations. Here we report the development of the machine learning workflow that directly bridges the instrument data stream into Python-based molecular dynamics and density functional theory environments using pre-trained neural networks to convert imaging data to physical descriptors. The pathways to ensure the structural stability and compensate for the observational biases universally present in the data are identified in the workflow. This approach is used for a graphene system to reconstruct optimized geometry and simulate temperature-dependent dynamics including adsorption of Cr as an ad-atom and graphene healing effects. However, it is universal and can be used for other material systems.

Motivation & Objective

• Address the challenge of integrating high-fidelity experimental microscopy data with atomistic simulations to enable theory-driven analysis of experimentally observed structures.
• Overcome the mismatch in time scales and data latencies between microscopy and physics-based simulations, which hinders real-time feedback during experiments.
• Develop a machine learning workflow that converts raw imaging data into physical descriptors (e.g., atomic positions, energies) suitable for input into MD and DFT environments.
• Ensure structural stability and mitigate observational biases in microscopy data through a robust, ensemble-based deep learning framework.
• Enable feedback-driven simulation workflows by integrating the pipeline into live STEM experiments for dynamic material response prediction.

Proposed method

• Utilize pre-trained U-Net convolutional neural networks (with 2-3-3-4-3-3-2 architecture) to infer atomic positions and physical descriptors directly from 256×256 STEM image windows.
• Implement an ensemble of 20 independently trained AtomAI models with different random seeds to improve prediction robustness and reduce uncertainty.
• Map predicted atomic positions from deep learning to simulation-ready input files for DFT, AIMD, and DFTB simulations using Python-based interfaces.
• Perform DFT geometry optimizations using VASP with PAW-PBE functionals and 400-eV plane-wave cutoff, enforcing convergence when Hellmann-Feynman forces drop below 10⁻³ eV/Å.
• Conduct ab initio molecular dynamics (AIMD) at 300–1200 K with Nose-Hoover thermostats and 1-fs timesteps over 2000 steps, using the same DFT parameters.
• Calculate adsorption energies via the formula: E_adsorbate = E_system + E_adsorbate – E(system + adsorbate), to quantify Cr ad-atom binding stability.

Experimental results

Research questions

• RQ1 Can a deep learning pipeline reliably convert experimental STEM images into physically consistent atomic configurations suitable for ab initio simulations?
• RQ2 How can observational biases and noise in microscopy data be mitigated to ensure structural stability and physical plausibility in downstream simulations?
• RQ3 To what extent can the workflow enable real-time, feedback-driven simulation of dynamic processes such as ad-atom adsorption and self-healing in 2D materials?
• RQ4 How do temperature-dependent dynamics of Cr ad-atoms and surface reconstruction manifest in graphene under AIMD simulations initiated from experimental images?
• RQ5 Can the integration of DFTB simulations extend the scalability of the pipeline to larger systems (e.g., 2,043 atoms) while preserving accuracy?

Key findings

• The AtomAI pipeline successfully converted experimental STEM images of graphene into simulation-ready atomic structures with sub-angstrom precision, enabling direct simulation without manual reconstruction.
• Ensemble prediction using 20 independently trained models reduced uncertainty and improved robustness, ensuring physically stable configurations across multiple runs.
• AIMD simulations initiated from AtomAI-predicted structures revealed Cr ad-atoms preferentially bind at hollow sites at 300 K, with increasing mobility and bond weakening at higher temperatures (up to 1200 K).
• The system exhibited self-healing behavior at elevated temperatures (≥700 K), where carbon vacancies restructured and re-bonded, consistent with experimental observations of dynamic surface recovery.
• DFTB simulations on the full 2,043-atom graphene supercell achieved convergence in a few hours, enabling long-timescale dynamics and non-local effects such as long-range self-healing to be probed.
• The end-to-end workflow, including data conversion, simulation setup, and execution, was completed in under 10 CPU hours for a 91-atom system and under a few hours for the full 2,043-atom system, making it feasible for real-time feedback during STEM experiments.