[Paper Review] The 4D Camera: an 87 kHz direct electron detector for scanning/transmission electron microscopy

Peter Ercius, I. J. M. Johnson+20 more|arXiv (Cornell University)|May 19, 2023

Advanced Electron Microscopy Techniques and Applications4 citations

TL;DR

The 4D Camera is an 87 kHz direct electron detector with a 576×576 pixel active pixel sensor designed for scanning and transmission electron microscopy, enabling real-time 4D-STEM data acquisition at 480 Gbit/s. By leveraging electron counting and sparsity-based processing, it reduces raw data by 10–300×, allowing rapid analysis of massive datasets and enabling high-speed, high-sensitivity structural characterization of materials such as solid-state battery interfaces.

ABSTRACT

We describe the development, operation, and application of the 4D Camera -- a 576 by 576 pixel active pixel sensor for scanning/transmission electron microscopy which operates at 87,000 Hz. The detector generates data at approximately 480 Gbit/s which is captured by dedicated receiver computers with a parallelized software infrastructure that has been implemented to process the resulting 10 - 700 Gigabyte-sized raw datasets. The back illuminated detector provides the ability to detect single electron events at accelerating voltages from 30 - 300 keV. Through electron counting, the resulting sparse data sets are reduced in size by 10 - 300x compared to the raw data, and open-source sparsity-based processing algorithms offer rapid data analysis. The high frame rate allows for large and complex 4D-STEM experiments to be accomplished with typical STEM scanning parameters.

Motivation & Objective

• To develop a high-speed, high-sensitivity direct electron detector for scanning and transmission electron microscopy capable of capturing 4D-STEM data at unprecedented frame rates.
• To enable real-time, high-fidelity electron scattering data acquisition with minimal dead time and improved quantum efficiency.
• To reduce massive raw datasets (10–700 GB) through electron counting and sparsity-based processing for rapid analysis.
• To integrate the detector with high-performance computing resources to provide feedback in minutes rather than days.
• To support complex, multi-modal electron microscopy experiments by treating the STEM as a multi-functional beamline capturing full scattering information.

Proposed method

• The 4D Camera employs a back-illuminated, 576×576 pixel active pixel sensor operating at 87,000 Hz, enabling real-time acquisition of 4D-STEM data at 480 Gbit/s.
• It uses direct electron detection with single-electron event sensitivity across 30–300 keV accelerating voltages, eliminating the need for scintillators.
• Raw data is processed in real time using a parallelized software stack with four FPGAs and four receiver servers, buffering data until scan completion.
• Event-based processing identifies and counts individual electron events, reducing data volume by 10–300× through sparsity-based algorithms.
• Data is offloaded to a high-performance computing system (NERSC) via a dedicated 'Mothership 6' PC for rapid local or remote analysis.
• Advanced reconstruction techniques such as rotational summing of diffraction patterns generate vADF-STEM images from full 4D datasets.

Figure 1: A schematic of the 4D Camera detector, data acquisition, and data processing system. Each 1/4 sector of the detector is separately processed by four FPGAs and forwarded to four receiver servers. The receiver servers buffer the data in main memory at the full data rate until a scan is compl

Experimental results

Research questions

• RQ1 Can a direct electron detector achieve 87 kHz frame rates while maintaining single-electron sensitivity across a broad energy range?
• RQ2 To what extent can electron counting and sparsity-based processing reduce the size of 4D-STEM datasets without loss of structural information?
• RQ3 How does the integration of high-speed data acquisition with high-performance computing enable real-time feedback in electron microscopy experiments?
• RQ4 Can the 4D Camera enable new imaging modalities such as vADF-STEM and ptychographic reconstruction at scale?
• RQ5 What structural insights can be extracted from large-scale 4D-STEM data of complex materials like solid-state battery interfaces?

Key findings

• The 4D Camera achieves a frame rate of 87,000 Hz with a data rate of 480 Gbit/s, enabling high-speed 4D-STEM acquisition.
• The detector achieves single-electron sensitivity at accelerating voltages from 30 to 300 keV, significantly improving signal-to-noise and dynamic range.
• Electron counting reduces raw 4D-STEM datasets by a factor of 10 to 300×, enabling efficient storage and processing.
• Sparsity-based processing algorithms allow rapid analysis of datasets up to 700 GB, reducing feedback time from days to minutes.
• vADF-STEM images reconstructed from full 4D-STEM data reveal crystallographic orientation and structural variations in LiCoO₂, including grain growth direction and interface roughness.
• The data reveals preferential crystal growth along the vertical direction in LiCoO₂, supported by anisotropic intensity in diffraction rings matching the scanning direction.

Figure 2: Schematics of (a)-(b) 80 kV and (c)-(d) 300 kV electron scattering (red lines) in 10 $\mu$ m wide APS pixels (vertical gray lines). (a) and (c) show how a thinner inactive layer reduces electron scattering leading to reduced lateral energy deposition compared to a sensor with a thicker ina