[Paper Review] Reduction of detection limit and quantification uncertainty due to interferent by neural classification with abstention

Alex Hagen‐Zanker, Ken Jarman+6 more|arXiv (Cornell University)|Apr 22, 2022

Nuclear Physics and Applications19 references1 citations

TL;DR

This paper proposes a neural classification method with abstention to reduce detection limits and quantification uncertainty in physical counting experiments with interfering background events. By dynamically setting optimal decision thresholds based on classifier confidence, the method achieves up to 3× lower uncertainty than accuracy-optimized models, especially at low signal-to-background ratios.

ABSTRACT

Many measurements in the physical sciences can be cast as counting experiments, where the number of occurrences of a physical phenomenon informs the prevalence of the phenomenon's source. Often, detection of the physical phenomenon (termed signal) is difficult to distinguish from naturally occurring phenomena (termed background). In this case, the discrimination of signal events from background can be performed using classifiers, and they may range from simple, threshold-based classifiers to sophisticated neural networks. These classifiers are often trained and validated to obtain optimal accuracy, however we show that the optimal accuracy classifier does not generally coincide with a classifier that provides the lowest detection limit, nor the lowest quantification uncertainty. We present a derivation of the detection limit and quantification uncertainty in the classifier-based counting experiment case. We also present a novel abstention mechanism to minimize the detection limit or quantification uncertainty \emph{a posteriori}. We illustrate the method on two data sets from the physical sciences, discriminating Ar-37 and Ar-39 radioactive decay from non-radioactive events in a gas proportional counter, and discriminating neutrons from photons in an inorganic scintillator and report results therefrom.

Motivation & Objective

• To address the limitation of accuracy-optimized classifiers in minimizing detection limits and quantification uncertainty.
• To demonstrate that maximal accuracy does not yield optimal performance in metrologically critical metrics.
• To develop and validate a novel abstention mechanism that improves detection and quantification performance.
• To show the method's effectiveness across diverse physical science datasets with low signal-to-background ratios.

Proposed method

• Proposes a threshold optimization strategy based on classifier confidence scores to minimize detection limit or quantification uncertainty.
• Introduces an abstention mechanism that excludes low-confidence events from classification to improve overall performance.
• Uses simplex-based optimization to determine optimal thresholds for minimal uncertainty across varying analyte-to-interferent ratios.
• Applies the method to two real-world datasets: Ar-37/Ar-39 discrimination in gas proportional counters and neutron-photon discrimination in scintillators.
• Employs continuous classifier outputs (e.g., neural networks) to enable confidence-based decision-making.
• Validates performance using true activity estimation and uncertainty bands, comparing optimal thresholds to maximal-accuracy thresholds.

Experimental results

Research questions

• RQ1 Does a classifier optimized for maximum accuracy minimize detection limit and quantification uncertainty in counting experiments with background interference?
• RQ2 Can abstaining from low-confidence events improve detection limit and measurement uncertainty?
• RQ3 How does the performance of optimal-threshold classification compare to accuracy-optimized classification across varying signal-to-background ratios?
• RQ4 Can the abstention method be generalized across different classifiers and physical measurement systems?
• RQ5 What is the quantitative impact of threshold optimization on uncertainty reduction in low-SBR scenarios?

Key findings

• The optimal threshold for minimizing quantification uncertainty does not coincide with the threshold that maximizes accuracy.
• At low analyte-to-interferent ratios, the optimal uncertainty is up to 3× smaller than that achieved with maximal-accuracy thresholds.
• Even when estimated activity is unbiased, the uncertainty band for maximal-accuracy classification can include zero within two standard deviations, indicating a failure to detect low-level signals.
• The abstention mechanism effectively isolates ambiguous events in phase space, particularly in overlapping signal and background regions.
• The method reduces uncertainty across all tested signal-to-background ratios and achieves significant improvement in low-SBR regimes.
• The approach is generalizable and effective across different classifiers and physical measurement systems, including gas proportional counters and inorganic scintillators.