[Paper Review] MLography: An Automated Quantitative Metallography Model for Impurities Anomaly Detection using Novel Data Mining and Deep Learning Approach

Matan Rusanovsky, Gal Oren+2 more|arXiv (Cornell University)|Feb 27, 2020

Mineral Processing and Grinding14 references4 citations

TL;DR

MLography proposes a novel deep learning and data mining framework for automated, quantitative anomaly detection in metallographic impurities within engineering alloys. It introduces three complementary anomaly measures—spatial, shape, and area—using distance-based metrics, autoencoders, and market clustering to identify and score anomalous inclusions, with validation showing strong alignment between model-predicted anomalies and physical inspection results.

ABSTRACT

The micro-structure of most of the engineering alloys contains some inclusions and precipitates, which may affect their properties, therefore it is crucial to characterize them. In this work we focus on the development of a state-of-the-art artificial intelligence model for Anomaly Detection named MLography to automatically quantify the degree of anomaly of impurities in alloys. For this purpose, we introduce several anomaly detection measures: Spatial, Shape and Area anomaly, that successfully detect the most anomalous objects based on their objective, given that the impurities were already labeled. The first two measures quantify the degree of anomaly of each object by how each object is distant and big compared to its neighborhood, and by the abnormally of its own shape respectively. The last measure, combines the former two and highlights the most anomalous regions among all input images, for later (physical) examination. The performance of the model is presented and analyzed based on few representative cases. We stress that although the models presented here were developed for metallography analysis, most of them can be generalized to a wider set of problems in which anomaly detection of geometrical objects is desired. All models as well as the data-set that was created for this work, are publicly available at: https://github.com/matanr/MLography.

Motivation & Objective

• To develop an objective, quantitative alternative to expert-based anomaly assessment in metallographic analysis.
• To automate the detection and scoring of impurity anomalies in engineering alloys using AI.
• To create a publicly available dataset and model for reproducible research in quantitative metallography.
• To enable early identification of potentially defective material regions through multi-scale anomaly scoring.

Proposed method

• Proposes a novel spatial anomaly measure based on distance from k-th nearest neighbor, adjusted for object area.
• Introduces a shape anomaly score using autoencoders to reconstruct impurity contours and quantify deviation from normal shapes.
• Employs market clustering to group impurities into regions and compute an area anomaly score based on cluster cohesion and outlier inclusion.
• Combines spatial and shape anomaly scores multiplicatively to produce a composite anomaly score for each cluster.
• Uses a trained autoencoder with normal impurities only to detect shape anomalies in test images.
• Orders clusters by anomaly score using Equation 2 to prioritize regions for physical examination.

Experimental results

Research questions

• RQ1 Can a data-driven, deep learning-based approach reliably detect anomalous impurities in metallographic images without relying on expert opinion?
• RQ2 How can spatial, shape, and area-based anomaly measures be combined to improve detection of potentially defective regions in alloys?
• RQ3 To what extent do model-predicted anomaly scores correlate with physical defect indicators such as microhardness and composition?
• RQ4 Can the model distinguish between normal and anomalous inclusions when physical tests show no measurable difference?

Key findings

• The model successfully ranked the most anomalous clusters in test scans at positions 1588, 1642, and 916 out of 1653 clusters, with the most anomalous clusters falling within the first decile.
• No significant difference was found in microhardness or EDS composition between anomalous and normal inclusions, indicating that visual anomalies may not always correlate with physical property changes.
• The spatial and shape anomaly measures effectively identified regions with high deviation from normal patterns, even when physical tests did not confirm defects.
• The area anomaly measure, using market clustering, successfully highlighted clusters with high internal anomaly scores, enabling prioritization of regions for further inspection.
• The model’s anomaly scores were monotonically increasing with the degree of anomaly, validating the mathematical consistency of the scoring framework.
• The entire model, including the dataset of 243 annotated metallographic scans, is publicly available at https://github.com/matanr/MLography.