[Paper Review] MudrockNet: Semantic Segmentation of Mudrock SEM Images through Deep Learning

Abhishek Bihani, Hugh Daigle+4 more|arXiv (Cornell University)|Feb 5, 2021

Electron and X-Ray Spectroscopy Techniques70 references55 citations

TL;DR

MudrockNet is a deep learning-based semantic segmentation model built on TensorFlow using Google's DeepLab-v3+ architecture to automatically identify silt grains and pores in mudrock scanning electron microscope (SEM) images. Trained on ground-truth data from Kumano Basin mudrocks, it achieved a mean intersection over union (IoU) of 0.6591 for silt grains and 0.6642 for pores, outperforming traditional methods like Weka random forest in accuracy and speed.

ABSTRACT

Segmentation and analysis of individual pores and grains of mudrocks from scanning electron microscope images is non-trivial because of noise, imaging artifacts, variation in pixel grayscale values across images, and overlaps in grayscale values among different physical features such as silt grains, clay grains, and pores in an image, which make their identification difficult. Moreover, because grains and pores often have overlapping grayscale values, direct application of threshold-based segmentation techniques is not sufficient. Recent advances in the field of computer vision have made it easier and faster to segment images and identify multiple occurrences of such features in an image, provided that ground-truth data for training the algorithm is available. Here, we propose a deep learning SEM image segmentation model, MudrockNet based on Google's DeepLab-v3+ architecture implemented with the TensorFlow library. The ground-truth data was obtained from an image-processing workflow applied to scanning electron microscope images of uncemented muds from the Kumano Basin offshore Japan at depths < 1.1 km. The trained deep learning model obtained a pixel-accuracy about 90%, and predictions for the test data obtained a mean intersection over union (IoU) of 0.6591 for silt grains and 0.6642 for pores. We also compared our model with the random forest classifier using trainable Weka segmentation in ImageJ, and it was observed that MudrockNet gave better predictions for both silt grains and pores. The size, concentration, and spatial arrangement of the silt and clay grains can affect the petrophysical properties of a mudrock, and an automated method to accurately identify the different grains and pores in mudrocks can help improve reservoir and seal characterization for petroleum exploration and anthropogenic waste sequestration.

Motivation & Objective

• To develop an automated, deep learning-based method for semantic segmentation of pores and silt grains in mudrock SEM images, reducing reliance on manual, user-guided segmentation workflows.
• To overcome challenges in traditional segmentation such as noise, grayscale overlap between features (e.g., clay vs. silt vs. pores), and inconsistent pixel intensity across images.
• To improve the accuracy and efficiency of digital rock physics analysis by enabling high-throughput, repeatable, and scalable segmentation of microstructural features in mudrocks.
• To demonstrate that deep learning models can outperform conventional thresholding and random forest-based segmentation in complex grayscale SEM images of fine-grained rocks.
• To provide a publicly available, trained model and codebase to support reproducibility and future development in mudrock image analysis.

Proposed method

• The model is based on the DeepLab-v3+ architecture, a state-of-the-art semantic segmentation framework using atrous spatial pyramid pooling (ASPP) and encoder-decoder structure for dense prediction.
• The network was trained using a dataset of 100 SEM images from uncemented muds in the Kumano Basin, Japan, with ground-truth labels generated via a conventional image-processing workflow.
• Training was performed using the TensorFlow library, with data augmentation applied to improve generalization despite limited training samples.
• The model uses pixel-wise cross-entropy loss and Adam optimization to learn end-to-end feature representations from raw grayscale SEM images.
• Inference was evaluated using mean intersection over union (IoU) and pixel accuracy, with comparisons made against a Weka-based random forest classifier in ImageJ.
• Post-processing steps were applied to identify individual grains and pores via instance-level segmentation using connected component labeling.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve higher accuracy than conventional threshold-based segmentation in distinguishing silt grains and pores in grayscale mudrock SEM images?
• RQ2 How does the performance of MudrockNet compare to a traditional machine learning classifier (e.g., random forest) in terms of IoU and inference speed?
• RQ3 To what extent can a deep learning model generalize across variations in grayscale intensity and imaging artifacts in SEM images of fine-grained rocks?
• RQ4 Can the model segment large-area SEM images (e.g., 2048×1767 pixels) that exceed the training image size with high fidelity?
• RQ5 What is the impact of limited training data and overlapping grayscale values on model generalization and prediction reliability?

Key findings

• MudrockNet achieved a mean IoU of 0.6591 for silt grains and 0.6642 for pores on test data, indicating strong performance in distinguishing these features in complex grayscale images.
• The model outperformed the Weka random forest classifier in both silt grain and pore segmentation, with higher IoU values and significantly faster inference time.
• Prediction time for MudrockNet was 0.218 seconds per image, compared to 7.767 seconds for the conventional segmentation algorithm, representing a 35-fold speedup.
• On a large test image (15,000× magnification, 2048×1767 pixels), MudrockNet achieved an IoU of 0.739 for silt grains and 0.682 for pores, demonstrating robustness to larger image sizes.
• Despite high accuracy, the model showed limitations in detecting very small silt grains due to limited training data and grayscale overlap with clay, suggesting a need for data augmentation or size-based filtering.
• The ground truth data itself contained uncertainty due to manual filtering steps, which may have contributed to discrepancies between expert-labeled and model-predicted segmentations.