[Paper Review] Soft Attention Convolutional Neural Networks for Rare Event Detection in Sequences

Mandar Kulkarni, Aria Abubakar|arXiv (Cornell University)|Nov 2, 2020

Time Series Analysis and Forecasting4 citations

TL;DR

This paper proposes a soft-attention U-Net-style CNN architecture that combines global and local receptive fields to detect rare geological markers in well logs with high precision. By leveraging attention mechanisms to dynamically weight relevant features, the model achieves robust performance across diverse markers and log combinations, outperforming prior methods in well log correlation tasks.

ABSTRACT

Automated event detection in the sequences is an important aspect of temporal data analytics. The events can be in the form of peaks, changes in data distribution, changes of spectral characteristics etc. In this work, we propose a Soft-Attention Convolutional Neural Network (CNN) based approach for rare event detection in sequences. For the purpose of demonstration, we experiment with well logs where we aim to detect events depicting the changes in the geological layers (a.k.a. well tops/markers). Well logs (single or multivariate) are inputted to a soft attention CNN and a model is trained to locate the marker position. Attention mechanism enables the machine to relatively scale the relevant log features for the task. Experimental results show that our approach is able to locate the rare events with high precision.

Motivation & Objective

• Address the challenge of rare event detection in long sequential well log data where marker events are sparse and geologically significant.
• Emulate expert interpreters’ workflow by enabling the model to focus on relevant log patterns using attention mechanisms.
• Develop a single, reusable model architecture that generalizes across different markers and input log types (e.g., GR, GR-RES, GR-RES-DEN) without case-specific tuning.
• Improve detection accuracy for subtle markers that are difficult to identify manually due to weak or ambiguous signatures.

Proposed method

• Employ a dual-branch CNN architecture: a global-view U-Net with skip connections and an encoder-decoder structure, and a local-view stack of 1D convolutions with dilated filters.
• Apply soft attention by element-wise multiplication of the global and local feature maps, followed by a sigmoid activation to produce attention-weighted features.
• Use inception blocks with multiple filter sizes in both global and local branches to capture multi-scale patterns in the well logs.
• Apply Gaussian label smoothing to mitigate class imbalance caused by sparse marker annotations, improving training stability.
• Utilize Monte Carlo dropout for uncertainty estimation, with detection validity determined by probability and uncertainty thresholds (set to 5 ft).
• Train end-to-end using binary cross-entropy loss, with early stopping based on validation loss.

Experimental results

Research questions

• RQ1 Can a soft-attention CNN effectively localize rare geological markers in well logs despite their low frequency and subtle signatures?
• RQ2 How does the integration of global and local receptive fields improve marker detection compared to using either branch alone?
• RQ3 To what extent does the proposed model generalize across different markers and combinations of input logs (e.g., GR, GR-RES, GR-RES-DEN)?
• RQ4 Does label smoothing improve model performance on imbalanced rare event detection tasks in well log data?
• RQ5 Can uncertainty estimation via Monte Carlo dropout help identify reliable predictions in low-confidence regions?

Key findings

• The model achieves high precision in marker detection, with over 80% of predictions within 5 ft of the expert-annotated marker depth when using GR logs.
• Incorporating multiple log types (e.g., GR-RES-DEN) improves precision compared to single-log inputs, particularly for strong signature markers.
• The model successfully detects subtle markers even when visual signatures are weak, suggesting that global context aids localization.
• The ablation study confirms that the combined global-local attention mechanism outperforms models using only one branch, demonstrating the value of multi-receptive-field learning.
• The F1-score of the proposed method exceeds that of the baseline approach by Maniar et al. (2018), particularly at a 2 ft tolerance threshold.
• Model uncertainty estimates via Monte Carlo dropout effectively identify low-confidence predictions, with a 5 ft threshold used to filter unreliable detections.