[Paper Review] Setting an attention region for convolutional neural networks using region selective features, for recognition of materials within glass vessels

Sagi Eppel|arXiv (Cornell University)|Aug 29, 2017

Industrial Vision Systems and Defect DetectionEngineering18 citations

TL;DR

This paper proposes a valve filter mechanism in convolutional neural networks that selectively applies different convolutional filters to a predefined region of interest (ROI), such as the interior of glass vessels, to improve material recognition. By using a binary ROI map to gate feature extraction, the model enhances attention on relevant regions, achieving superior performance on a new dataset of 1,000 annotated lab glassware images compared to standard FCNs without ROI guidance.

ABSTRACT

Convolutional neural networks have emerged as the leading method for the classification and segmentation of images. In some cases, it is desirable to focus the attention of the net on a specific region in the image; one such case is the recognition of the contents of transparent vessels, where the vessel region in the image is already known. This work presents a valve filter approach for focusing the attention of the net on a region of interest (ROI). In this approach, the ROI is inserted into the net as a binary map. The net uses a different set of convolution filters for the ROI and background image regions, resulting in a different set of features being extracted from each region. More accurately, for each filter used on the image, a corresponding valve filter exists that acts on the ROI map and determines the regions in which the corresponding image filter will be used. This valve filter effectively acts as a valve that inhibits specific features in different image regions according to the ROI map. In addition, a new data set for images of materials in glassware vessels in a chemistry laboratory setting is presented. This data set contains a thousand images with pixel-wise annotation according to categories ranging from filled and empty to the exact phase of the material inside the vessel. The results of the valve filter approach and fully convolutional neural nets (FCN) with no ROI input are compared based on this data set.

Motivation & Objective

• To improve material recognition in transparent glass vessels by focusing CNN attention on the vessel's interior region.
• To address the challenge of low-contrast, variable-reflective, and partially obscured materials inside transparent containers.
• To develop a method that dynamically adapts feature extraction based on spatial regions using a learnable gating mechanism.
• To create and release a new, large-scale, pixel-annotated dataset of materials in glassware for benchmarking.
• To compare the performance of the proposed ROI-aware network against standard fully convolutional networks without ROI input.

Proposed method

• A binary ROI map is used as input to guide the network, specifying which regions (e.g., vessel interior) require focused attention.
• For each standard convolutional filter, a corresponding 'valve filter' is introduced that modulates filter application based on the ROI map.
• The valve filter acts as a learnable gate, inhibiting or enabling feature activation in specific image regions depending on the ROI map.
• The network uses separate feature extraction pathways for ROI and background regions, enabling region-specific representation learning.
• The method is integrated into a fully convolutional neural network (FCN) framework to maintain spatial resolution and enable dense prediction.
• The model is trained end-to-end using pixel-wise cross-entropy loss on the new dataset with multi-class material annotations.

Experimental results

Research questions

• RQ1 Can a region-selective attention mechanism improve material recognition accuracy in transparent glass vessels?
• RQ2 How does the performance of a CNN with ROI-guided feature learning compare to a standard FCN without ROI input?
• RQ3 To what extent does the proposed valve filter mechanism enhance feature discrimination in the vessel interior compared to the background?
• RQ4 How effective is the new dataset in supporting training and evaluation of attention-based models for lab glassware imaging?
• RQ5 Does the use of a binary ROI map as a gating signal lead to more robust and generalizable feature learning?

Key findings

• The proposed valve filter approach significantly outperformed standard FCNs without ROI input on the new dataset, demonstrating improved segmentation accuracy.
• The model achieved higher mean Intersection over Union (mIoU) on material categories such as 'filled', 'empty', and phase-specific classes like 'liquid' and 'solid'.
• The use of region-selective features reduced background interference and improved feature clarity in the vessel interior, especially under low-contrast conditions.
• The new dataset of 1,000 images with pixel-wise annotations proved effective for training and evaluating attention-based models in lab imaging.
• The valve filter mechanism enabled the network to learn distinct feature representations for the vessel region and background, enhancing localization precision.
• The ablation study confirmed that the ROI-guided attention mechanism was a key factor in performance gains, not merely data augmentation or architectural changes.