[Paper Review] Feature Recalibration Based Olfactory-Visual Multimodal Model for Enhanced Rice Deterioration Detection

Rongqiang Zhao, Hengrui Hu+3 more|arXiv (Cornell University)|Feb 16, 2026

Smart Agriculture and AI0 citations

TL;DR

Proposes a feature-recalibration olfactory–visual multimodal framework (FDEC + FDRA-Net) to detect fine-grained rice deterioration using RGB images and e-nose signals, achieving near-perfect offline accuracy and strong field performance.

ABSTRACT

Multimodal methods are widely used in rice deterioration detection, but they exhibit limited capability in representing and extracting fine-grained abnormal features. Moreover, these methods rely on devices such as hyperspectral cameras and mass spectrometers, which increase detection costs and prolong data acquisition time. To address these issues, we propose a feature recalibration based olfactory-visual multimodal model for enhanced rice deterioration detection. A fine-grained deterioration embedding constructor (FDEC) is proposed to reconstruct the labeled multimodal embedded feature dataset, thereby enhancing sample representation. A fine-grained deterioration recalibration attention network (FDRA-Net) is proposed to emphasize signal variations and improve sensitivity to fine-grained deterioration on the rice surface. Compared with SS-Net, the proposed method improves classification accuracy by 8.67%, with an average improvement of 11.51% over other traditional baseline models, while simultaneously simplifying the detection procedure. Furthermore, field detection results demonstrate advantages in both accuracy and operational simplicity. The proposed method can also be extended to other agrifood applications in agriculture and the food industry.

Motivation & Objective

• Improve representation of fine-grained rice deterioration cues by leveraging multimodal data (visual and olfactory).
• Reduce reliance on costly spectral imaging devices while maintaining high detection accuracy.
• Reconstruct and enrich multimodal embeddings to enhance sample representation for downstream detection.
• Develop a modality-aware attention mechanism to emphasize discriminative regions and olfactory channels.
• Demonstrate practical viability in offline and field (online) rice quality assessment scenarios.

Proposed method

• Introduce FDEC to produce a structured multimodal embedding by independently processing olfactory and visual data and organizing them into a unified embedding set.
• Apply PLR to enhance olfactory signals and partition visual data into local patches processed by a patch-level convolutional embedder.
• Develop FDRA-Net with SE-based channel recalibration for olfactory features and CBAM-based visual attention to focus on sparse, fine-grained mold patterns.
• Stack multiple FDRA layers to progressively recalibrate and fuse modalities, followed by linear projections and a classifier for three-class output (Expired, Moldy, Normal).
• Train and evaluate using offline dataset with data from eight days for training/validation and one day for testing; perform online field tests with real granary samples.

Experimental results

Research questions

• RQ1 Can a feature recalibration–driven olfactory–visual framework outperform single-modality and existing multimodal methods in fine-grained rice deterioration detection?
• RQ2 Does the FDEC embedding construction improve sample representation and downstream detection robustness?
• RQ3 Do SE and CBAM submodules provide complementary benefits for olfactory and visual modalities in a joint detection task?
• RQ4 Is the approach practically deployable with RGB cameras and a cheap e-nose in real storage environments?
• RQ5 How does the model perform in online field conditions compared to offline evaluation?

Key findings

• FDRA-Net achieves 99.89% accuracy on the offline test set, with Moldy at 100.00% and Expired at 100.00% precision/recall in the reported results.
• FDRA-Net outperforms a range of baselines (CNN, NAM, ResNet, SENet, SKNet, SS-Net) in overall accuracy and recalls, with acceptable model size (~48.7 MB) and latency (~9.48 ms).
• Ablation shows FDEC is critical, with removing it dropping accuracy from 99.89% to 85.44%; SE and CBAM provide complementary gains for olfactory and visual features.
• Grad-CAM analysis indicates FDRA-Net concentrates attention on localized mold-related regions, unlike the model without FDRA.
• Online field tests yield 93.33% accuracy, with some drop in Expired detection under real-world conditions, yet still outperforming baselines.