[论文解读] DualMamba: A Lightweight Spectral-Spatial Mamba-Convolution Network for Hyperspectral Image Classification

Jiamu Sheng, Jingyi Zhou+3 more|arXiv (Cornell University)|Jun 11, 2024

Remote-Sensing Image Classification被引用 5

一句话总结

DualMamba 提出了一种轻量级并行架构，将基于 Mamba 的全局光谱-空间建模与用于局部特征的轻量级 CNN 相结合，在公开高光谱图像数据集上以更少参数和 FLOPs 实现更高的准确性。

ABSTRACT

The effectiveness and efficiency of modeling complex spectral-spatial relations are both crucial for Hyperspectral image (HSI) classification. Most existing methods based on CNNs and transformers still suffer from heavy computational burdens and have room for improvement in capturing the global-local spectral-spatial feature representation. To this end, we propose a novel lightweight parallel design called lightweight dual-stream Mamba-convolution network (DualMamba) for HSI classification. Specifically, a parallel lightweight Mamba and CNN block are first developed to extract global and local spectral-spatial features. First, the cross-attention spectral-spatial Mamba module is proposed to leverage the global modeling of Mamba at linear complexity. Within this module, dynamic positional embedding is designed to enhance the spatial location information of visual sequences. The lightweight spectral/spatial Mamba blocks comprise an efficient scanning strategy and a lightweight Mamba design to efficiently extract global spectral-spatial features. And the cross-attention spectral-spatial fusion is designed to learn cross-correlation and fuse spectral-spatial features. Second, the lightweight spectral-spatial residual convolution module is proposed with lightweight spectral and spatial branches to extract local spectral-spatial features through residual learning. Finally, the adaptive global-local fusion is proposed to dynamically combine global Mamba features and local convolution features for a global-local spectral-spatial representation. Compared with state-of-the-art HSI classification methods, experimental results demonstrate that DualMamba achieves significant classification accuracy on three public HSI datasets and a superior reduction in model parameters and floating point operations (FLOPs).

研究动机与目标

• 通过建模全局和局部光谱-空间关系来实现高效且准确的高光谱图像（HSI）分类。
• 开发一种轻量级并行架构，解耦全局和局部特征提取。
• 引入新颖模块，以高效捕捉全局光谱-空间上下文和局部光谱-空间细节。

提出的方法

• 引入一种并行双流设计，结合用于全局特征的跨注意力光谱-空间 Mamba 模块和用于局部特征的轻量级光谱-空间残差卷积模块。
• 在扫描前使用动态位置嵌入以增强空间位置信息。
• 实现具有高效扫描策略的轻量级时空 Mamba 块（空间单向扫描与光谱双向扫描）。
• 通过自适应全局-局部融合模块调节它们的贡献，从而融合全局与局部特征。
• 在融合后的全局-局部光谱-空间表示上应用简单分类器。

实验结果

研究问题

• RQ1 在保持轻量级的同时，是否可以用并行的 Mamba-卷积架构有效建模 HSIs 的全局光谱-空间关系？
• RQ2 动态位置嵌入和专门的扫描策略是否能提升全球光谱-空间特征提取？
• RQ3 自适应全局-局部特征融合是否能在减少参数和 FLOPs 的同时实现更优性能？

主要发现

• DualMamba 在三个公开的 HSI 数据集上实现了卓越的分类准确性。
• 与最先进方法相比，该模型显著减少了参数量和 FLOPs。
• 带有动态位置嵌入的跨注意力光谱-空间 Mamba 能够有效捕捉全局光谱-空间关系。
• 轻量级光谱-空间残差卷积模块高效学习局部光谱-空间特征。
• 自适应全局-局部融合动态平衡全局与局部特征，从而得到鲁棒的表征。