[论文解读] Spatiotemporal Feature Learning Based on Two-Step LSTM and Transformer for CT Scans

Chih–Chung Hsu, Chi-Han Tsai+3 more|arXiv (Cornell University)|Jul 4, 2022

COVID-19 diagnosis using AI被引用 6

一句话总结

本文提出一种两阶段时空特征学习框架，用于CT扫描中的COVID-19分类，采用2D CNN提取切片级嵌入，随后利用LSTM与Swin Transformer模型进行时序建模。该方法在保持较低假阴性率（LSTM）和假阳性率（Swin）的同时，实现了最先进性能，表明其在诊断可靠性方面具有互补优势。

ABSTRACT

Computed tomography (CT) imaging could be very practical for diagnosing various diseases. However, the nature of the CT images is even more diverse since the resolution and number of the slices of a CT scan are determined by the machine and its settings. Conventional deep learning models are hard to tickle such diverse data since the essential requirement of the deep neural network is the consistent shape of the input data. In this paper, we propose a novel, effective, two-step-wise approach to tickle this issue for COVID-19 symptom classification thoroughly. First, the semantic feature embedding of each slice for a CT scan is extracted by conventional backbone networks. Then, we proposed a long short-term memory (LSTM) and Transformer-based sub-network to deal with temporal feature learning, leading to spatiotemporal feature representation learning. In this fashion, the proposed two-step LSTM model could prevent overfitting, as well as increase performance. Comprehensive experiments reveal that the proposed two-step method not only shows excellent performance but also could be compensated for each other. More specifically, the two-step LSTM model has a lower false-negative rate, while the 2-step Swin model has a lower false-positive rate. In summary, it is suggested that the model ensemble could be adopted for more stable and promising performance in real-world applications.

研究动机与目标

• 解决因患者间CT扫描分辨率和切片数量不一致，导致深度学习模型泛化困难的问题。
• 开发一种稳健方法，从长度可变的CT扫描中提取时空特征，实现可靠的COVID-19分类。
• 通过两阶段流水线将特征提取与时序建模解耦，降低过拟合风险并提升性能。
• 比较基于LSTM与Transformer模型的诊断可靠性，重点关注假阴性和假阳性率。

提出的方法

• 使用EfficientNet-B3作为2D CNN主干网络，从单个CT切片中提取224维语义嵌入。
• 采用两阶段方法：首先提取切片级特征，然后利用双向LSTM（4层，128个隐藏单元）建模切片间的时序依赖关系。
• 对于基于Transformer的分支，每例扫描随机采样224个切片（若少于224个则放回采样），以标准化Swin Transformer的输入长度。
• 通过平均10至50组随机切片采样集的预测结果进行模型集成，以提升鲁棒性并降低方差。
• 使用Adam优化器，初始学习率为0.0001，权重衰减为0.0005进行训练，最终预测采用0.5分类阈值。

实验结果

研究问题

• RQ1 两阶段深度学习框架能否有效处理分辨率和切片数量不一致的可变长度CT扫描？
• RQ2 将特征提取与时序建模分离，是否能降低过拟合风险并提升医学影像数据的分类性能？
• RQ3 在COVID-19 CT分类任务中，基于LSTM与Transformer的模型在假阴性和假阳性率方面表现如何？
• RQ4 对互补架构（LSTM与Swin）进行模型集成，能否带来更稳定可靠的诊断结果？

主要发现

• 两阶段LSTM模型达到0.900的敏感度（SE），表明其假阴性率低于其他模型。
• 两阶段Swin Transformer模型达到0.981的特异性（SP），表明其假阳性率低于其他模型。
• 2D CNN基线模型的宏平均F1分数为0.930，经50次随机切片采样集平均后提升至0.937。
• 两阶段LSTM模型在敏感度方面优于2D CNN和ADLeaST，表现出更强的真正阳性病例检测能力。
• 两阶段Swin模型达到最高特异性（0.981），展现出更优的非COVID病例正确识别能力。
• 建议将LSTM与Swin分支的模型集成，以结合低假阴性和低假阳性性能，实现真实世界诊断的稳定性。