[论文解读] DeepRA: Predicting Joint Damage From Radiographs Using CNN with Attention

Neelambuj Chaturvedi|arXiv (Cornell University)|Feb 13, 2021

Rheumatoid Arthritis Research and Therapies被引用 6

一句话总结

该论文提出DeepRA，一种基于注意力机制的两阶段卷积神经网络，用于检测手部和足部X光片中的关节，并使用Sharp/van der Heijde（SvH）方法预测关节水平的狭窄与侵蚀评分。其在狭窄和侵蚀预测上的加权均方根误差（RMSE）分别降低了31%和19%，整体损伤预测的绝对误差减少了79%，并通过注意力图提供了可解释性。

ABSTRACT

Joint damage in Rheumatoid Arthritis (RA) is assessed by manually inspecting and grading radiographs of hands and feet. This is a tedious task which requires trained experts whose subjective assessment leads to low inter-rater agreement. An algorithm which can automatically predict the joint level damage in hands and feet can help optimize this process, which will eventually aid the doctors in better patient care and research. In this paper, we propose a two-staged approach which amalgamates object detection and convolution neural networks with attention which can efficiently and accurately predict the overall and joint level narrowing and erosion from patients radiographs. This approach has been evaluated on hands and feet radiographs of patients suffering from RA and has achieved a weighted root mean squared error (RMSE) of 1.358 and 1.404 in predicting joint level narrowing and erosion Sharp van der Heijde (SvH) scores which is 31% and 19% improvement with respect to the baseline SvH scores, respectively. The proposed approach achieved a weighted absolute error of 1.456 in predicting the overall damage in hands and feet radiographs for the patients which is a 79% improvement as compared to the baseline. Our method also provides an inherent capability to provide explanations for model predictions using attention weights, which is essential given the black box nature of deep learning models. The proposed approach was developed during the RA2 Dream Challenge hosted by Dream Challenges and secured 4th and 8th position in predicting overall and joint level narrowing and erosion SvH scores from radiographs.

研究动机与目标

• 自动化类风湿性关节炎（RA）手部和足部X光片中关节水平损伤的预测，减少放射科医生对主观、耗时的人工评分的依赖。
• 通过训练基于RetinaNet的目标检测模型，解决X光片中缺乏关节标注的问题，以定位手指和腕部。
• 通过引入注意力增强的卷积神经网络，提升对Sharp/van der Heijde（SvH）评分中关节狭窄与侵蚀的预测准确性。
• 通过注意力图实现模型的内在可解释性，增强临床应用中对深度学习预测结果的信任。
• 在自动化RA损伤评分方面超越基线方法，尤其在整体损伤和关节水平损伤预测方面表现更优。

提出的方法

• 采用两阶段框架：首先，使用RetinaNet目标检测模型在X光片中定位手指和腕部。
• 其次，基于EfficientNet和DenseNet主干网络并集成自注意力模块的关节水平回归模型，预测狭窄与侵蚀的SvH评分。
• 注意力机制使模型聚焦于检测到的关节内的显著区域，提升表征学习能力并增强可解释性。
• 通过CBAM风格模块集成注意力机制，实现基于空间与通道重要性的特征重校准。
• 在RA X光片上对模型进行微调，并在RA2 Dream Challenge测试集上使用加权RMSE和绝对误差进行评估。
• 可视化注意力图以验证模型是否聚焦于生物上相关的区域，如关节间隙和侵蚀区域。

实验结果

研究问题

• RQ1 结合目标检测与基于注意力机制的CNN的两阶段深度学习框架，能否提升自动化RA关节损伤评分的准确性？
• RQ2 在从X光片预测SvH评分时，引入注意力机制在多大程度上提升了性能与可解释性？
• RQ3 当缺乏真实关节标注时，通过RetinaNet实现的关节定位在多大程度上能支持准确的下游关节水平损伤预测？
• RQ4 注意力图能否为模型预测提供临床有意义的解释，从而增强对自动化评分的信任？
• RQ5 将关节检测与注意力机制相结合，是否能显著提升性能，超越未包含这些组件的基线模型？

主要发现

• 所提方法在关节水平狭窄的加权RMSE为1.358，侵蚀评分为1.404，分别较基线提升31%和19%。
• 模型将整体RA损伤预测的加权绝对误差降低至1.456，相比基线的7.023降低了79%。
• 基于EfficientNet的模型在结合关节检测与注意力机制后，整体评分和关节水平评分分别在RA2 Dream Challenge中获得第4名和第8名。
• 注意力图清晰突出了手指和腕部中关节间隙与侵蚀区域等显著区域，证实模型聚焦于生物相关特征。
• 目标检测与注意力机制的结合显著提升了性能，而微调后的DenseNet-169在缺乏这些组件时仅获得微小提升。
• 注意力机制提供了内在可解释性，使临床医生可通过检查输入图像中注意力加权区域来验证模型预测。