[论文解读] Exploiting full Resolution Feature Context for Liver Tumor and Vessel Segmentation via Integrate Framework: Application to Liver Tumor and Vessel 3D Reconstruction under embedded microprocessor

Xiangyu Meng, Xudong Zhang+6 more|arXiv (Cornell University)|Nov 26, 2021

Radiomics and Machine Learning in Medical Imaging被引用 6

一句话总结

该论文提出TransFusionNet，一种基于Transformer的多尺度特征融合网络，通过利用全分辨率特征上下文，提升了CT图像中肝脏肿瘤和血管的分割性能。通过整合全局上下文建模与高效的特征重建，该方法在肝脏肿瘤和血管分割上取得了SOTA的Dice分数（分别为0.961和0.899），并成功将模型部署于嵌入式微处理器上，实现了高精度的实时3D重建。

ABSTRACT

Liver cancer is one of the most common malignant diseases in the world. Segmentation and labeling of liver tumors and blood vessels in CT images can provide convenience for doctors in liver tumor diagnosis and surgical intervention. In the past decades, many state-of-the-art medical image segmentation algorithms appeared during this period. With the development of embedded devices, embedded deployment for medical segmentation and automatic reconstruction brings prospects for future automated surgical tasks. Yet, most of the existing segmentation methods mostly care about the spatial feature context and have a perception defect in the semantic relevance of medical images, which significantly affects the segmentation accuracy of liver tumors and blood vessels. Deploying large and complex models into embedded devices requires a reasonable trade-off between model accuracy, reasoning speed and model capacity. Given these problems, we introduce a multi-scale feature fusion network called TransFusionNet based on Transformer. This network achieved very competitive performance for liver vessel and liver tumor segmentation tasks, meanwhile it can improve the recognition of morphologic margins of liver tumors by exploiting the global information of CT images. Experiments show that in vessel segmentation task TransFusionNet achieved mean Dice coefficients of 0.899 and in liver tumor segmentation task TransFusionNet achieved mean Dice coefficients of 0.961. Compared with the state-of-the-art framework, our model achieves the best segmentation result. In addition, we deployed the model into an embedded micro-structure and constructed an integrated model for liver tumor vascular segmentation and reconstruction. This proprietary structure will be the exclusive component of the future medical field.

研究动机与目标

• 解决现有医学图像模型中因语义与空间上下文有限而导致的肝脏肿瘤和血管分割精度低的问题。
• 开发一种轻量化、高精度的分割模型，适用于在资源受限的嵌入式微处理器上部署。
• 实现实时、自动化的肝脏肿瘤与血管结构3D重建，用于临床应用。
• 提升对细小血管边缘和肿瘤边界识别的能力，这对手术规划至关重要。
• 弥合高精度深度学习模型与临床嵌入式系统中实际部署之间的差距。

提出的方法

• 提出TransFusionNet，一种类似U-Net的编码器-解码器架构，通过多尺度特征融合解码器恢复高分辨率特征。
• 集成视觉Transformer（ViT）编码器，以捕捉长距离全局上下文，提升对复杂解剖结构的语义理解。
• 采用知识蒸馏技术，将完整版TransFusionNet压缩为适合嵌入式部署的轻量化模型。
• 应用后处理量化技术，进一步减小模型尺寸，并加速在Jetson TX2上的推理速度。
• 采用基于高斯滤波的后处理步骤以优化分割掩码：y = G * argmax(o)，其中o为预测的标签图。
• 将蒸馏与量化后的模型部署于嵌入式Jetson TX2系统，实现实时从CT体数据重建3D模型。

实验结果

研究问题

• RQ1 基于Transformer的网络能否有效利用全分辨率特征上下文，提升CT图像中肝脏肿瘤与血管的分割性能？
• RQ2 多尺度特征融合在多大程度上增强了模型恢复细微解剖细节的能力，尤其是在肿瘤边界与小血管区域？
• RQ3 在不造成显著精度损失的前提下，大型高精度分割模型能在多大程度上被压缩并部署于嵌入式微处理器上？
• RQ4 嵌入式部署是否能够实现实时、高保真度的肝脏肿瘤与血管结构3D重建？
• RQ5 与当前SOTA模型相比，该方法在Dice分数与重建保真度方面表现如何？

主要发现

• 在LITS数据集上，TransFusionNet在肝脏肿瘤分割上实现了0.961的平均Dice系数，优于现有SOTA方法。
• 在3DIRCADB数据集上，该模型在肝脏血管分割上实现了0.899的平均Dice系数，表现出卓越性能。
• 在作者自建标注数据集上，模型在肿瘤分割上达到0.927的IOU，在血管分割上达到0.822，表明其具有强大的泛化能力。
• 知识蒸馏与量化流程成功将模型压缩，适用于Jetson TX2嵌入式系统部署。
• 从嵌入式系统重建的3D模型与人工标注高度相似，仅存在少量噪声与细微细节损失。
• 该系统实现了肝脏肿瘤与血管的实时、自动化3D重建，相比传统人工标注显著减少了工作量。