[论文解读] Preserving privacy in domain transfer of medical AI models comes at no performance costs: The integral role of differential privacy

Soroosh Tayebi Arasteh, Mahshad Lotfinia+7 more|arXiv (Cornell University)|Jun 10, 2023

Privacy-Preserving Technologies in Data参考文献 33被引用 4

一句话总结

该论文表明，在医疗AI模型的领域迁移过程中应用差分隐私（DP）不会带来显著的性能损失，即使在高隐私级别（ε ≈ 1）下亦是如此。基于来自五个机构的超过59万张胸部X光片，作者证明，经过DP增强的领域迁移在五种疾病上的诊断准确性与非DP方法相当，几乎所有亚组的AUC差异均小于1%。

ABSTRACT

Developing robust and effective artificial intelligence (AI) models in medicine requires access to large amounts of patient data. The use of AI models solely trained on large multi-institutional datasets can help with this, yet the imperative to ensure data privacy remains, particularly as membership inference risks breaching patient confidentiality. As a proposed remedy, we advocate for the integration of differential privacy (DP). We specifically investigate the performance of models trained with DP as compared to models trained without DP on data from institutions that the model had not seen during its training (i.e., external validation) - the situation that is reflective of the clinical use of AI models. By leveraging more than 590,000 chest radiographs from five institutions, we evaluated the efficacy of DP-enhanced domain transfer (DP-DT) in diagnosing cardiomegaly, pleural effusion, pneumonia, atelectasis, and in identifying healthy subjects. We juxtaposed DP-DT with non-DP-DT and examined diagnostic accuracy and demographic fairness using the area under the receiver operating characteristic curve (AUC) as the main metric, as well as accuracy, sensitivity, and specificity. Our results show that DP-DT, even with exceptionally high privacy levels (epsilon around 1), performs comparably to non-DP-DT (P>0.119 across all domains). Furthermore, DP-DT led to marginal AUC differences - less than 1% - for nearly all subgroups, relative to non-DP-DT. Despite consistent evidence suggesting that DP models induce significant performance degradation for on-domain applications, we show that off-domain performance is almost not affected. Therefore, we ardently advocate for the adoption of DP in training diagnostic medical AI models, given its minimal impact on performance.

研究动机与目标

• 评估在向未见机构进行领域迁移时，差分隐私（DP）是否会降低医疗AI模型的性能。
• 评估多中心医疗影像AI模型中隐私与诊断准确率之间的权衡。
• 调查DP是否能在跨领域医疗图像分类中保持不同人口统计子组的公平性。
• 确定高隐私DP设置（ε ≈ 1）是否能在真实世界部署场景中保持临床实用性。
• 倡导在医疗AI训练流程中集成DP，而不会损害诊断性能。

提出的方法

• 在来自五个机构的59万张胸部X光片的大型多中心数据集上训练了一个基于视觉Transformer的模型。
• 在领域迁移阶段应用差分隐私，通过精心校准的隐私预算（ε ≈ 1）进行噪声注入。
• 在训练期间未见过的机构的数据上进行外部验证，以模拟真实世界的临床部署。
• 使用AUC、准确率、敏感性与特异性等标准指标，评估五项诊断任务（心脏扩大、胸腔积液、肺炎、肺不张和健康对照）的模型性能。
• 在所有任务和人口统计子组中，将DP增强的领域迁移（DP-DT）与非DP领域迁移（non-DP-DT）进行比较。
• 进行子组分析，以评估人口统计公平性，并确保隐私保护性能在多样化人群中保持一致。

实验结果

研究问题

• RQ1 在领域迁移过程中应用差分隐私是否会降低医疗影像模型的诊断性能？
• RQ2 DP-DT在未见机构的外部验证中与non-DP-DT相比表现如何？
• RQ3 高隐私DP（ε ≈ 1）对不同诊断任务的AUC、准确率、敏感性与特异性有何影响？
• RQ4 与non-DP-DT相比，DP-DT是否在人口统计子组中保持了公平性？
• RQ5 差分隐私能否在不损害跨领域设置临床实用性的前提下，有效集成到医疗AI中？

主要发现

• 在所有五项诊断任务中，DP-DT的AUC值与non-DP-DT相比在统计上无法区分（所有比较的p > 0.119）。
• 在几乎所有子组和任务中，DP-DT与non-DP-DT之间的最大AUC差异小于1%。
• 在高隐私DP设置下，任何诊断条件下均未观察到敏感性、特异性或准确率的显著性能下降。
• DP-DT在人口统计子组中保持了稳定的性能，表明对公平性的影响极小。
• 结果表明，差分隐私可在医疗AI中大规模应用，而不会损害跨领域设置下的诊断准确性。
• 本研究提供了实证证据，表明在医疗AI领域迁移中，隐私与性能并非相互排斥，支持在临床AI流程中采用DP。