[论文解读] The Risk to Population Health Equity Posed by Automated Decision Systems: A Narrative Review

Mitchell Burger|arXiv (Cornell University)|Jan 18, 2020

Artificial Intelligence in Healthcare and Education被引用 6

一句话总结

这篇叙事综述探讨了医疗与公共卫生领域中的自动化决策系统（ADS）如何通过嵌入的偏见、缺乏透明度以及隐私侵蚀，加剧人口健康不平等。基于学术文献与灰色文献，本文认为，受新冠疫情加速推动，狭义人工智能在临床与公共卫生领域的迅速采用，若缺乏审慎治理，将可能放大现有的社会、经济与健康不平等。

ABSTRACT

Artificial intelligence is already ubiquitous, and is increasingly being used to autonomously make ever more consequential decisions. However, there has been relatively little research into the existing and possible consequences for population health equity. A narrative review was undertaken using a hermeneutic approach to explore current and future uses of narrow AI and automated decision systems (ADS) in medicine and public health, issues that have emerged, and implications for equity. Accounts reveal a tremendous expectation on AI to transform medical and public health practices. Prominent demonstrations of AI capability - particularly in diagnostic decision making, risk prediction, and surveillance - are stimulating rapid adoption, spurred by COVID-19. Automated decisions being made have significant consequences for individual and population health and wellbeing. Meanwhile, it is evident that hazards including bias, incontestability, and privacy erosion have emerged in sensitive domains such as criminal justice where narrow AI and ADS are in common use. Reports of issues arising from their use in health are already appearing. As the use of ADS in health expands, it is probable that these hazards will manifest more widely. Bias, incontestability, and privacy erosion give rise to mechanisms by which existing social, economic and health disparities are perpetuated and amplified. Consequently, there is a significant risk that use of ADS in health will exacerbate existing population health inequities. The industrial scale and rapidity with which ADS can be applied heightens the risk to population health equity. It is incumbent on health practitioners and policy makers therefore to explore the potential implications of using ADS, to ensure the use of artificial intelligence promotes population health and equity.

研究动机与目标

• 调查自动化决策系统（ADS）在医疗与公共卫生背景下对人口健康公平性构成的风险。
• 审视健康领域中ADS所呈现的现有与新兴风险，如算法偏见、不可申诉性及隐私侵蚀。
• 评估ADS在医疗领域广泛应用的长期影响，特别是对精准医学与公共卫生监测中健康差异的影响。
• 强调医疗从业者与政策制定者亟需主动评估ADS影响，以确保实现公平结果。
• 呼吁对AI在医疗领域的部署进行系统性审视，以防止既存不平等被强化或加剧。

提出的方法

• 采用诠释学方法，对学术与灰色文献进行叙事综述，以解读与综合研究发现。
• 聚焦于医疗与公共卫生领域中的狭义人工智能及自动化决策系统，包括诊断工具、风险预测模型与资金分配算法。
• 分析刑事司法与教育等高风险领域中的案例研究与报告，识别可迁移至健康语境的风险。
• 评估ADS可能持续或加剧健康不平等的机制，包括结构性偏见与问责缺失。
• 综合有关算法决策过程中隐私侵蚀与不透明性的证据。
• 运用批判性视角评估大规模ADS在人群健康中部署的社会与伦理影响。

实验结果

研究问题

• RQ1 医疗与公共卫生领域中的自动化决策系统如何危及健康公平？
• RQ2 在健康语境中，ADS所引发的主要风险（如偏见、不可申诉性、隐私损失）是什么？
• RQ3 人工智能在医疗领域，尤其是在新冠疫情后，快速推广可能如何加剧现有健康差异？
• RQ4 算法决策机制以何种方式再现或放大社会、经济与健康不平等？
• RQ5 医疗从业者与政策制定者在确保AI应用促进而非损害人群健康公平方面，应承担何种责任？

主要发现

• 自动化决策系统在医疗与公共卫生领域中正日益用于疾病检测、诊断、治疗规划与医疗资金分配，对个体与人群健康产生重大影响。
• 已有报告指出健康相关ADS中存在算法偏见与隐私侵蚀问题，其表现与刑事司法及教育系统中的问题相似。
• 在人群健康领域中，ADS的快速且工业化规模的部署，由于系统性偏见，可能加剧现有的社会、经济与健康不平等。
• ADS决策过程中缺乏透明度与不可申诉性，削弱了问责机制并损害信任，尤其对边缘化群体影响显著。
• AI有潜力减少全球健康不平等，但若治理不善，其风险可能使不平等固化或恶化。
• 若缺乏主动监管，ADS在医疗领域的扩展极有可能固化结构性不平等，尤其在精准公共卫生与风险预测应用中。