[论文解读] Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes

Qian Yang, Aaron Steinfeld+1 more|arXiv (Cornell University)|Apr 21, 2019

Artificial Intelligence in Healthcare and Education参考文献 24被引用 69

一句话总结

本研究设计并在现场测试一种预后性决策支持工具，该工具通过生成带有嵌入式机器预后信息的幻灯片静默地融入临床医生的决策会议，目标是在危重护理中实现不显眼、符合工作流程的AI。它报告在三个VAD中心的现场评估以及跨领域医生的访谈，以评估采用度和推广性。

ABSTRACT

Clinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of "Unremarkable Computing", that by augmenting the users' routines technology/AI can have significant importance for the users yet remain unobtrusive. Our field evaluation suggests clinicians are more likely to encounter and embrace such a DST. Drawing on their responses, we discuss the importance and intricacies of finding the right level of unremarkableness in DST design, and share lessons learned in prototyping critical AI systems as a situated experience.

研究动机与目标

• Investigate why prognostic decision-support tools struggle to be adopted in clinical practice.
• Design a DST that integrates seamlessly into clinicians’ existing decision-making workflow.
• Evaluate whether subtly embedded prognostic information on decision slides is acceptable and effective.
• Assess the generalizability of the unremarkable DST design to other critical medical decisions.

提出的方法

• Embed DST predictions into the top-right corner of automatically generated decision-meeting slides populated from EMR data.
• Use a field-driven design process grounded in Unremarkable Computing to keep the tool subservient to routine workflows.
• Conduct multi-site field studies in three VAD implant centers with one-on-one interviews and observed decision meetings.
• Employ synthetic patient cases for prototyping and evaluate clinicians’ reactions and discussions.
• Analyze data via affinity diagramming and thematic analysis to extract insights about acceptance, practicality, and generalizability.

实验结果

研究问题

• RQ1 Can a DST be encountered naturally within clinicians’ current decision-making workflow?
• RQ2 Will clinicians accept computational decision support when it is publicly visible during meetings?
• RQ3 Does placing the prediction in a corner provide the right level of unremarkableness to slow decisions only when there is disagreement?
• RQ4 Is the unremarkable DST design generalizable to other critical medical decision meetings?

主要发现

• Clinicians are likely to encounter the DST output when embedded in decision meeting slides across sites.
• There is broad acceptance of prognostic DSTs in the meeting context, with perceived value in providing additional context and a different perspective.
• The right level of unremarkableness is nuanced; the DST slow-down effect is not guaranteed and depends on meeting dynamics and data realism.
• Clinicians prefer models that are validated, locally applicable, and linked to credible evidence; synthetic data pose interpretation challenges.
• Mid-level clinicians feel empowered by clear, visual meeting slides, while senior clinicians maintain control over decision agendas; hierarchy shapes DST use and influence.
• The design shows potential generalizability to other disciplines that use interdisciplinary decision meetings, though real-patient validation and trust are critical for adoption.