[논문 리뷰] Complementarity in Human-AI Collaboration: Concept, Sources, and Evidence

Patrick Hemmer, Max Schemmer외 3명|arXiv (Cornell University)|2024. 03. 21.

Ethics and Social Impacts of AI인용 수 12

한 줄 요약

본 논문은 human-AI 보완성에 대한 이론적 프레임워크를 개발하고, 보완성 잠재력(CP)과 그 고유(inherent) 및 협력적(collaborative) 구성요소를 정의하며, 정보 비대칭성과 능력 비대칭성 두 가지 경험적 연구를 통해 이를 검증한다.

ABSTRACT

Artificial intelligence (AI) has the potential to significantly enhance human performance across various domains. Ideally, collaboration between humans and AI should result in complementary team performance (CTP) -- a level of performance that neither of them can attain individually. So far, however, CTP has rarely been observed, suggesting an insufficient understanding of the principle and the application of complementarity. Therefore, we develop a general concept of complementarity and formalize its theoretical potential as well as the actual realized effect in decision-making situations. Moreover, we identify information and capability asymmetry as the two key sources of complementarity. Finally, we illustrate the impact of each source on complementarity potential and effect in two empirical studies. Our work provides researchers with a comprehensive theoretical foundation of human-AI complementarity in decision-making and demonstrates that leveraging these sources constitutes a viable pathway towards designing effective human-AI collaboration, i.e., the realization of CTP.

연구 동기 및 목표

• Define the concept of human-AI complementarity and formalize complementarity potential (CP).
• Identify and categorize sources of CP, including information and capability asymmetries.
• Differentiate inherent and collaborative CP and relate them to realized performance gains (CE).
• Empirically validate the framework in two studies: information asymmetry in real estate appraisal and capability asymmetry in image classification.

제안 방법

• Formalize decision-making with human (H) and AI (AI) decisions and a collaboration mechanism I(·).
• Define complementarity potential (CP) as the difference between the best single actor’s loss and 0-loss ground truth, decomposed into CPinh and CPcoll.
• Define realized complementarity effect (CE) as the realized loss reduction from collaboration, with CEinh and CEcoll components.
• Derive equations for CP, CPinh, CPcoll, CE, and CE components (Equations 3–10).
• Develop two experimental studies to isolate information asymmetry (real estate valuation) and capability asymmetry (image classification).
• Discuss implications for designing and evaluating human-AI collaboration in decision-making.

실험 결과

연구 질문

• RQ1 RQ1: How can we model human-AI decision-making to enable a nuanced understanding of synergies in a human-AI team?
• RQ2 RQ2: What factors contribute to complementary team performance (CTP) in human-AI decision-making?

주요 결과

• Complementarity potential (CP) is decomposed into inherent CP and collaborative CP, and together they explain how H and AI can outperform each on their own.
• Evidence from two studies shows that asymmetric information and asymmetric capabilities can increase CP and enable CTP in decision tasks.
• In the real estate study, humans leveraged unique contextual information alongside AI predictions to contribute to CP.
• In the image-classification study, heterogeneous capabilities between humans and AI contributed to CP and enabled CTP.
• The framework provides a way to measure realized collaboration (CE) and its inherent/collaborative components across tasks.