[论文解读] Converging Measures and an Emergent Model: A Meta-Analysis of Human-Automation Trust Questionnaires

Yosef S. Razin, Karen M. Feigh|arXiv (Cornell University)|Mar 24, 2023

Cognitive Functions and Memory被引用 8

一句话总结

本论文对人机协作中经过验证的信任问卷进行元分析，以识别趋同因素、映射术语，并提出一个新兴的、整合的信任模型。

ABSTRACT

A significant challenge to measuring human-automation trust is the amount of construct proliferation, models, and questionnaires with highly variable validation. However, all agree that trust is a crucial element of technological acceptance, continued usage, fluency, and teamwork. Herein, we synthesize a consensus model for trust in human-automation interaction by performing a meta-analysis of validated and reliable trust survey instruments. To accomplish this objective, this work identifies the most frequently cited and best-validated human-automation and human-robot trust questionnaires, as well as the most well-established factors, which form the dimensions and antecedents of such trust. To reduce both confusion and construct proliferation, we provide a detailed mapping of terminology between questionnaires. Furthermore, we perform a meta-analysis of the regression models that emerged from those experiments which used multi-factorial survey instruments. Based on this meta-analysis, we demonstrate a convergent experimentally validated model of human-automation trust. This convergent model establishes an integrated framework for future research. It identifies the current boundaries of trust measurement and where further investigation is necessary. We close by discussing choosing and designing an appropriate trust survey instrument. By comparing, mapping, and analyzing well-constructed trust survey instruments, a consensus structure of trust in human-automation interaction is identified. Doing so discloses a more complete basis for measuring trust emerges that is widely applicable. It integrates the academic idea of trust with the colloquial, common-sense one. Given the increasingly recognized importance of trust, especially in human-automation interaction, this work leaves us better positioned to understand and measure it.

研究动机与目标

• Identify the most cited and best-validated human-automation and human-robot trust questionnaires.
• Map and reconcile terminology across different trust instruments to reduce construct proliferation.
• Perform a meta-analysis of regression models from studies using multi-factorial trust instruments to derive a convergent model of trust in HAI.
• Assess instrument reliability and validity to guide the selection and design of future trust surveys.
• Outline a practical framework for choosing and designing trust survey instruments in HAI research.

提出的方法

• Systematically identify trust instruments from a broad literature base in HCI, HRI, and HAI using a PRISMA-like approach.
• Classify instruments into single-factor and multi-factor categories and assess their usage and impact.
• Evaluate instrument reliability using internal consistency metrics (Cronbach’s alpha, Guttman’s lambda-6, McDonald’s omega) and discuss their assumptions and limitations.
• Evaluate validity through content, construct, and statistical validity criteria including exploratory and confirmatory factor analyses.
• Compare and map the terminology and factors across instruments to derive a unified view of trust constructs in HAI.
• Provide criteria to classify instruments as Complete Reliability and Validity, Some Reliability and Validity, or No Reliability or Validity.]

实验结果

研究问题

• RQ1 Which trust survey instruments are most frequently cited and best validated in HAI/HRI/HAI literature?
• RQ2 What are the common factors or dimensions across multi-factor trust instruments, and how do they relate to each other?
• RQ3 Can a convergent, emergent model of human-automation trust be derived from existing reliable instruments?
• RQ4 What guidelines emerge for selecting or designing trust questionnaires for future HAI research?
• RQ5 Where do gaps and uncertainties remain in trust measurement for human-automation interaction?

主要发现

• The field shows a mix of single-, two-, and multi-item instruments, with a notable rise in multi-item surveys in the last decade.
• Twelve instruments account for nearly half of citations in reviews, indicating key instruments (e.g., McKnight and Gefen) dominate practice, though their applicability varies by sub-field.
• Reliability and validity reporting is inconsistent; about 24% of instruments show no validity/reliability analysis, and only 17% of single-factor instruments report both validity and reliability.
• A structured assessment framework classifies instruments by Complete Reliability/Validity, Some Reliability/Validity, or No Reliability/Validity, highlighting quality variability across measures.
• A convergent, experimentally validated model of trust is proposed, based on synthesis and meta-analysis of validated survey instruments, offering an integrated framework for future research.
• The study provides guidance on choosing and designing trust instruments and identifies current measurement boundaries and areas needing further validation