[Paper Review] Explainable AI and Adoption of Financial Algorithmic Advisors: an Experimental Study

Daniel Ben David, Yehezkel S. Resheff+1 more|arXiv (Cornell University)|Jan 5, 2021

Explainable Artificial Intelligence (XAI)26 references4 citations

TL;DR

This experimental study investigates how different types of explainable AI (XAI) explanations—local and global—affect user adoption, trust, and willingness to pay for financial algorithmic advisors. Using a web-based lemonade stand game with real monetary stakes, the authors find that accuracy-based explanations boost initial adoption, while feature-based or accuracy-based explanations mitigate trust loss after model failure, and that autopilot features significantly increase adoption rates.

ABSTRACT

We study whether receiving advice from either a human or algorithmic advisor, accompanied by five types of Local and Global explanation labelings, has an effect on the readiness to adopt, willingness to pay, and trust in a financial AI consultant. We compare the differences over time and in various key situations using a unique experimental framework where participants play a web-based game with real monetary consequences. We observed that accuracy-based explanations of the model in initial phases leads to higher adoption rates. When the performance of the model is immaculate, there is less importance associated with the kind of explanation for adoption. Using more elaborate feature-based or accuracy-based explanations helps substantially in reducing the adoption drop upon model failure. Furthermore, using an autopilot increases adoption significantly. Participants assigned to the AI-labeled advice with explanations were willing to pay more for the advice than the AI-labeled advice with a No-explanation alternative. These results add to the literature on the importance of XAI for algorithmic adoption and trust.

Motivation & Objective

• To investigate how various types of explainable AI (XAI) explanations influence user adoption, trust, and willingness to pay for financial algorithmic advisors.
• To assess the impact of explanation type on user behavior over time, particularly in response to model performance changes (success vs. failure).
• To compare human vs. AI advice in terms of adoption rates and user perception, especially under conditions of model failure.
• To evaluate the role of autopilot functionality and explanation satisfaction in shaping user decisions and trust.
• To develop and validate a dynamic experimental framework using real financial incentives to study evolving user behavior toward AI systems.

Proposed method

• Conducted a controlled web-based experiment using a lemonade stand game with real monetary consequences to simulate financial decision-making.
• Randomly assigned participants to receive advice from either a human or algorithmic advisor, each paired with one of five explanation types: global (model performance, feature importance, prototype examples), local (accuracy-based, feature-based), or no explanation.
• Measured adoption rates, trust levels, and willingness to pay (WTP) at multiple time points, including before and after model failure events.
• Collected post-game questionnaires to assess user perceptions of explanation quality, trust antecedents, and satisfaction.
• Used statistical analysis to compare adoption and WTP across conditions, with a focus on time-dependent effects and failure recovery.
• Employed a mixed-methods approach combining behavioral metrics with self-reported trust and satisfaction to validate findings.

Experimental results

Research questions

• RQ1 Does the type of explanation (local vs. global, accuracy-based vs. feature-based) significantly affect initial adoption of AI financial advice?
• RQ2 How does model failure impact user adoption and trust, and can specific explanation types mitigate this negative effect?
• RQ3 Does the presence of an autopilot feature increase user adoption of AI advice compared to manual decision-making?
• RQ4 How does the inclusion of explanations affect users' willingness to pay for AI-generated financial advice?
• RQ5 What is the relationship between explanation satisfaction, trust antecedents, and long-term adoption behavior after failure events?

Key findings

• Accuracy-based explanations in initial phases led to significantly higher adoption rates compared to no explanations or other explanation types.
• When the model performed perfectly, the type of explanation had minimal impact on adoption, indicating that high performance reduces the need for detailed explanations.
• After model failure, feature-based and accuracy-based explanations significantly reduced the drop in adoption and helped preserve trust, outperforming simpler or no explanations.
• Participants given AI advice with explanations were willing to pay more than those receiving AI advice without explanations, indicating that explanations increase perceived value.
• The autopilot feature significantly increased adoption rates, suggesting that users prefer delegating decisions when possible, especially under uncertainty.
• There was a strong positive correlation between explanation satisfaction and initial adoption, and between trust antecedents and resilience to failure-induced adoption drops.