[Paper Review] I Need Your Advice... Human Perceptions of Robot Moral Advising Behaviors

Nichole D. Starr, Bertram F. Malle+1 more|arXiv (Cornell University)|Apr 14, 2021

Psychology of Moral and Emotional Judgment4 citations

TL;DR

This study investigates how human perceptions of robot moral advising vary based on whether robots recommend actions that prioritize the common good over individual lives. Findings show humans and robots are perceived more positively when advising for the common good, even when the advice conflicts with what the advisee might personally choose, highlighting a key design principle for ethically aligned autonomous agents in human-robot interaction.

ABSTRACT

Due to their unique persuasive power, language-capable robots must be able to both act in line with human moral norms and clearly and appropriately communicate those norms. These requirements are complicated by the possibility that humans may ascribe blame differently to humans and robots. In this work, we explore how robots should communicate in moral advising scenarios, in which the norms they are expected to follow (in a moral dilemma scenario) may be different from those their advisees are expected to follow. Our results suggest that, in fact, both humans and robots are judged more positively when they provide the advice that favors the common good over an individual's life. These results raise critical new questions regarding people's moral responses to robots and the design of autonomous moral agents.

Motivation & Objective

• To examine how people perceive robots giving moral advice in dilemmas where the robot's moral norms may differ from those of the human advisee.
• To investigate whether humans judge robot advisors more favorably when they recommend actions aligned with the common good, even if those actions conflict with human intuitive preferences.
• To explore the implications of differential blame attribution between humans and robots in moral decision-making scenarios.
• To assess the role of moral communication in shaping perceptions of trustworthiness and likability in autonomous agents.
• To inform the design of ethically competent robots that can communicate moral norms effectively in human-robot interaction contexts.

Proposed method

• Conducted a vignette-based experimental study using a moral dilemma inspired by the Trolley Problem, where participants evaluated robot and human advisors.
• Manipulated the advisor's recommendation (action vs. inaction) and the advisor's identity (robot vs. human) to assess differential perceptions.
• Measured perceptions of trustworthiness, likability, and blame using standardized rating scales following each scenario.
• Analyzed responses across different advisor types and advice types to identify patterns in moral evaluation and social attribution.
• Used controlled, hypothetical scenarios to isolate the effects of advisor identity and advice content on moral judgments.
• Collected and analyzed qualitative data to explore rationales behind participants’ evaluations of advice and advisor behavior.

Experimental results

Research questions

• RQ1 How do people perceive robot advisors who recommend actions that favor the common good over individual lives?
• RQ2 How do perceptions of robot moral advisors differ from those of human advisors in moral dilemma scenarios?
• RQ3 Does the consistency of moral norms between advisor and advisee affect perceptions of trustworthiness and likability?
• RQ4 How does blame attribution differ between human and robot advisors when they recommend inaction versus action in a moral dilemma?
• RQ5 What role does moral communication play in shaping human perceptions of autonomous agents in ethical decision-making contexts?

Key findings

• Robots were perceived more positively when advising to sacrifice one life to save five, even when the advisee was a human who might otherwise be blamed for such an action.
• Both human and robot advisors were judged more favorably when recommending actions that served the common good, indicating a shared moral evaluation standard.
• Participants attributed more blame to robots for inaction than for action in moral dilemmas, a pattern not consistently observed for human advisors.
• Humans were also seen as more likable and trustworthy when advising for the common good, even when they themselves would avoid such actions.
• The results suggest that moral consistency between advisor and recommended action enhances perceived trustworthiness, regardless of the advisor’s identity.
• Differential blame patterns between humans and robots indicate that mechanomorphic robots may be held to a different moral standard, especially regarding inaction.