[Paper Review] Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews

Samantha Chan, Pat Pataranutaporn+4 more|arXiv (Cornell University)|Aug 8, 2024

Interpreting and Communication in Healthcare5 citations

TL;DR

Generative chatbot interactions significantly increase immediate false memories and confidence compared to survey or pre-scripted chatbots, with effects persisting after one week.

ABSTRACT

This study examines the impact of AI on human false memories -- recollections of events that did not occur or deviate from actual occurrences. It explores false memory induction through suggestive questioning in Human-AI interactions, simulating crime witness interviews. Four conditions were tested: control, survey-based, pre-scripted chatbot, and generative chatbot using a large language model (LLM). Participants (N=200) watched a crime video, then interacted with their assigned AI interviewer or survey, answering questions including five misleading ones. False memories were assessed immediately and after one week. Results show the generative chatbot condition significantly increased false memory formation, inducing over 3 times more immediate false memories than the control and 1.7 times more than the survey method. 36.4% of users' responses to the generative chatbot were misled through the interaction. After one week, the number of false memories induced by generative chatbots remained constant. However, confidence in these false memories remained higher than the control after one week. Moderating factors were explored: users who were less familiar with chatbots but more familiar with AI technology, and more interested in crime investigations, were more susceptible to false memories. These findings highlight the potential risks of using advanced AI in sensitive contexts, like police interviews, emphasizing the need for ethical considerations.

Motivation & Objective

• Investigate how AI-mediated questioning influences false memory formation in a witness-like scenario.
• Compare four interaction methods (control, survey-based, pre-scripted chatbot, generative chatbot) on false memories.
• Examine immediate and one-week persistence and confidence of induced false memories.
• Identify moderating individual factors affecting susceptibility to AI-induced false memories.

Proposed method

• Two-phase experiment with 200 participants randomly assigned to four conditions.
• Participants watch a crime video, then answer 25 questions including five misleading ones.
• Phase 1 uses control, survey-based, pre-scripted chatbot, or generative chatbot conditions.
• Phase 2, one week later, reassesses memories and confidence to measure persistence.
• False memories quantified by immediate and one-week recall; confidence measured for false and true memories.

Figure 1: Manipulation of Eyewitness Memory by AI: This figure illustrates the process of AI-induced false memories in three stages. It begins with a person witnessing a crime scene involving a knife, then shows an AI system introducing misinformation by asking about a non-existent gun, and conclude

Experimental results

Research questions

• RQ1 How do different AI interaction modes affect the formation of false memories in a witness-like interview setting?
• RQ2 Are generative chatbots more effective at inducing false memories than survey-based or pre-scripted chatbots?
• RQ3 What moderating user factors influence susceptibility to AI-induced false memories?
• RQ4 Do false memories induced by AI persist over one week and how does confidence change over time?

Key findings

• Generative chatbot induced more immediate false memories than survey-based and pre-scripted chatbot conditions (mean numbers: control 0.54, survey 1.08, pre-scripted 1.34, generative 1.82).
• 36.4% of responses in the generative chatbot condition were misled immediately.
• All interventions increased immediate false memories and confidence versus control, with generative chatbot yielding the highest confidence in false memories.
• After one week, the generative chatbot’s false memories remained roughly constant (immediate 36.4% vs. 36.8%), unlike control and survey where false memories increased.
• Moderators: lower chatbot familiarity and higher AI technology familiarity, plus greater interest in crime investigations, elevated susceptibility to AI-induced false memories.
• Generative chatbot-induced false memories showed higher long-term confidence than control and pre-scripted conditions.

Figure 2: Experimental Design for Studying AI-Induced False Memories: This figure outlines a two-phase study on AI-induced false memories. In Phase 1, participants watch a CCTV crime video, complete emotional assessments and filler tasks, and are randomly assigned to one of four conditions: control,