[Paper Review] LLMediator: GPT-4 Assisted Online Dispute Resolution

Hannes Westermann, Jaromír Šavelka+1 more|arXiv (Cornell University)|Jul 27, 2023

Dispute Resolution and Class Actions13 citations

TL;DR

LLMediator is an experimental platform that uses GPT-4 to reformulate inflammatory messages, draft mediator interventions, and potentially autonomously engage in online dispute resolution to facilitate amicable settlements.

ABSTRACT

In this article, we introduce LLMediator, an experimental platform designed to enhance online dispute resolution (ODR) by utilizing capabilities of state-of-the-art large language models (LLMs) such as GPT-4. In the context of high-volume, low-intensity legal disputes, alternative dispute resolution methods such as negotiation and mediation offer accessible and cooperative solutions for laypeople. These approaches can be carried out online on ODR platforms. LLMediator aims to improve the efficacy of such processes by leveraging GPT-4 to reformulate user messages, draft mediator responses, and potentially autonomously engage in the discussions. We present and discuss several features of LLMediator and conduct initial qualitative evaluations, demonstrating the potential for LLMs to support ODR and facilitate amicable settlements. The initial proof of concept is promising and opens up avenues for further research in AI-assisted negotiation and mediation.

Motivation & Objective

• Address high-volume, low-intensity disputes by enhancing ODR with AI-assisted negotiation and mediation.
• Investigate how GPT-4 can reformulate user messages to be more neutral and settlement-friendly.
• Provide draft mediator interventions and evaluate autonomous intervention as an experimental feature.

Proposed method

• Develop and deploy an experimental web-based ODR platform integrated with GPT-4 via API.
• Implement F1: reformulate inflammatory messages to maintain content while neutralizing tone.
• Implement F2: generate draft mediator interventions based on conversation context.
• Implement F3: optional autonomous intervention by the system in negotiations.
• Describe prompting strategies and system workflows for each feature.

Experimental results

Research questions

• RQ1 Can GPT-4 effectively reformulate inflammatory user messages without altering core content?
• RQ2 Can GPT-4 generate useful, context-aware mediator interventions that assist human mediators?
• RQ3 Is autonomous system intervention feasible and what are the associated risks in ODR?
• RQ4 What are the practical considerations and risks of integrating LLMs into ODR workflows?

Key findings

• GPT-4 demonstrates the ability to reformulate messages to a more neutral tone while preserving content.
• GPT-4 can draft mediator interventions that adapt to recent messages and mediator instructions.
• The system can optionally autonomously intervene in negotiations, raising discussion of risks and the need for careful evaluation.
• The authors discuss benefits as augmented intelligence and emphasize human-in-the-loop safeguards to mitigate hallucination and misguidance.