[Paper Review] BERT in Negotiations: Early Prediction of Buyer-Seller Negotiation Outcomes

Kushal Chawla, Gale Lucas+2 more|arXiv (Cornell University)|Apr 6, 2020

Artificial Intelligence in Law25 references4 citations

TL;DR

This paper proposes a BERT-based model that predicts buyer-seller negotiation outcomes early in the conversation—using only 60% of the dialogue—achieving over 70% accuracy within 10% of the actual outcome. By leveraging pre-trained language representations, the method demonstrates that natural language contains sufficient strategic signals for early outcome prediction, enabling feedback for training autonomous negotiation agents.

ABSTRACT

The task of building automatic agents that can negotiate with humans in free-form natural language has gained recent interest in the literature. Although there have been initial attempts, combining linguistic understanding with strategy effectively still remains a challenge. Towards this end, we aim to understand the role of natural language in negotiations from a data-driven perspective by attempting to predict a negotiation's outcome, well before the negotiation is complete. Building on the recent advancements in pre-trained language encoders, our model is able to predict correctly within 10% for more than 70% of the cases, by looking at just 60% of the negotiation. These results suggest that rather than just being a way to realize a negotiation, natural language should be incorporated in the negotiation planning as well. Such a framework can be directly used to get feedback for training an automatically negotiating agent.

Motivation & Objective

• To investigate whether natural language in negotiations contains predictive signals for outcome determination before the negotiation concludes.
• To develop a data-driven approach that leverages pre-trained language models to predict negotiation outcomes early in the conversation.
• To explore the feasibility of using early outcome prediction as feedback for training autonomous negotiation agents.
• To assess the extent to which linguistic content alone can inform strategic predictions in human negotiations.

Proposed method

• Fine-tune a pre-trained BERT model on dialogue data from buyer-seller negotiations to predict final agreement outcomes.
• Use only the first 60% of each negotiation turn sequence as input to the model for early prediction.
• Train the model to predict the final agreement value within a 10% margin of error.
• Employ sequence classification head on the [CLS] token representation to output predicted outcome.
• Optimize the model using mean squared error loss between predicted and actual agreement values.
• Evaluate performance using accuracy within 10% of the true outcome across test negotiations.

Experimental results

Research questions

• RQ1 Can natural language in negotiations be used to predict final outcomes before the negotiation is complete?
• RQ2 How early can a model accurately predict negotiation outcomes using only a portion of the dialogue?
• RQ3 To what extent does linguistic content encode strategic information relevant to negotiation outcomes?
• RQ4 Can early prediction performance serve as a proxy for training autonomous negotiation agents?

Key findings

• The model achieves over 70% accuracy in predicting negotiation outcomes within a 10% margin of error when using only 60% of the dialogue.
• The model's predictive power demonstrates that linguistic content alone carries significant strategic information for outcome forecasting.
• Early prediction is feasible and reliable, suggesting that natural language should be integrated into negotiation planning, not just execution.
• The results indicate that BERT-based models can extract meaningful strategic signals from raw dialogue without explicit feature engineering.
• The framework provides a viable feedback mechanism for training autonomous negotiation agents by enabling early outcome estimation.