[论文解读] (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts

Minghao Wu, Xu, Jiahao+5 more|arXiv (Cornell University)|May 20, 2024

Translation Studies and Practices被引用 5

一句话总结

本文提出 TransAgents，是一个基于多代理 LLM 的系统，用于翻译超长文学文本，并提出 Monolingual Human Preference (MHP) 与 Bilingual LLM Preference (BLP) 评估方法，显示 TransAgents 在读者偏好方面可超越人类参考，尽管 d-BLEU 分数较低，同时显著降低成本。

ABSTRACT

Literary translation remains one of the most challenging frontiers in machine translation due to the complexity of capturing figurative language, cultural nuances, and unique stylistic elements. In this work, we introduce TransAgents, a novel multi-agent framework that simulates the roles and collaborative practices of a human translation company, including a CEO, Senior Editor, Junior Editor, Translator, Localization Specialist, and Proofreader. The translation process is divided into two stages: a preparation stage where the team is assembled and comprehensive translation guidelines are drafted, and an execution stage that involves sequential translation, localization, proofreading, and a final quality check. Furthermore, we propose two innovative evaluation strategies: Monolingual Human Preference (MHP), which evaluates translations based solely on target language quality and cultural appropriateness, and Bilingual LLM Preference (BLP), which leverages large language models like GPT-4} for direct text comparison. Although TransAgents achieves lower d-BLEU scores, due to the limited diversity of references, its translations are significantly better than those of other baselines and are preferred by both human evaluators and LLMs over traditional human references and GPT-4} translations. Our findings highlight the potential of multi-agent collaboration in enhancing translation quality, particularly for longer texts.

研究动机与目标

• Motivate and address the challenges of literary translation with MT and LLMs.
• Present a multi-agent framework that mimics a traditional publication workflow.
• Introduce novel evaluation strategies tailored to literature (MHP and BLP).
• Demonstrate cost benefits and analyze strengths/limitations across genres.

提出的方法

• Propose TransAgents, a virtual translation company with roles like CEO, Senior/Junior Editors, Translator, Localization Specialist, and Proofreader.
• Implement two collaboration strategies: Addition-by-Subtraction (Algorithm 1) and Trilateral Collaboration (Algorithm 2).
• Use a two-stage translation workflow: preparation (role assignment, guidelines) and execution (translation, localization, proofreading, final review).
• Use 30 agent profiles generated via gpt-4-turbo to simulate diverse agents and a ghost/self-reflection mechanism for role assignment.
• Evaluate translations with standard d-BLEU (document-level) and with preference-based metrics (MHP and BLP).
• Compare with baselines including Llama-MT, GPT-4 variants, Google Translate, DUT, HW-TSC.

实验结果

研究问题

• RQ1 How does a multi-agent, LLM-powered process compare to traditional MT and human references in translating ultra-long literary texts?
• RQ2 Can novel evaluation strategies (MHP and BLP) capture literary translation quality better than traditional metrics?
• RQ3 What are the trade-offs between BLEU-based quality and reader/LLM preferences in literary translation?
• RQ4 What are the cost implications and limitations of deploying a multi-agent translation system for literature?

主要发现

• TransAgents achieves the lowest d-BLEU among baselines but is consistently preferred by both human evaluators and LLM evaluators over human-written references and even GPT-4 translations for certain genres.
• The system excels in genres requiring domain-specific knowledge (historical, cultural nuance) and produces more diverse, vivid descriptions than references.
• There is a substantial cost advantage, with TransAgents offering up to 80x cost reduction versus professional human translation.
• LLM-based translation systems still face issues like content omission and quality fluctuations across long iterations.