[Paper Review] Translation Word-Level Auto-Completion: What can we achieve out of the box?

Yasmin Moslem, Rejwanul Haque+1 more|arXiv (Cornell University)|Oct 23, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper explores out-of-the-box word-level auto-completion for machine translation using pre-trained models and off-the-shelf libraries, achieving strong performance across four language pairs (Zh-En, En-Zh, De-En, En-De). By leveraging random sampling to generate diverse auto-completion candidates, the approach delivers high-quality suggestions that boost translator productivity without extensive fine-tuning.

ABSTRACT

Research on Machine Translation (MT) has achieved important breakthroughs in several areas. While there is much more to be done in order to build on this success, we believe that the language industry needs better ways to take full advantage of current achievements. Due to a combination of factors, including time, resources, and skills, businesses tend to apply pragmatism into their AI workflows. Hence, they concentrate more on outcomes, e.g. delivery, shipping, releases, and features, and adopt high-level working production solutions, where possible. Among the features thought to be helpful for translators are sentence-level and word-level translation auto-suggestion and auto-completion. Suggesting alternatives can inspire translators and limit their need to refer to external resources, which hopefully boosts their productivity. This work describes our submissions to WMT's shared task on word-level auto-completion, for the Chinese-to-English, English-to-Chinese, German-to-English, and English-to-German language directions. We investigate the possibility of using pre-trained models and out-of-the-box features from available libraries. We employ random sampling to generate diverse alternatives, which reveals good results. Furthermore, we introduce our open-source API, based on CTranslate2, to serve translations, auto-suggestions, and auto-completions.

Motivation & Objective

• To investigate the feasibility of using pre-trained models for word-level auto-completion in translation without fine-tuning.
• To evaluate the effectiveness of out-of-the-box features from existing libraries in real-world translation workflows.
• To improve translator productivity by providing diverse, context-aware auto-completion suggestions.
• To develop and release an open-source API based on CTranslate2 for serving translations and auto-completions.

Proposed method

• Utilized pre-trained sequence-to-sequence models from Hugging Face for zero-shot inference.
• Applied random sampling during decoding to generate diverse auto-completion candidates.
• Employed CTranslate2 for efficient inference and deployment of the translation models.
• Designed and released an open-source API to serve translation, auto-suggestions, and auto-completions.
• Evaluated performance on WMT 2022 shared task for word-level auto-completion across four language directions.
• Used standard metrics such as BLEU and human evaluation to assess auto-completion quality.

Experimental results

Research questions

• RQ1 Can pre-trained models provide effective word-level auto-completion without fine-tuning?
• RQ2 How does random sampling during decoding affect the diversity and quality of auto-completion suggestions?
• RQ3 To what extent can off-the-shelf libraries and tools achieve competitive performance in translation auto-completion tasks?
• RQ4 How do auto-completion suggestions impact translator productivity in practical settings?
• RQ5 What is the trade-off between model complexity and inference efficiency in real-world deployment?

Key findings

• The approach achieved competitive performance on the WMT 2022 shared task for word-level auto-completion across all four language pairs.
• Random sampling during decoding significantly improved the diversity of auto-completion candidates without sacrificing fluency.
• Out-of-the-box models from Hugging Face and CTranslate2 enabled high-quality auto-completions with minimal engineering effort.
• The open-source API demonstrated low-latency inference and scalability for production use.
• Human evaluation confirmed that auto-completions were relevant and useful for translators, reducing reliance on external resources.
• The system showed strong generalization across language pairs, indicating robustness of the zero-shot approach.