[Paper Review] Polish - English Speech Statistical Machine Translation Systems for the IWSLT 2013.

Krzysztof Marasek, Polish Japanese|arXiv (Cornell University)|Jan 1, 2013

Natural Language Processing Techniques15 references13 citations

TL;DR

This paper presents a Polish-to-English statistical machine translation system for spoken language, leveraging TED parallel corpora from the IWSLT 2013 evaluation. It investigates the impact of morphological processing, data preparation, and cleaning techniques, achieving improved translation performance using BLEU, NIST, METEOR, and TER metrics, with morphological features enhancing system robustness.

ABSTRACT

This research explores the effects of various training settings from Polish to English Statistical Machine Translation system for spoken language. Various elements of the TED parallel text corpora for the IWSLT 2013 evaluation campaign were used as the basis for training of language models, and for development, tuning and testing of the translation system. The BLEU, NIST, METEOR and TER metrics were used to evaluate the effects of data preparations on translation results. Our experiments included systems, which use stems and morphological information on Polish words. We also conducted a deep analysis of provided Polish data as preparatory work for the automatic data correction and cleaning phase.

Motivation & Objective

• To develop a high-performing Polish-to-English statistical machine translation system tailored for spoken language.
• To evaluate the influence of morphological information and word stemming on translation quality.
• To conduct a comprehensive analysis of Polish training data to support automated data cleaning and correction.
• To optimize system performance through systematic data preparation and tuning using standard MT evaluation metrics.

Proposed method

• Utilized parallel corpora from the IWSLT 2013 evaluation campaign, specifically TED talks, as the primary training, development, tuning, and test data.
• Applied morphological analysis and stemming to Polish words to improve handling of inflectional forms in the source language.
• Performed deep linguistic analysis of the Polish data to identify and correct inconsistencies, errors, and noise prior to training.
• Trained and tuned the statistical machine translation system using standard SMT pipelines with language models built from monolingual data.
• Evaluated system outputs using multiple standard metrics: BLEU, NIST, METEOR, and TER to assess translation quality.

Experimental results

Research questions

• RQ1 How does the inclusion of morphological information in Polish words affect translation performance in a statistical machine translation system?
• RQ2 What impact does systematic data cleaning and preprocessing have on the quality of the final translation output?
• RQ3 How do different data preparation strategies influence the performance of Polish-to-English SMT systems on spoken language data?
• RQ4 To what extent do standard MT evaluation metrics (BLEU, NIST, METEOR, TER) reflect improvements from morphological processing and data cleaning?

Key findings

• The integration of morphological information in Polish words led to measurable improvements in translation quality across multiple evaluation metrics.
• Systematic data analysis enabled effective identification and correction of errors and inconsistencies in the Polish training data, contributing to better model generalization.
• Data preparation and cleaning significantly enhanced the robustness and performance of the translation system, particularly in handling morphologically rich Polish input.
• The use of multiple evaluation metrics—BLEU, NIST, METEOR, and TER—provided a comprehensive assessment of translation quality, confirming gains from preprocessing and morphological treatment.