[Paper Review] Recursive Neural Network Based Preordering for English-to-Japanese Machine Translation

Yuki Kawara, Chenhui Chu+1 more|arXiv (Cornell University)|May 25, 2018

Natural Language Processing Techniques20 references3 citations

TL;DR

This paper proposes a recursive neural network (RvNN)-based preordering method for English-to-Japanese machine translation that automatically learns reordering decisions from raw syntactic trees without manual feature engineering. The method achieves translation quality comparable to state-of-the-art feature-engineered methods, with significant BLEU and RIBES improvements over baseline systems, particularly in phrase-based SMT.

ABSTRACT

The word order between source and target languages significantly influences the translation quality in machine translation. Preordering can effectively address this problem. Previous preordering methods require a manual feature design, making language dependent design costly. In this paper, we propose a preordering method with a recursive neural network that learns features from raw inputs. Experiments show that the proposed method achieves comparable gain in translation quality to the state-of-the-art method but without a manual feature design.

Motivation & Objective

• To address the challenge of language-dependent manual feature design in preordering for statistical and neural machine translation.
• To improve translation quality by learning optimal word order reordering in English-to-Japanese translation using syntactic structure.
• To develop a preordering method that leverages sub-tree representations through recursive neural networks, avoiding reliance on handcrafted features.
• To evaluate the proposed method in both phrase-based SMT and neural MT settings, demonstrating generalization across architectures.

Proposed method

• The method uses a recursive neural network (RvNN) to predict whether to reorder child nodes in a binary syntax tree based on sub-tree representations and syntactic categories.
• Sub-tree vectors are computed in a bottom-up manner using a non-linear transformation of concatenated child node embeddings, with ReLU activation.
• The RvNN outputs a probability distribution over 'Straight' and 'Inverted' labels via a softmax layer applied to a weighted sum of the hidden representation.
• Gold-standard labels are generated using Kendall’s τ correlation between source word alignments and target sentence order, with 'Inverted' assigned if reordering increases τ.
• Word embeddings, part-of-speech tags, and syntactic categories are used as input features to enrich the representation of leaf nodes.
• The model is trained end-to-end on syntax trees, with reordering decisions made at each internal node based on learned sub-tree semantics.

Experimental results

Research questions

• RQ1 Can a recursive neural network learn effective preordering decisions without manual feature engineering for English-to-Japanese translation?
• RQ2 How does the performance of an RvNN-based preordering method compare to state-of-the-art feature-based preordering methods in terms of BLEU and RIBES scores?
• RQ3 To what extent do syntactic categories and POS tags improve the quality of learned preordering decisions?
• RQ4 Why does preordering improve phrase-based SMT but degrade neural MT performance, and how can this be mitigated?
• RQ5 Can the RvNN model generalize across different MT architectures, including both PBSMT and NMT?

Key findings

• The RvNN-based preordering method achieved a BLEU score of 32.1 and RIBES score of 0.71 on PBSMT, matching the performance of the state-of-the-art feature-based method (BTG) with p-values of 0.068 and 0.226, respectively.
• The method outperformed plain PBSMT without preordering by a significant margin, with BLEU increasing from 29.8 to 32.1.
• Incorporating POS tags and syntactic categories improved BLEU scores, with optimal performance at λ=200; further increases in λ did not improve results.
• The ratio of high Kendall’s τ values increased substantially after preordering with RvNN, indicating better alignment with target word order.
• Despite improvements in PBSMT, NMT models with preordering showed lower BLEU scores than the baseline, suggesting a misalignment between preordering and NMT training objectives.
• Manual analysis revealed that parse errors led to preordering failures, but overall translation quality improved significantly when preordering succeeded.