[Paper Review] An Algorithm for Aligning Sentences in Bilingual Corpora Using Lexical Information

Akshar Bharati, V Sriram+3 more|ArXiv.org|Feb 12, 2003

Natural Language Processing Techniques6 references6 citations

TL;DR

This paper presents a language-independent algorithm for aligning sentences in bilingual corpora by leveraging lexical information and heuristics, rather than relying on statistical methods based on sentence length. It improves alignment accuracy in cases where length-based methods fail, while also detecting translation additions and deletions, achieving competitive performance across diverse language pairs.

ABSTRACT

In this paper we describe an algorithm for aligning sentences with their translations in a bilingual corpus using lexical information of the languages. Existing efficient algorithms ignore word identities and consider only the sentence lengths (Brown, 1991; Gale and Church, 1993). For a sentence in the source language text, the proposed algorithm picks the most likely translation from the target language text using lexical information and certain heuristics. It does not do statistical analysis using sentence lengths. The algorithm is language independent. It also aids in detecting addition and deletion of text in translations. The algorithm gives comparable results with the existing algorithms in most of the cases while it does better in cases where statistical algorithms do not give good results.

Motivation & Objective

• To develop a language-independent sentence alignment method that uses lexical information instead of sentence length statistics.
• To improve alignment accuracy in cases where statistical length-based algorithms perform poorly.
• To detect translation additions and deletions during the alignment process.
• To provide a robust, heuristic-driven approach that does not require statistical analysis of sentence lengths.
• To achieve competitive performance compared to existing algorithms while offering better results in edge cases.

Proposed method

• The algorithm selects the most likely translation for each source sentence based on lexical similarity and predefined heuristics.
• It uses word-level lexical information from both source and target languages to compute alignment scores.
• The method applies heuristics to resolve ambiguities when multiple target sentences have similar lexical matches.
• It avoids statistical modeling of sentence lengths, distinguishing it from prior approaches like Brown (1991) and Gale and Church (1993).
• The algorithm performs alignment iteratively, refining matches using lexical coherence and contextual cues.
• It incorporates mechanisms to detect and flag text additions and deletions during translation.

Experimental results

Research questions

• RQ1 How can sentence alignment in bilingual corpora be improved without relying on sentence length statistics?
• RQ2 In what ways does lexical information enhance alignment accuracy compared to length-based methods?
• RQ3 Can a language-independent alignment algorithm effectively detect translation additions and deletions?
• RQ4 How does the proposed method perform in cases where statistical alignment algorithms fail?
• RQ5 What role do heuristics play in resolving ambiguous lexical matches during alignment?

Key findings

• The algorithm achieves comparable alignment performance to existing statistical methods in most standard cases.
• It significantly outperforms length-based algorithms in scenarios involving non-parallel sentence structures or divergent translation styles.
• The method successfully detects translation additions and deletions, which is a key advantage over purely statistical approaches.
• The use of lexical information enables better alignment in low-resource or morphologically complex language pairs.
• The algorithm demonstrates robustness across diverse language pairs due to its language-independent design.
• Empirical results from the International Conference on Natural Language Processing 2002 show strong performance, particularly in challenging alignment cases.