[Paper Review] A New Aligned Simple German Corpus

Vanessa Toborek, Moritz Busch+3 more|arXiv (Cornell University)|Sep 2, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces a new sentence-aligned monolingual corpus of Simple German (Leichte Sprache) and standard German, created by automatically aligning texts from seven German websites offering Leichte Sprache and one for Einfache Sprache. Using multiple alignment methods—including BERT-based embeddings and n-gram similarity—the authors achieve superior F1-scores (up to 0.48) on a manually annotated subset, surpassing prior work. The dataset and code are publicly released under CC BY-SA and MIT licenses.

ABSTRACT

"Leichte Sprache", the German counterpart to Simple English, is a regulated language aiming to facilitate complex written language that would otherwise stay inaccessible to different groups of people. We present a new sentence-aligned monolingual corpus for Simple German -- German. It contains multiple document-aligned sources which we have aligned using automatic sentence-alignment methods. We evaluate our alignments based on a manually labelled subset of aligned documents. The quality of our sentence alignments, as measured by F1-score, surpasses previous work. We publish the dataset under CC BY-SA and the accompanying code under MIT license.

Motivation & Objective

• To address the lack of high-quality, sentence-aligned parallel corpora for German text simplification.
• To create a publicly available, reproducible dataset of Simple German (Leichte Sprache) and standard German texts for NLP research.
• To evaluate and compare multiple automatic sentence-alignment techniques on this new corpus.
• To establish a benchmark for future work in German text simplification using neural and lexical methods.
• To support inclusive language research by providing a resource for training and evaluating simplification models.

Proposed method

• Collected articles from seven websites offering Leichte Sprache and one for Einfache Sprache, resulting in multiple document-aligned sources.
• Applied automatic sentence-alignment techniques including bag-of-words, 4-gram, cosine similarity, maximum matching, CWASA, bipartite matching, and SBERT embeddings.
• Used MST-LIS matching algorithm with a similarity threshold of 1.5 to optimize alignment quality.
• Conducted two manual evaluations: one on ground-truth-aligned sentence pairs for precision, recall, and F1; another on algorithm-generated matches for classification accuracy.
• Developed a GUI for manual annotation of alignments and matches to ensure reliable evaluation.
• Released the final dataset upon request and the full codebase on GitHub under MIT license.

Experimental results

Research questions

• RQ1 Which automatic sentence-alignment method achieves the highest F1-score on a manually annotated subset of Simple German and standard German text pairs?
• RQ2 How does alignment quality vary across different source websites, and what factors influence this variation?
• RQ3 Can neural embedding-based methods like SBERT outperform traditional lexical methods in aligning simplified and standard German texts?
• RQ4 How accurate are the algorithm-generated matches when evaluated by human annotators on a per-match classification task?
• RQ5 To what extent can the proposed alignment pipeline be extended to crawl and align additional websites?

Key findings

• The SBERT-based alignment method achieved the highest F1-score of 0.48 on the ground-truth evaluation, significantly outperforming previous work.
• The maximum matching strategy with a threshold of 1.5 achieved the highest alignment classification accuracy of 0.56 in the second evaluation, indicating strong performance on algorithm-generated matches.
• Performance varied widely across sources, with the highest F1-score (0.71) on the beb website and the lowest (0.13) on the taz website, indicating source-dependent quality.
• The bipartite matching strategy achieved an alignment classification accuracy of 0.54, showing strong generalization across diverse text types.
• The average F1-score across all websites was 0.32, demonstrating that the new corpus provides a robust benchmark for future simplification models.
• The dataset and code are publicly available, enabling reproducibility and extension to new sources through automated crawling and parsing.