[Paper Review] Lemmatization of Historical Old Literary Finnish Texts in Modern Orthography

Mika Hämäläinen, Niko Partanen+1 more|arXiv (Cornell University)|Jul 7, 2021

Natural Language Processing Techniques5 citations

TL;DR

This paper presents a neural network-based approach for simultaneously normalizing and lemmatizing Old Literary Finnish texts into modern Finnish orthography, achieving 96.3% accuracy on Mikael Agricola's texts and 87.7% on out-of-domain contemporary texts. The method is implemented in a publicly available Python library and released on Zenodo for open research use.

ABSTRACT

Texts written in Old Literary Finnish represent the first literary work ever written in Finnish starting from the 16th century. There have been several projects in Finland that have digitized old publications and made them available for research use. However, using modern NLP methods in such data poses great challenges. In this paper we propose an approach for simultaneously normalizing and lemmatizing Old Literary Finnish into modern spelling. Our best model reaches to 96.3\\% accuracy in texts written by Agricola and 87.7\\% accuracy in other contemporary out-of-domain text. Our method has been made freely available on Zenodo and Github.

Motivation & Objective

• To address the challenge of applying modern NLP tools to Old Literary Finnish, which uses archaic spelling and morphology not supported by current Finnish NLP systems.
• To develop a unified model that performs both normalization (spelling standardization) and lemmatization (word form reduction) on historical Finnish texts.
• To evaluate model performance on both in-domain texts (Agricola) and out-of-domain contemporary historical texts to assess generalization.
• To release a user-friendly, open-source Python library and model weights to support reproducible and accessible research on historical Finnish texts.
• To enable large-scale NLP analysis of Finland's early printed literary corpus, which remains underutilized due to linguistic and orthographic barriers.

Proposed method

• A sequence-to-sequence neural network model is trained to map Old Literary Finnish tokens into their modern Finnish orthographic equivalents.
• The model is jointly trained on normalization and lemmatization, with target outputs derived from morpho-syntactic annotations in the Agricola corpus.
• Training data includes texts from Mikael Agricola and other contemporary sources, with orthographic variants normalized to modern Finnish spelling.
• The model uses attention mechanisms and bidirectional LSTM layers to capture long-range dependencies and historical spelling patterns.
• A custom preprocessing pipeline aligns historical tokens with modern lemma forms using the Dictionary of Old Literary Finnish and existing annotations.
• The final system is packaged into a lightweight Python library (murre) for easy integration into NLP workflows.

Experimental results

Research questions

• RQ1 Can a deep learning model effectively perform joint normalization and lemmatization on Old Literary Finnish texts?
• RQ2 How well does the model generalize to out-of-domain historical texts not used in training?
• RQ3 What level of accuracy can be achieved in lemmatizing Old Literary Finnish when mapped to modern Finnish orthography?
• RQ4 To what extent can existing modern Finnish NLP tools be enhanced through preprocessing with this normalization model?
• RQ5 Can a publicly released, user-friendly tool significantly lower the barrier to entry for NLP research on historical Finnish corpora?

Key findings

• The proposed model achieves 96.3% accuracy in lemmatizing texts written by Mikael Agricola, the primary source of Old Literary Finnish.
• On out-of-domain contemporary historical texts, the model reaches 87.7% accuracy, demonstrating strong generalization capability.
• The model significantly improves the usability of historical Finnish texts for modern NLP pipelines by resolving spelling and morphological variation.
• The integration of normalization and lemmatization into a single neural sequence-to-sequence framework yields superior performance compared to sequential processing.
• The release of the model and Python library on Zenodo and GitHub enables immediate, reproducible use by researchers and developers.
• The study establishes a foundational NLP pipeline for Old Literary Finnish, opening new avenues for computational historical linguistics and digital humanities research.