[Paper Review] Deep Learning for Classical Japanese Literature

Tarin Clanuwat, Mikel Bober-Irizar+4 more|arXiv (Cornell University)|Dec 3, 2018

Topic ModelingComputer Science15 references485 citations

TL;DR

The paper introduces Kuzushiji datasets (Kuzushiji-MNIST, Kuzushiji-49, Kuzushiji-Kanji) to benchmark ML on classical Japanese script and demonstrates baseline classification and domain-transfer experiments to modern Kanji.

ABSTRACT

Much of machine learning research focuses on producing models which perform well on benchmark tasks, in turn improving our understanding of the challenges associated with those tasks. From the perspective of ML researchers, the content of the task itself is largely irrelevant, and thus there have increasingly been calls for benchmark tasks to more heavily focus on problems which are of social or cultural relevance. In this work, we introduce Kuzushiji-MNIST, a dataset which focuses on Kuzushiji (cursive Japanese), as well as two larger, more challenging datasets, Kuzushiji-49 and Kuzushiji-Kanji. Through these datasets, we wish to engage the machine learning community into the world of classical Japanese literature. Dataset available at https://github.com/rois-codh/kmnist

Motivation & Objective

• Motivate ML research on culturally and historically relevant tasks by focusing on Kuzushiji (cursive Japanese) reading challenges.
• Provide easy-to-use preprocessed datasets for machine learning research in classical Japanese literature.
• Establish baseline classification results and explore domain transfer from Kuzushiji-Kanji to Modern Kanji.
• Encourage cross-disciplinary collaboration between ML and Japanese literature communities.

Proposed method

• Release three preprocessed datasets (Kuzushiji-MNIST, Kuzushiji-49, Kuzushiji-Kanji) as drop-in replacements for familiar benchmarks.
• Evaluate baseline classification using 4-nearest neighbors, a small CNN, and ResNet-18 variants with regularizers.
• Conduct a domain-transfer experiment to generate Modern Kanji strokes from Kuzushiji-Kanji using two VAEs, a MDN, and a Sketch-RNN decoder.
• Compare performance to MNIST and Fashion-MNIST-like baselines to assess difficulty and distributional challenges.

Experimental results

Research questions

• RQ1 Can Kuzushiji datasets provide a meaningful, challenging benchmark for image classification in historical scripts?
• RQ2 How do modern architectures perform on Kuzushiji-MNIST and Kuzushiji-49 relative to MNIST-based benchmarks?
• RQ3 Is it feasible to transfer from Kuzushiji-Kanji to Modern Kanji in both pixel and stroke-based representations?
• RQ4 What modeling strategies (e.g., Mixup, manifold mixup) improve accuracy on imbalanced Kuzushiji datasets?]
• RQ5 How does multi-modal writing (Hentaigana) affect class formation and recognition?

Key findings

• Baseline results show high accuracy on MNIST and strong but lower performance on Kuzushiji-MNIST and Kuzushiji-49 across models, with PreActResNet-18 reaching 99.56% (MNIST), 97.82% (Kuzushiji-MNIST), and 96.64% (Kuzushiji-49).
• More advanced variants (Input Mixup, Manifold Mixup) improve Kuzushiji accuracy to 98.41% (Kuzushiji-MNIST) and 97.33% (Kuzushiji-49).
• Kuzushiji-Kanji contains 3832 classes with severe imbalance, ranging from 1 to over 1,000 samples per class, highlighting its challenging nature for recognition tasks.
• Domain-transfer experiments generate Modern Kanji from Kuzushiji-Kanji using a two-stage VAE approach and Sketch-RNN, demonstrating potential for cross-domain transcription support.
• The datasets capture real-world class imbalances and multi-modal character representations (Hentaigana), emphasizing the need for robust models beyond standard MNIST-like benchmarks.
• The authors anticipate expanding the dataset to over a million character images and fostering competitions to spur further research.