[Paper Review] An open dataset for the evolution of oracle bone characters: EVOBC

Haisu Guan, Jinpeng Wan+7 more|arXiv (Cornell University)|Jan 23, 2024

Image Processing and 3D Reconstruction4 citations

TL;DR

This paper introduces EVOBC, an open, large-scale dataset of 229,170 images spanning six historical stages of Chinese character evolution, from Oracle Bone Script (15th century B.C.) to Clerical Script (220 A.D.). The dataset enables AI-driven research on character evolution, demonstrating strong performance in image classification (86.66% top-1 accuracy with Swin Transformer v2) and effective baseline results in oracle bone script deciphering simulation using both classification and diffusion-based image generation models.

ABSTRACT

The earliest extant Chinese characters originate from oracle bone inscriptions, which are closely related to other East Asian languages. These inscriptions hold immense value for anthropology and archaeology. However, deciphering oracle bone script remains a formidable challenge, with only approximately 1,600 of the over 4,500 extant characters elucidated to date. Further scholarly investigation is required to comprehensively understand this ancient writing system. Artificial Intelligence technology is a promising avenue for deciphering oracle bone characters, particularly concerning their evolution. However, one of the challenges is the lack of datasets mapping the evolution of these characters over time. In this study, we systematically collected ancient characters from authoritative texts and websites spanning six historical stages: Oracle Bone Characters - OBC (15th century B.C.), Bronze Inscriptions - BI (13th to 221 B.C.), Seal Script - SS (11th to 8th centuries B.C.), Spring and Autumn period Characters - SAC (770 to 476 B.C.), Warring States period Characters - WSC (475 B.C. to 221 B.C.), and Clerical Script - CS (221 B.C. to 220 A.D.). Subsequently, we constructed an extensive dataset, namely EVolution Oracle Bone Characters (EVOBC), consisting of 229,170 images representing 13,714 distinct character categories. We conducted validation and simulated deciphering on the constructed dataset, and the results demonstrate its high efficacy in aiding the study of oracle bone script. This openly accessible dataset aims to digitalize ancient Chinese scripts across multiple eras, facilitating the decipherment of oracle bone script by examining the evolution of glyph forms.

Motivation & Objective

• To address the lack of comprehensive, curated datasets for studying the evolution of oracle bone characters (OBC) across historical periods.
• To facilitate AI-assisted research in deciphering undeciphered oracle bone characters, which remain largely unknown despite 4,500 extant characters.
• To construct a high-quality, multi-period dataset that captures morphological transformations from OBC to modern forms.
• To validate the dataset’s utility through classification and novel deciphering simulation tasks using deep learning models.
• To release all data, code, and training scripts to ensure reproducibility and support future research in computational epigraphy.

Proposed method

• Systematically collected 229,170 character images from authoritative books and online repositories across six historical periods: OBC, BI, SS, SAC, WSC, and CS.
• Automated data pipeline for image extraction, denoising, alignment, and automatic labeling from scanned texts and digital archives.
• Constructed a unified dataset with 13,714 distinct character categories, preserving evolutionary trajectories across time.
• Evaluated the dataset using state-of-the-art models: ResNet-101 and Swin Transformer v2 for image classification.
• Proposed a novel 'Oracle Bone Character Deciphering Simulation' task using two baselines: image classification and conditional diffusion-based image generation.
• Trained a conditional diffusion model to generate evolved character forms from OBC inputs, comparing outputs to ground truth labels.

Experimental results

Research questions

• RQ1 Can a large-scale, multi-period dataset of ancient Chinese characters effectively support AI-based research on character evolution?
• RQ2 To what extent can deep learning models classify oracle bone characters and their evolved forms with high accuracy?
• RQ3 Can image generation models be used to simulate the decipherment of undeciphered oracle bone characters by predicting their evolved forms?
• RQ4 How effective is the EVOBC dataset in enabling the discovery of morphological patterns across historical script stages?
• RQ5 Can the dataset serve as a reliable benchmark for future AI-assisted decipherment of ancient scripts?

Key findings

• The EVOBC dataset achieved a top-1 accuracy of 85.56% with ResNet-101 and 86.66% with Swin Transformer v2 on the validation set, confirming high annotation quality and dataset reliability.
• In the oracle bone character deciphering simulation, the image classification baseline achieved 16.7% top-1 and 55.8% top-20 accuracy, demonstrating the feasibility of AI-assisted decipherment.
• The conditional diffusion model successfully generated plausible evolved character forms, with qualitative results showing close alignment to ground truth labels in multiple test cases.
• The dataset contains 229,170 images across 13,714 distinct character categories, with 90,882 images from books and 138,288 from online sources, ensuring broad coverage and diversity.
• The dataset supports the study of evolutionary patterns from OBC to modern Chinese characters, with clear morphological trajectories observable across the six historical stages.
• All data processing scripts, training code, and model weights are publicly released on GitHub, enabling full reproducibility and community extension.