[Paper Review] Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-stage Span Labeling

Duc-Vu Nguyen, Linh-Bao Vo+3 more|arXiv (Cornell University)|Dec 17, 2021

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes SpanSegTag, a two-stage span labeling model for joint Chinese word segmentation and part-of-speech tagging that uses biaffine operations on character boundary representations to model n-grams and their POS tags. The BERT-based model achieves state-of-the-art performance on CTB7 and CTB9, with significant improvements over prior methods like TwASP, while maintaining a smaller model size and avoiding reliance on external toolkits.

ABSTRACT

Chinese word segmentation and part-of-speech tagging are necessary tasks in terms of computational linguistics and application of natural language processing. Many re-searchers still debate the demand for Chinese word segmentation and part-of-speech tagging in the deep learning era. Nevertheless, resolving ambiguities and detecting unknown words are challenging problems in this field. Previous studies on joint Chinese word segmentation and part-of-speech tagging mainly follow the character-based tagging model focusing on modeling n-gram features. Unlike previous works, we propose a neural model named SpanSegTag for joint Chinese word segmentation and part-of-speech tagging following the span labeling in which the probability of each n-gram being the word and the part-of-speech tag is the main problem. We use the biaffine operation over the left and right boundary representations of consecutive characters to model the n-grams. Our experiments show that our BERT-based model SpanSegTag achieved competitive performances on the CTB5, CTB6, and UD, or significant improvements on CTB7 and CTB9 benchmark datasets compared with the current state-of-the-art method using BERT or ZEN encoders.

Motivation & Objective

• To address the challenges of ambiguity resolution and unknown word detection in Chinese word segmentation and POS tagging.
• To overcome limitations of character-based tagging models in handling overlapping ambiguous strings.
• To develop a joint model that explicitly models n-gram spans and their POS tags without relying on off-the-shelf linguistic toolkits.
• To improve performance on large-scale benchmarks like CTB7 and CTB9 while reducing model complexity.
• To explore span labeling as an alternative to traditional character-based tagging in sequence labeling tasks.

Proposed method

• The model uses a two-stage span labeling framework: first predicting word spans, then assigning POS tags to those spans.
• It employs BERT or BiLSTM encoders to generate contextual representations for input characters.
• Left and right boundary representations of consecutive characters are combined via a biaffine operation to predict span likelihoods for words and POS tags.
• A post-processing heuristic resolves overlapping spans to ensure non-overlapping word boundaries.
• The model jointly optimizes for word segmentation and POS tagging using a unified loss function over span predictions.
• The architecture avoids memory networks and external linguistic features, relying solely on learned span representations.

Experimental results

Research questions

• RQ1 Can a span labeling approach outperform character-based tagging in joint Chinese word segmentation and POS tagging?
• RQ2 How does modeling n-gram spans with biaffine operations improve ambiguity resolution compared to traditional tagging paradigms?
• RQ3 To what extent can a BERT-based model achieve state-of-the-art performance without using off-the-shelf linguistic toolkits?
• RQ4 How does model size and inference speed compare to existing SOTA methods like TwASP?
• RQ5 Does the span-based approach better capture in-vocabulary ambiguity resolution and OOV word detection?

Key findings

• The BERT-based SpanSegTag achieved significant improvements on CTB7 (F1: 80.19) and CTB9 (F1: 78.52), outperforming the prior SOTA method TwASP using BERT or ZEN.
• On CTB5, CTB6, and UD datasets, SpanSegTag achieved competitive F1 scores, with 95.33 F1 on CTB7 and 95.80 on CTB9.
• The model achieved high recall for in-vocabulary words and their POS tags (R_POS-iV), indicating strong ambiguity resolution capability.
• SpanSegTag demonstrated a smaller model size (433–441 MB) compared to TwASP (433–1170 MB), showing reduced parameter dependency on dataset scale.
• Inference speed was slightly slower than TwASP (264 vs. 239 sentences/second), but the model avoids time-consuming external toolkit calls.
• The model showed slightly better CAS (combination ambiguity string) detection on CTB6, CTB7, and CTB9, suggesting improved syntactic and semantic understanding.