[Paper Review] Incorporating Chinese Characters of Words for Lexical Sememe Prediction

Huiming Jin, Hao Zhu+5 more|arXiv (Cornell University)|Jun 17, 2018

Topic Modeling32 references4 citations

TL;DR

This paper proposes a character-enhanced sememe prediction framework (CSP) for Chinese lexical sememe prediction that integrates both internal character-level information and external context embeddings. By leveraging character-level semantics through novel models SPWCF and SPCSE, CSP achieves state-of-the-art performance on HowNet, significantly improving prediction accuracy—especially for low-frequency and out-of-vocabulary words—demonstrating robustness and effectiveness in real-world sememe annotation scenarios.

ABSTRACT

Sememes are minimum semantic units of concepts in human languages, such that each word sense is composed of one or multiple sememes. Words are usually manually annotated with their sememes by linguists, and form linguistic common-sense knowledge bases widely used in various NLP tasks. Recently, the lexical sememe prediction task has been introduced. It consists of automatically recommending sememes for words, which is expected to improve annotation efficiency and consistency. However, existing methods of lexical sememe prediction typically rely on the external context of words to represent the meaning, which usually fails to deal with low-frequency and out-of-vocabulary words. To address this issue for Chinese, we propose a novel framework to take advantage of both internal character information and external context information of words. We experiment on HowNet, a Chinese sememe knowledge base, and demonstrate that our framework outperforms state-of-the-art baselines by a large margin, and maintains a robust performance even for low-frequency words.

Motivation & Objective

• To address the limitation of existing lexical sememe prediction methods that rely solely on external context, which fail for low-frequency and out-of-vocabulary words.
• To explore the integration of internal character-level information in Chinese words as a complementary source of semantic knowledge for sememe prediction.
• To develop a unified framework that combines both internal (character-level) and external (contextual) information for improved sememe prediction performance.
• To evaluate the effectiveness and robustness of the proposed method on real-world data, particularly for rare and unseen words in HowNet.

Proposed method

• Proposes a novel framework, Character-enhanced Sememe Prediction (CSP), that jointly models external context embeddings and internal character embeddings for sememe prediction.
• Introduces Sememe Prediction with Word-to-Character Filtering (SPWCF), which uses character-level representations to refine word embeddings by filtering semantically relevant character features.
• Develops Sememe Prediction with Character and Sememe Embeddings (SPCSE), which models the semantic contribution of individual characters and their combinations to predict sememes.
• Employs a two-branch architecture: one branch learns from word-level context embeddings (based on SPWE and SPSE), and another from character-level representations, with late fusion via ensemble learning.
• Uses word and character embeddings trained on large-scale corpora, with character-level representations derived from subword units or morphological decomposition.
• Applies collaborative filtering and matrix factorization techniques to predict sememe relevance, enhanced by character-level priors to improve generalization.

Experimental results

Research questions

• RQ1 Can internal character-level information in Chinese words improve lexical sememe prediction performance, especially for low-frequency or out-of-vocabulary words?
• RQ2 How does the integration of character-level semantics with external context embeddings affect the accuracy and robustness of sememe prediction?
• RQ3 To what extent can character-level knowledge compensate for the lack of contextual signals in rare or unseen words?
• RQ4 Does the proposed framework maintain consistent performance across different word frequency distributions in HowNet?
• RQ5 Can character-level modeling capture semantically meaningful components (e.g., '铁' for 'iron') that are not inferable from context alone?

Key findings

• The CSP framework achieves state-of-the-art performance on the HowNet sememe prediction benchmark, outperforming existing baselines significantly.
• The model maintains robust performance across all word frequency bins, with stable MAP scores, indicating strong generalization to low-frequency and rare words.
• For the compound word '钟表匠' (clockmaker), the external model alone fails to recommend sememes like '时间' (time) and '用具' (tool), while CSP correctly infers '时间' and '工具' (tool) through character-level reasoning.
• For the transliterated word '奥斯卡' (Oscar), the internal model correctly identifies '专' (ProperName) and '地方' (place), reflecting that character-level knowledge captures morphological patterns in loanwords.
• The case study confirms that character-level information is effectively incorporated into the prediction process, enabling accurate sememe inference even when context embeddings are misleading.
• The ensemble model (CSP) achieves the best performance, demonstrating that combining internal and external signals yields superior results compared to either modality alone.