[Paper Review] Multi Task Deep Morphological Analyzer: Context Aware Joint Morphological Tagging and Lemma Prediction

Saurav Jha, Akhilesh Sudhakar+1 more|arXiv (Cornell University)|Nov 21, 2018

Natural Language Processing Techniques64 references4 citations

TL;DR

This paper proposes MT-DMA, a character-level deep neural network that jointly performs morphological tagging and lemma prediction for Hindi and Urdu using multi-task learning. By optimizing six morphological tags (POS, Gender, Number, Person, Case, TAM) and lemma prediction simultaneously with shared initial parameters and context-aware features, the model achieves state-of-the-art performance across all seven tasks, outperforming prior methods through multi-objective optimization of character-level representations.

ABSTRACT

The ambiguities introduced by the recombination of morphemes constructing several possible inflections for a word makes the prediction of syntactic traits in Morphologically Rich Languages (MRLs) a notoriously complicated task. We propose the Multi Task Deep Morphological analyzer (MT-DMA), a character-level neural morphological analyzer based on multitask learning of word-level tag markers for Hindi and Urdu. MT-DMA predicts a set of six morphological tags for words of Indo-Aryan languages: Parts-of-speech (POS), Gender (G), Number (N), Person (P), Case (C), Tense-Aspect-Modality (TAM) marker as well as the Lemma (L) by jointly learning all these in one trainable framework. We show the effectiveness of training of such deep neural networks by the simultaneous optimization of multiple loss functions and sharing of initial parameters for context-aware morphological analysis. Exploiting character-level features in phonological space optimized for each tag using multi-objective genetic algorithm, our model establishes a new state-of-the-art accuracy score upon all seven of the tasks for both the languages. MT-DMA is publicly accessible: code, models and data are available at https://github.com/Saurav0074/morph_analyzer.

Motivation & Objective

• To address the challenge of morphological ambiguity in Morphologically Rich Languages (MRLs) like Hindi and Urdu.
• To improve the accuracy of morphological tagging and lemma prediction by jointly learning multiple tasks.
• To leverage character-level representations optimized via a multi-objective genetic algorithm for better context awareness.
• To establish a new state-of-the-art performance benchmark across seven morphological prediction tasks for Indo-Aryan languages.

Proposed method

• The model employs a deep neural network architecture trained on character-level input sequences to capture morphological patterns.
• It uses multi-task learning to jointly optimize six morphological tag predictions (POS, Gender, Number, Person, Case, TAM) and lemma prediction.
• Shared initial layers extract low-level features, with task-specific branches for each output head.
• Character-level features are optimized in phonological space using a multi-objective genetic algorithm to enhance context awareness.
• Multiple loss functions are minimized simultaneously during training to improve generalization and representation learning.
• The framework enables parameter sharing across tasks, improving data efficiency and performance on low-resource morphological analysis.

Experimental results

Research questions

• RQ1 Can joint learning of multiple morphological tagging and lemma prediction tasks improve performance in morphologically rich languages?
• RQ2 How effective is multi-task learning with shared representations for character-level morphological analysis in Hindi and Urdu?
• RQ3 Can context-aware character-level features, optimized via a multi-objective genetic algorithm, enhance morphological tagging accuracy?
• RQ4 Does simultaneous optimization of multiple loss functions lead to better generalization and state-of-the-art results?
• RQ5 To what extent does parameter sharing across tasks improve performance on low-resource morphological analysis tasks?

Key findings

• MT-DMA achieves a new state-of-the-art accuracy score on all seven morphological prediction tasks for both Hindi and Urdu.
• The model outperforms existing methods in predicting Parts-of-Speech, Gender, Number, Person, Case, TAM markers, and lemmas.
• Multi-task training with shared initial parameters significantly improves performance compared to single-task baselines.
• The use of character-level features optimized via a multi-objective genetic algorithm enhances context awareness and predictive accuracy.
• Joint optimization of multiple loss functions leads to better generalization and robustness across morphological tags.
• The model is publicly available, including code, trained models, and datasets, enabling reproducibility and further research.