[Paper Review] Authorship Identification in Bengali Literature: a Comparative Analysis

Tanmoy Chakraborty|arXiv (Cornell University)|Aug 30, 2012

Authorship Attribution and Profiling11 references17 citations

TL;DR

This paper proposes a novel authorship identification system for Bengali literature using stylometric features and machine learning, demonstrating that SVM outperforms statistical models and other classifiers. It identifies key stylistic markers—such as word length, keyword overlap, and phrase frequency—that consistently distinguish authors, even in the absence of genre or temporal cues.

ABSTRACT

Stylometry is the study of the unique linguistic styles and writing behaviors of individuals. It belongs to the core task of text categorization like authorship identification, plagiarism detection etc. Though reasonable number of studies have been conducted in English language, no major work has been done so far in Bengali. In this work, We will present a demonstration of authorship identification of the documents written in Bengali. We adopt a set of fine-grained stylistic features for the analysis of the text and use them to develop two different models: statistical similarity model consisting of three measures and their combination, and machine learning model with Decision Tree, Neural Network and SVM. Experimental results show that SVM outperforms other state-of-the-art methods after 10-fold cross validations. We also validate the relative importance of each stylistic feature to show that some of them remain consistently significant in every model used in this experiment.

Motivation & Objective

• To develop a machine learning-based authorship identification system for Bengali literature, a low-resource language with limited prior stylometric research.
• To compare statistical similarity models (cosine, chi-square, Euclidean) with machine learning models (SVM, Decision Tree, Neural Network) on the same feature set.
• To identify and validate the relative importance of stylistic features in distinguishing authors across different models.
• To establish a baseline for future stylometric studies in Indian languages by analyzing a curated corpus of Bengali literary texts.

Proposed method

• Preprocessing Bengali text using a shallow parser to extract part-of-speech, chunk boundaries, and sentence segmentation.
• Extracting 11 normalized stylistic features at token, phrase, and context levels, including word length, punctuation count, noun/verb phrase frequency, and keyword overlap.
• Applying TF-IDF to identify top 50 high-frequency keywords per author, excluding stop words, to form author-specific keyword sets.
• Using 10-fold cross-validation to evaluate model performance and prevent overfitting due to limited dataset size.
• Employing SVM, Decision Tree, and Neural Network classifiers to compare predictive accuracy and feature importance.
• Conducting ablation studies by removing one feature at a time to measure its impact on model accuracy and assess feature relevance.

Experimental results

Research questions

• RQ1 Can machine learning models effectively identify authors in Bengali literary texts using stylometric features when genre and time period are controlled?
• RQ2 How do statistical similarity measures (cosine, chi-square, Euclidean) compare to machine learning models (SVM, DT, NN) in authorship identification for Bengali?
• RQ3 Which stylistic features are most consistently significant across different machine learning models in Bengali authorship identification?
• RQ4 How does feature ablation affect model accuracy, and which features are most responsible for performance differences?
• RQ5 What are the primary sources of misclassification in the authorship identification system, and how do they relate to author-specific writing patterns?

Key findings

• SVM achieved the highest average accuracy among all models, with significantly lower variance in performance across 10-fold cross-validation, indicating robustness.
• The word length feature was consistently the most important across all models, followed by keyword overlap with target authors (KW(R), KW(A), KW(O)) and noun phrase frequency.
• Features like dialog length and unknown word count showed high importance in SVM but were less influential in Decision Tree and Neural Network models, indicating model-dependent feature relevance.
• Cohen’s Kappa analysis revealed high inter-model agreement between Decision Tree and Neural Network, but both were less accurate than SVM, suggesting SVM’s superior discriminative power.
• The system exhibited a bias toward overestimating author R, likely due to less diverse and less well-structured training data for that author’s corpus.
• Ablation studies showed that removing key features like word length or keyword overlap caused the largest drops in accuracy, confirming their central role in author discrimination.