[Paper Review] Inverse-Category-Frequency Based Supervised Term Weighting Schemes for Text Categorization *

Deqing Wang, Hui Zhang|arXiv (Cornell University)|Mar 1, 2013

Text and Document Classification Technologies34 references67 citations

TL;DR

This paper proposes two novel supervised term weighting schemes—tf.icf and icf-based—by replacing the traditional inverse document frequency (idf) with inverse category frequency (icf), which prioritizes terms rare across categories rather than documents. Experiments show both methods outperform or match six supervised and three traditional schemes in macro-F1 and micro-F1 across multiple classifiers and corpora.

ABSTRACT

Term weighting schemes often dominate the performance of many classifiers, such as kNN, centroid-based classifier and SVMs. The widely used term weighting scheme in text categorization, i.e., tf.idf, is originated from information retrieval (IR) field. The intuition behind idf for text categorization seems less reasonable than IR. In this paper, we introduce inverse category frequency (icf) into term weighting scheme and propose two novel approaches, i.e., tf.icf and icf-based supervised term weighting schemes. The tf.icf adopts icf to substitute idf factor and favors terms occurring in fewer categories, rather than fewer documents. And the icf-based approach combines icf and relevance frequency (rf) to weight terms in a supervised way. Our cross-classifier and cross-corpus experiments have shown that our proposed approaches are superior or comparable to six supervised term weighting schemes and three traditional schemes in terms of macro-F1 and micro-F1.

Motivation & Objective

• To address the limitation of tf.idf in text categorization, where idf's focus on document rarity is less suitable than category rarity.
• To explore whether inverse category frequency (icf) can better reflect term discriminative power in supervised text categorization.
• To develop supervised term weighting schemes that combine icf with relevance frequency (rf) for improved categorization performance.
• To evaluate the proposed schemes across multiple classifiers and corpora to ensure generalization and robustness.

Proposed method

• Proposes tf.icf, a term weighting scheme that substitutes idf with icf, favoring terms that appear in fewer categories rather than fewer documents.
• Introduces an icf-based supervised term weighting scheme that combines icf with relevance frequency (rf), where rf measures term frequency in relevant categories.
• Uses icf = log(C / N_c), where C is the total number of categories and N_c is the number of categories containing the term.
• Applies the weighted terms in kNN, centroid-based, and SVM classifiers to evaluate performance across different learning paradigms.
• Employs cross-classifier and cross-corpus evaluation to validate the robustness of the proposed schemes.
• Compares the proposed methods against six supervised and three traditional term weighting schemes using macro-F1 and micro-F1 as metrics.

Experimental results

Research questions

• RQ1 Can replacing idf with icf in term weighting improve text categorization performance in supervised settings?
• RQ2 How does the combination of icf and relevance frequency (rf) compare to existing supervised term weighting schemes?
• RQ3 Do the proposed schemes generalize across different classifiers and text corpora?
• RQ4 Is the performance of tf.icf and icf-based schemes superior or comparable to established term weighting methods in terms of macro-F1 and micro-F1?

Key findings

• The tf.icf scheme outperforms or matches the performance of six supervised and three traditional term weighting schemes in terms of macro-F1 and micro-F1.
• The icf-based supervised term weighting scheme achieves superior or comparable performance across multiple classifiers and corpora.
• The use of icf, which emphasizes category rarity, leads to better discriminative term selection than document-based idf in text categorization.
• Cross-classifier and cross-corpus experiments confirm the robustness and generalization capability of the proposed term weighting schemes.
• The results demonstrate that icf-based weighting is particularly effective in scenarios where category-level discriminability is critical.
• The proposed methods show consistent improvements in both macro-F1 and micro-F1, indicating balanced performance across all categories.