[Paper Review] Interpretable Text Classification Using CNN and Max-pooling

Hao Cheng, Xiaoqing Yang+3 more|arXiv (Cornell University)|Oct 14, 2019

Topic Modeling22 references4 citations

TL;DR

This paper proposes a CNN-based text classification model with interpretable mechanisms using convolution attribution and n-gram feature analysis to trace input token contributions to predictions. It achieves state-of-the-art performance while enabling detailed, human-readable explanations of model decisions through visualization and pattern matching with training samples.

ABSTRACT

Deep neural networks have been widely used in text classification. However, it is hard to interpret the neural models due to the complicate mechanisms. In this work, we study the interpretability of a variant of the typical text classification model which is based on convolutional operation and max-pooling layer. Two mechanisms: convolution attribution and n-gram feature analysis are proposed to analyse the process procedure for the CNN model. The interpretability of the model is reflected by providing posterior interpretation for neural network predictions. Besides, a multi-sentence strategy is proposed to enable the model to beused in multi-sentence situation without loss of performance and interpret ability. We evaluate the performance of the model on several classification tasks and justify the interpretable performance with some case studies.

Motivation & Objective

• To address the lack of interpretability in deep neural network text classifiers, especially for CNN-based models.
• To establish pointwise associations between input tokens, learned n-gram features, and final predictions.
• To enable interpretation in multi-sentence contexts without performance or interpretability loss.
• To validate interpretability through case studies and visualization of prediction contributions.
• To identify and analyze misprediction causes by tracing judgment bases to training samples.

Proposed method

• Convolution attribution dissects the convolution operation to compute the contribution of each word vector value to n-gram features.
• N-gram feature parsing backtracks max-pooling operations to link specific n-gram features to final classification decisions.
• A multi-sentence weighting strategy aggregates representations across multiple sentences while preserving interpretability.
• The model uses 1D convolutions with multiple kernel sizes to detect local n-gram patterns in word vector sequences.
• Interpretability is visualized by mapping word-level contributions to target categories, enabling human-readable explanations.
• Error analysis is performed by matching prediction patterns to similar training samples to identify misleading features.

Experimental results

Research questions

• RQ1 How can we interpret the contribution of individual input tokens to the final prediction in a CNN-based text classifier?
• RQ2 Can we establish a reliable link between learned n-gram features and input word vectors in a CNN model?
• RQ3 How can interpretability be preserved in multi-sentence text classification without sacrificing performance?
• RQ4 To what extent can the model’s decision-making process be explained through feature-level attribution and pattern matching?
• RQ5 Can we identify and diagnose mispredictions by tracing judgment bases to specific training samples?

Key findings

• The model achieves performance comparable to state-of-the-art classifiers on text classification tasks, including TREC and other benchmarks.
• Convolution attribution successfully identifies specific words that contribute most to the prediction, such as 'intelligent' for the 'technology brand' category.
• N-gram feature analysis reveals that the model learns semantically meaningful patterns, such as 'how long' strongly indicating the 'NUM' category.
• Visualization shows that the model's attention is more precise than attention mechanisms in locating relevant words, with clearer and more focused contributions.
• Pattern matching with training samples reveals that low-confidence predictions often stem from unreasonable or misleading feature combinations.
• The multi-sentence strategy maintains performance and interpretability, enabling effective analysis of long input texts.