[Paper Review] Message Classification in the Call Center

Stephan Busemann, Sven Schmeier+1 more|ArXiv.org|Mar 14, 2000

Spam and Phishing Detection13 references4 citations

TL;DR

This paper presents a machine learning-based system for classifying customer e-mails in call centers using shallow text processing and statistical learning techniques. The ICC-mail system achieves 78% accuracy in suggesting correct solutions (top five proposals), enabling experienced agents to double their performance by reducing time per request through automated classification and solution retrieval.

ABSTRACT

Customer care in technical domains is increasingly based on e-mail communication, allowing for the reproduction of approved solutions. Identifying the customer's problem is often time-consuming, as the problem space changes if new products are launched. This paper describes a new approach to the classification of e-mail requests based on shallow text processing and machine learning techniques. It is implemented within an assistance system for call center agents that is used in a commercial setting.

Motivation & Objective

• To reduce call center agent workload by automating the classification of customer e-mail requests.
• To enable faster and more consistent response generation by suggesting pre-approved solution blocks.
• To support dynamic category changes due to new products or regulations without requiring deep linguistic modeling.
• To develop a scalable, trainable system that improves over time through relearning with new data.
• To maintain high usability and integration within real-world call center workflows.

Proposed method

• The system applies shallow text processing (STP) to extract linguistic features such as part-of-speech tags, word stems, negations, and sentence types from German e-mail texts.
• Statistical machine learning (SML) techniques are used to train a classifier on labeled e-mail data, mapping messages to predefined solution categories.
• The classifier is retrained periodically ('relearning') using new data and categories, allowing adaptation to evolving problem spaces.
• A three-level category system is used, with new categories introduced by a knowledge engineer and manually classified until sufficient training data is available.
• The system integrates with a graphical user interface that presents top solution suggestions to agents for selection and customization.
• The approach avoids deep syntactic or semantic analysis, favoring efficiency and adaptability in informal, domain-specific e-mail communication.

Experimental results

Research questions

• RQ1 Can shallow text processing and statistical learning effectively classify informal, domain-specific e-mails in a dynamic call center environment?
• RQ2 How does the system maintain accuracy when new product-related categories are introduced?
• RQ3 To what extent can automated classification reduce agent response time and improve solution consistency?
• RQ4 What is the impact of document length, informal language, and morphological variation on classification performance?
• RQ5 How does relearning improve classifier accuracy over time with incremental data?

Key findings

• The ICC-mail system achieves 78% accuracy in suggesting the correct solution among the top five recommendations, corresponding to an overall performance of 73% due to processing 94% of incoming e-mails.
• Experienced agents can process requests in 50–70 seconds on average when using the system, approximately doubling their performance compared to manual processing.
• Classifier accuracy improves with each relearning cycle, especially after 30 manually classified documents are added for a new category.
• The system is deployable in real-world settings, with users accepting its interface and functionality under real-time conditions.
• The approach is scalable and adaptable to changing categories, supporting ongoing maintenance through periodic retraining.
• Initial integration with GermaNet shows promise for enhancing semantic features, though full exploration is ongoing.