[Paper Review] Towards Real-time Customer Experience Prediction for Telecommunication Operators

Ernesto Diaz-Aviles, Fabio Pinelli+8 more|arXiv (Cornell University)|Aug 12, 2015

Customer churn and segmentation12 references4 citations

TL;DR

This paper proposes a real-time customer experience prediction system for telecom operators using a Restricted Random Forest model trained on anonymized mobile network transaction data to predict whether a user will call customer care, leveraging call events as a proxy for poor experience. The approach achieves promising predictive performance, enabling proactive issue detection and improved customer service with potential to reduce churn.

ABSTRACT

Telecommunications operators (telcos) traditional sources of income, voice and SMS, are shrinking due to customers using over-the-top (OTT) applications such as WhatsApp or Viber. In this challenging environment it is critical for telcos to maintain or grow their market share, by providing users with as good an experience as possible on their network. But the task of extracting customer insights from the vast amounts of data collected by telcos is growing in complexity and scale everey day. How can we measure and predict the quality of a user's experience on a telco network in real-time? That is the problem that we address in this paper. We present an approach to capture, in (near) real-time, the mobile customer experience in order to assess which conditions lead the user to place a call to a telco's customer care center. To this end, we follow a supervised learning approach for prediction and train our 'Restricted Random Forest' model using, as a proxy for bad experience, the observed customer transactions in the telco data feed before the user places a call to a customer care center. We evaluate our approach using a rich dataset provided by a major African telecommunication's company and a novel big data architecture for both the training and scoring of predictive models. Our empirical study shows our solution to be effective at predicting user experience by inferring if a customer will place a call based on his current context. These promising results open new possibilities for improved customer service, which will help telcos to reduce churn rates and improve customer experience, both factors that directly impact their revenue growth.

Motivation & Objective

• To enable real-time prediction of customer experience for telecom operators using existing network data.
• To address the challenge of extracting timely customer insights from massive, high-velocity mobile data streams.
• To use customer care calls as a proxy for poor user experience, avoiding reliance on surveys.
• To develop a scalable, interpretable machine learning model that identifies key factors driving user dissatisfaction.
• To support proactive customer care by detecting potential issues before widespread complaints occur.

Proposed method

• Train a Restricted Random Forest (RRF) model using network transaction features such as download rate, retransmission rate, app type, device model, and time of day as input features.
• Use customer care call events as the positive label in a binary classification task, treating all transactions before a call as indicative of bad experience.
• Apply a two-pronged approach: (I) transaction-level prediction using raw data streams, and (II) user profile-level prediction using aggregated user context.
• Leverage decision tree interpretability to identify key performance indicators (KPIs) and application-specific issues affecting user experience.
• Implement a big data pipeline for real-time scoring and model training using a novel architecture deployed in a real telco environment.
• Use majority voting across trees for final prediction and analyze feature importance for actionable insights.

Experimental results

Research questions

• RQ1 Can network performance metrics and user context be used to predict whether a user will call customer care in the near future?
• RQ2 How effective is a supervised learning approach using call logs as a proxy for poor customer experience in real-time?
• RQ3 Can a Restricted Random Forest model provide both high predictive performance and interpretability for telecom operators?
• RQ4 To what extent can real-time prediction help identify network issues before they affect large user groups?
• RQ5 How do different modeling approaches (transaction-level vs. profile-level) compare in capturing user experience dynamics?

Key findings

• The proposed approach successfully predicts customer care calls with promising performance, demonstrating the feasibility of real-time experience monitoring.
• The model identified a specific issue with Apple Maps retransmission rates that correlated with a surge in customer care calls, confirming its ability to detect real network problems.
• Approach II, which uses aggregated user profiles, enabled the detection of application-specific issues and provided interpretable insights into root causes.
• The model's recall improvement indicates better detection of users likely to call, though precision remains low, suggesting a high number of false positives.
• The use of customer care calls as a proxy for bad experience proved effective, as it is more abundant than survey feedback and actionable for proactive intervention.
• Future work indicates potential for hybrid models combining transaction-level and profile-level predictions to improve accuracy and timeliness.