[Paper Review] IBM Employee Attrition Analysis

Shenghuan Yang, Tariqul Islam|arXiv (Cornell University)|Dec 2, 2020

AI and HR Technologies12 references18 citations

TL;DR

This study analyzes IBM employee attrition using Random Forest, K-means clustering, and binary logistic regression to identify key predictors of employee turnover. It finds that frequent travelers (2.4× higher attrition), HR department employees, and those with lower job satisfaction are more likely to leave, while higher income, age, and experience reduce attrition risk. Logistic regression achieved 88.43% accuracy in predicting attrition.

ABSTRACT

In this paper, we analyzed the dataset IBM Employee Attrition to find the main reasons why employees choose to resign. Firstly, we utilized the correlation matrix to see some features that were not significantly correlated with other attributes and removed them from our dataset. Secondly, we selected important features by exploiting Random Forest, finding monthlyincome, age, and the number of companies worked significantly impacted employee attrition. Next, we also classified people into two clusters by using K-means Clustering. Finally, We performed binary logistic regression quantitative analysis: the attrition of people who traveled frequently was 2.4 times higher than that of people who rarely traveled. And we also found that employees who work in Human Resource have a higher tendency to leave.

Motivation & Objective

• To identify the main factors driving employee attrition in IBM using machine learning.
• To predict employee turnover using binary logistic regression and ensemble models.
• To classify employee clusters based on attrition risk using K-means clustering.
• To quantify the impact of demographic and job-related variables on attrition through statistical analysis.
• To provide actionable insights for HR to reduce turnover and improve job satisfaction.

Proposed method

• Applied Random Forest to rank feature importance, identifying monthly income, age, and number of companies worked as top predictors.
• Used K-means clustering to group employees into two clusters based on attrition risk, revealing distinct behavioral patterns.
• Performed binary logistic regression to model the odds ratio of attrition across categorical variables, with attrition as a binary outcome (0: no, 1: yes).
• Preprocessed data by removing low-correlation features using a correlation matrix to improve model efficiency.
• Split the dataset (80% training, 20% testing) to evaluate model performance and prevent overfitting.
• Calculated odds ratios (OR) with 95% confidence intervals to quantify the effect of each predictor on attrition.

Experimental results

Research questions

• RQ1 Which employee characteristics most significantly influence the likelihood of voluntary resignation at IBM?
• RQ2 How do job role, travel frequency, and marital status affect employee attrition rates?
• RQ3 Can machine learning models accurately predict employee attrition using demographic and job-related features?
• RQ4 What are the distinct behavioral clusters of employees with high versus low attrition risk?
• RQ5 How does job satisfaction, as reflected in income and job level, correlate with employee retention?

Key findings

• Employees who travel frequently have a 2.411 times higher odds of attrition compared to those who rarely travel (p < 0.05).
• Employees in the Human Resources department have a 4.060 times higher odds of leaving than Sales Representatives (p < 0.05).
• Married and divorced employees have 0.427 and 0.304 times the odds of leaving, respectively, compared to single employees (p < 0.05).
• Females have 0.659 times the odds of leaving compared to males, indicating lower attrition rates among women (p < 0.05).
• Employees who work overtime have only 0.138 times the odds of leaving, suggesting a strong negative correlation with attrition (p < 0.05).
• The binary logistic regression model achieved 88.43% accuracy in predicting employee attrition, outperforming Random Forest (84.56% accuracy).