[Paper Review] Crime Prediction using Machine Learning with a Novel Crime Dataset

Faisal Tareque Shohan, Abu Ubaida Akash+2 more|arXiv (Cornell University)|Nov 3, 2022

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper introduces a novel, comprehensive crime dataset for Bangladesh—spanning 6,574 incidents over seven years—integrating spatio-temporal, weather, and demographic features through manual data collection from news archives, official databases, and census reports. Using feature engineering and SMOTE, it evaluates five supervised ML models (Random Forest, XGBoost, AdaBoost, Extra Tree, Decision Tree), achieving satisfactory prediction performance, thus establishing a foundational resource for crime forecasting in Bangladesh and similar contexts.

ABSTRACT

Crime is an unlawful act that carries legal repercussions. Bangladesh has a high crime rate due to poverty, population growth, and many other socio-economic issues. For law enforcement agencies, understanding crime patterns is essential for preventing future criminal activity. For this purpose, these agencies need structured crime database. This paper introduces a novel crime dataset that contains temporal, geographic, weather, and demographic data about 6574 crime incidents of Bangladesh. We manually gather crime news articles of a seven year time span from a daily newspaper archive. We extract basic features from these raw text. Using these basic features, we then consult standard service-providers of geo-location and weather data in order to garner these information related to the collected crime incidents. Furthermore, we collect demographic information from Bangladesh National Census data. All these information are combined that results in a standard machine learning dataset. Together, 36 features are engineered for the crime prediction task. Five supervised machine learning classification algorithms are then evaluated on this newly built dataset and satisfactory results are achieved. We also conduct exploratory analysis on various aspects the dataset. This dataset is expected to serve as the foundation for crime incidence prediction systems for Bangladesh and other countries. The findings of this study will help law enforcement agencies to forecast and contain crime as well as to ensure optimal resource allocation for crime patrol and prevention.

Motivation & Objective

• Address the lack of standardized, comprehensive, and systematically structured crime data for Bangladesh, which currently relies on aggregated or incomplete sources.
• Develop a high-quality, multi-dimensional dataset integrating crime incidents with temporal, geographic, meteorological, and demographic features for machine learning applications.
• Enable accurate crime prediction in Bangladesh by constructing a dataset suitable for supervised learning, overcoming data scarcity and imbalance issues.
• Provide a replicable framework for crime data curation and feature engineering applicable to other developing countries with limited official crime data infrastructure.

Proposed method

• Manually collected 6,574 crime news articles from a daily newspaper archive over a seven-year period (2013–2020), extracting basic incident details such as date, time, location, and crime type.
• Enhanced raw text data using standard geolocation services to derive precise latitude and longitude for each incident, and integrated historical weather data (temperature, humidity, precipitation) from a reliable weather database.
• Sourced demographic data (e.g., population density, age distribution, income levels) from the Bangladesh National Census to enrich the dataset with socio-economic context.
• Performed extensive feature engineering to create 36 structured features, including time-of-day, day-of-week, seasonal trends, and weather-condition indicators, to improve model interpretability and performance.
• Applied data curing techniques to handle missing values and inconsistencies, and used SMOTE oversampling to mitigate class imbalance in the crime type distribution.
• Evaluated five supervised machine learning classifiers—Random Forest, XGBoost, AdaBoost, Extra Tree, and Decision Tree—on the engineered dataset using standard metrics (accuracy, F1-score, AUC).

Experimental results

Research questions

• RQ1 How can a standardized, multi-source, and comprehensive crime dataset be systematically constructed for Bangladesh using publicly available and manually curated data?
• RQ2 To what extent do temporal, geographic, meteorological, and demographic factors influence crime occurrence in Bangladesh, and how can they be effectively encoded as features?
• RQ3 Which supervised machine learning models perform best for crime prediction on a newly constructed, multi-dimensional dataset from a developing country context?
• RQ4 How effective are feature engineering and data balancing techniques (e.g., SMOTE) in improving the predictive performance of crime forecasting models on this dataset?

Key findings

• The study successfully constructed a novel, standardized crime dataset of 6,574 incidents from Bangladesh, integrating temporal, geographic, weather, and demographic features across six crime types.
• Feature engineering significantly enhanced data quality and model readiness, resulting in a final dataset with 36 engineered features suitable for machine learning.
• Among the evaluated models, XGBoost and Random Forest achieved the highest prediction accuracy, with XGBoost outperforming others in F1-score and AUC, indicating strong generalization on imbalanced crime data.
• The application of SMOTE oversampling effectively reduced the impact of class imbalance, improving model performance on minority crime categories.
• Exploratory data analysis revealed clear temporal and spatial patterns in crime distribution, such as higher incident rates during specific months and in urban centers, supporting the utility of the dataset for policy and patrol planning.
• The dataset is the first of its kind for Bangladesh, offering a comprehensive, multi-source foundation for future research and operational crime prediction systems in the country.