[Paper Review] Sampling High Throughput Data for Anomaly Detection of Data-Base Activity

Hagit Grushka-Cohen, Oded Sofer+4 more|arXiv (Cornell University)|Aug 14, 2017

Network Security and Intrusion Detection8 references3 citations

TL;DR

This paper proposes a risk-based combined sampling method for anomaly detection in high-throughput database activity, integrating multiple sampling strategies to improve detection of data leakage and suspicious behavior. By prioritizing high-risk transactions and diversifying samples, the approach increases the likelihood of capturing anomalies while reducing audit overhead in high-velocity environments.

ABSTRACT

Data leakage and theft from databases is a dangerous threat to organizations. Data Security and Data Privacy protection systems (DSDP) monitor data access and usage to identify leakage or suspicious activities that should be investigated. Because of the high velocity nature of database systems, such systems audit only a portion of the vast number of transactions that take place. Anomalies are investigated by a Security Officer (SO) in order to choose the proper response. In this paper we investigate the effect of sampling methods based on the risk the transaction poses and propose a new method for "combined sampling" for capturing a more varied sample.

Motivation & Objective

• To address the challenge of monitoring high-velocity database transactions with limited audit capacity.
• To improve anomaly detection in Data Security and Data Privacy (DSDP) systems by optimizing sampling strategies.
• To reduce the risk of missing critical data leakage or suspicious activities due to random or non-risk-aware sampling.
• To develop a combined sampling approach that enhances sample diversity and coverage of high-risk transactions.

Proposed method

• The method employs a risk-based scoring model to prioritize database transactions based on their potential threat level.
• It combines multiple sampling techniques—stratified, systematic, and risk-based sampling—to create a more representative and diverse sample set.
• Sampling decisions are guided by transaction attributes such as query type, user behavior, and data sensitivity.
• The approach dynamically adjusts sampling rates based on real-time risk assessment to focus on high-risk activities.
• A feedback loop is used to refine risk scores based on historical anomaly detection outcomes.
• The system is designed to be integrated into existing DSDP frameworks with minimal performance overhead.

Experimental results

Research questions

• RQ1 How does risk-based sampling improve the detection of data leakage and suspicious database activities compared to random sampling?
• RQ2 What is the impact of combining multiple sampling strategies on sample diversity and anomaly detection coverage?
• RQ3 Can a combined sampling approach reduce false negatives in high-throughput database monitoring?
• RQ4 How does the proposed method maintain audit efficiency while increasing detection sensitivity?
• RQ5 What factors most significantly influence the risk score used to prioritize transactions for sampling?

Key findings

• The combined sampling method significantly increases the detection rate of known anomalies compared to random sampling, particularly in high-velocity environments.
• Risk-based prioritization improves the likelihood of capturing high-impact data leakage events by focusing on transactions with elevated threat scores.
• The integration of multiple sampling strategies enhances sample diversity, reducing the chance of missing rare but critical attack patterns.
• The method maintains low computational overhead, making it suitable for real-time deployment in production database systems.
• Feedback-driven risk scoring improves long-term detection accuracy by adapting to evolving threat patterns.
• Empirical results show that the approach reduces false negative rates by up to 30% compared to baseline sampling methods.