[Paper Review] Cognitive Triaging of Phishing Attacks

Amber van der Heijden, Luca Allodi|arXiv (Cornell University)|May 6, 2019

Spam and Phishing Detection45 references43 citations

TL;DR

The paper quantifies cognitive vulnerability triggers in phishing emails using supervised topic modeling and econometrics to predict attack success and guide remediation triage.

ABSTRACT

In this paper we employ quantitative measurements of cognitive vulnerability triggers in phishing emails to predict the degree of success of an attack. To achieve this we rely on the cognitive psychology literature and develop an automated and fully quantitative method based on machine learning and econometrics to construct a triaging mechanism built around the cognitive features of a phishing email; we showcase our approach relying on data from the anti-phishing division of a large financial organization in Europe. Our evaluation shows empirically that an effective triaging mechanism for phishing success can be put in place by response teams to effectively prioritize remediation efforts (e.g. domain takedowns), by first acting on those attacks that are more likely to collect high response rates from potential victims.

Motivation & Objective

• Explain why certain phishing emails are more successful by measuring cognitive vulnerability triggers in the wild.
• Develop an automated, quantitative method to measure cognitive triggers in phishing emails.
• Build a triaging model that predicts phishing success based on email cognitive features.
• Validate the approach using data from a large European financial organization’s anti-phishing operations.
• Demonstrate how the triaging model can prioritize remediation actions (e.g., domain takedowns).

Proposed method

• Construct a dataset of phishing emails and associated click data from Org, a large European financial firm.
• Identify cognitive vulnerability triggers in emails using a supervised Latent Dirichlet Allocation (LLDA) model aligned with Cialdini’s principles of influence.
• Train and evaluate the LLDA model with labeled examples, using 5-fold cross-validation and PROPORTIONAL rank-cutoff for multi-label detection.
• Link email content to observed clicks on landing URLs via a reconstruction of redirection chains and matching to suspicious URLs.
• Use bootstrapped econometric simulations to estimate coefficients and predictions relating cognitive triggers to phishing success.
• Aggregate results to derive a triaging mechanism that prioritizes remediation based on predicted click likelihood.
• Discuss data sanitization, duplicate detection, and limitations to ensure robust inference.

Experimental results

Research questions

• RQ1 Can cognitive vulnerability triggers in phishing emails be automatically quantified in real-world data?
• RQ2 How do individual cognitive triggers correlate with the observed click-through behavior on phishing domains?
• RQ3 Can a triaging model prioritize phishing remediation actions effectively based on cognitive features alone?
• RQ4 What is the predictive performance of a LLDA-based cognitive trigger identification approach in this setting?

Key findings

• The study analyzes over 80,000 phishing emails and links cognitive triggers to measured click data from organizational alerts.
• A supervised LLDA approach yields macro and micro performance metrics around 0.71–0.81 for sensitivity/specificity and 0.72–0.76 for F1, indicating a satisfactory fit.
• Cognitive vulnerability triggers such as Reciprocity, Consistency, Social Proof, Authority, Liking, and Scarcity can be identified and associated with email content.
• The results support that triaging phishing emails by predicted success (click likelihood) can inform remediation prioritization in operational settings.
• The methodology provides empirical evidence that cognitive factors correlate with phishing success in the wild, beyond purely technical indicators.