[Paper Review] User-Centred Security Education: A Game Design to Thwart Phishing Attacks

Nalin Asanka Gamagedara Arachchilage|arXiv (Cornell University)|Nov 11, 2015

Spam and Phishing Detection12 references8 citations

TL;DR

This study designs and evaluates a mobile game that teaches users to identify phishing URLs by simulating a fish-themed adventure where players distinguish legitimate from fake web addresses. The game significantly improved users' phishing avoidance behavior, with post-test scores rising from 56% to 84% after gameplay, driven by enhanced threat perception and self-efficacy.

ABSTRACT

Phishing is an online identity theft that aims to steal sensitive information such as username, password and online banking details from its victims. Phishing education needs to be considered as a means to combat this threat. This paper reports on a design and development of a mobile game prototype as an educational tool helping computer users to protect themselves against phishing attacks. The elements of a game design framework for avoiding phishing attacks were used to address the game design issues. Game design principles served as guidelines for structuring and presenting information. Our mobile game design aimed to enhance the users' avoidance behaviour through motivation to protect themselves against phishing threats. A think-aloud study was conducted, along with a pre- and post-test, to assess the game design framework though the developed mobile game prototype. The study results showed a significant improvement of participants' phishing avoidance behaviour in their post-test assessment. Furthermore, the study findings suggest that participants' threat perception, safeguard effectiveness, self-efficacy, perceived severity and perceived susceptibility elements positively impact threat avoidance behaviour, whereas safeguard cost had a negative impact on it.

Motivation & Objective

• To address the persistent vulnerability of end-users to phishing attacks, which remain a major security threat despite automated detection tools.
• To develop a user-centred mobile game that enhances users’ ability to identify phishing websites by focusing on URL analysis.
• To evaluate the effectiveness of a game design framework based on perceived threat, self-efficacy, and safeguard costs in improving phishing avoidance behavior.
• To investigate how storytelling and interactive feedback in a mobile game can strengthen users’ threat perception and motivation to avoid phishing.

Proposed method

• A mobile game prototype was developed for Android, featuring a narrative where a small fish must identify real worms (legitimate URLs) and avoid fake worms (phishing URLs).
• The game incorporates a teacher character who provides tips on URL characteristics (e.g., numbers at the start, hyphens in company names) to improve detection skills.
• Players earn points for correct identifications and lose lives or time for errors; using the teacher reduces remaining time by 100 seconds, modeling safeguard cost.
• A think-aloud study with 20 participants was conducted, combining pre- and post-tests to measure changes in phishing detection ability.
• System Usability Scale (SUS) was used to assess user satisfaction with the game interface, yielding a high score of 83.62 out of 100.
• Paired-samples t-tests were applied to compare pre- and post-test performance, assessing statistical significance in behavioral improvement.

Experimental results

Research questions

• RQ1 How can a game design framework be effectively applied to educate users about phishing detection through mobile gaming?
• RQ2 To what extent does the game prototype improve users’ ability to distinguish legitimate from phishing URLs?
• RQ3 Which psychological factors—such as threat perception, self-efficacy, and safeguard cost—influence users’ phishing avoidance behavior in the game context?
• RQ4 How does the integration of storytelling and interactive feedback affect learning outcomes in phishing education?

Key findings

• Post-test scores improved significantly from a mean of 56.00% (SD = 17.911) in the pre-test to 84.00% (SD = 13.139) in the post-test, with a statistically significant t-value of -7.97 (p < 0.005).
• Eighteen out of 20 participants scored above 80% on the post-test, and five achieved a perfect 100% score, indicating strong learning gains.
• Users’ threat perception, safeguard effectiveness, self-efficacy, perceived severity, and perceived susceptibility all had a positive impact on phishing avoidance behavior.
• Safeguard cost negatively influenced avoidance behavior, as users reduced their use of help features when time penalties were applied.
• The System Usability Scale (SUS) score of 83.62 indicates high user satisfaction with the game interface and overall user experience.
• The game prototype successfully enhanced motivation to protect against phishing by embedding psychological motivators into gameplay mechanics.