[Paper Review] Evaluating User Perception of Multi-Factor Authentication: A Systematic Review

Sanchari Das, Bingxing Wang+2 more|arXiv (Cornell University)|Aug 16, 2019

User Authentication and Security Systems33 references42 citations

TL;DR

The paper conducts a systematic review of MFA literature, finding few user evaluations and a trend of low adoption and notable demographic/research reporting biases.

ABSTRACT

Security vulnerabilities of traditional single factor authentication has become a major concern for security practitioners and researchers. To mitigate single point failures, new and technologically advanced Multi-Factor Authentication (MFA) tools have been developed as security solutions. However, the usability and adoption of such tools have raised concerns. An obvious solution can be viewed as conducting user studies to create more user-friendly MFA tools. To learn more, we performed a systematic literature review of recently published academic papers (N = 623) that primarily focused on MFA technologies. While majority of these papers (m = 300) proposed new MFA tools, only 9.1% of papers performed any user evaluation research. Our meta-analysis of user focused studies (n = 57) showed that researchers found lower adoption rate to be inevitable for MFAs, while avoidance was pervasive among mandatory use. Furthermore, we noted several reporting and methodological discrepancies in the user focused studies. We identified trends in participant recruitment that is indicative of demographic biases.

Motivation & Objective

• Assess the landscape of MFA research with a focus on user evaluation and usability.
• Quantify how many MFA studies include user-focused evaluations.
• Identify methodological and reporting gaps in user studies of MFA.
• Examine demographic biases and recruitment trends in participant samples.

Proposed method

• Systematically review recently published MFA-focused papers (n = 623).
• Classify papers by whether they propose new MFA tools or include user evaluations.
• Perform a meta-analysis on the subset of user-focused studies (n = 57).
• Quantify adoption and avoidance patterns in user studies.
• Identify reporting and methodological discrepancies across studies.

Experimental results

Research questions

• RQ1 What proportion of MFA papers include user evaluation research versus tool proposals?
• RQ2 What are the adoption and avoidance patterns for MFA in user-focused studies?
• RQ3 What methodological and reporting gaps exist in MFA user studies?
• RQ4 Are there demographic biases in participant recruitment for MFA usability research?

Key findings

• Most MFA papers (approximately 300) proposed new MFA tools rather than evaluating usability with users.
• Only about 9.1% of papers performed any user evaluation research.
• Meta-analysis of user-focused studies (n = 57) indicates lower adoption rates for MFAs and pervasive avoidance when use is mandatory.
• There are reporting and methodological discrepancies in user-focused MFA studies.
• Recruitment patterns show demographic biases among participants.