[Paper Review] Hermes: Unlocking Security Analysis of Cellular Network Protocols by Synthesizing Finite State Machines from Natural Language Specifications

Abdullah Al Ishtiaq, Sarkar Snigdha Sarathi Das+9 more|arXiv (Cornell University)|Oct 6, 2023

Hate Speech and Cyberbullying Detection4 citations

TL;DR

Hermes is an automated framework that synthesizes formal finite state machines (FSMs) from natural language cellular protocol specifications using a neural constituency parser (NEUTREX) and domain-specific language translation. It achieves 81–87% accuracy in FSM extraction and uncovers 3 new vulnerabilities and 19 known attacks in 4G and 5G protocols, enabling scalable security analysis without gold-standard models.

ABSTRACT

In this paper, we present Hermes, an end-to-end framework to automatically generate formal representations from natural language cellular specifications. We first develop a neural constituency parser, NEUTREX, to process transition-relevant texts and extract transition components (i.e., states, conditions, and actions). We also design a domain-specific language to translate these transition components to logical formulas by leveraging dependency parse trees. Finally, we compile these logical formulas to generate transitions and create the formal model as finite state machines. To demonstrate the effectiveness of Hermes, we evaluate it on 4G NAS, 5G NAS, and 5G RRC specifications and obtain an overall accuracy of 81-87%, which is a substantial improvement over the state-of-the-art. Our security analysis of the extracted models uncovers 3 new vulnerabilities and identifies 19 previous attacks in 4G and 5G specifications, and 7 deviations in commercial 4G basebands.

Motivation & Objective

• To address the lack of formal models for cellular network protocols, which hinders scalable security analysis.
• To automate the extraction of formal finite state machines (FSMs) from natural language specifications provided by 3GPP, reducing manual effort and error.
• To enable detection of design-level vulnerabilities in 4G and 5G protocols by generating accurate FSMs from evolving specifications.
• To overcome the limitations of hand-crafted models, which are time-consuming to build and update, especially with frequent specification changes.
• To provide a scalable, reusable framework for security analysis that supports evolving cellular standards without requiring gold-standard formal models.

Proposed method

• Hermes uses NEUTREX, a domain-informed neural constituency parser based on CellulaRoBERTa, to extract transition-relevant text and identify components such as states, conditions, and actions.
• It employs a domain-specific language (DSL) and dependency parsing to convert extracted components into logical formulas representing conditions and actions.
• The IRSynthesizer component translates these logical formulas into intermediate representations (IR) that can be transpiled into formal languages for model checking.
• The FSMSynthesizer compiles the IR into complete FSM transitions and constructs full finite state machines for security analysis.
• The framework supports multi-document processing, though experiments are limited to single documents for consistency with prior work.
• Manual verification and correction of model checker outputs are used to validate and refine generated FSMs, especially for spurious counterexamples.

Experimental results

Research questions

• RQ1 Can natural language specifications of cellular protocols be automatically transformed into accurate formal finite state machines?
• RQ2 How effective is the proposed neural parser NEUTREX in extracting transition components from complex, ambiguous natural language in 3GPP specifications?
• RQ3 To what extent can the synthesized FSMs detect known and previously unknown vulnerabilities in 4G and 5G protocols?
• RQ4 How does Hermes perform in comparison to existing approaches in terms of accuracy and vulnerability discovery?
• RQ5 What are the key limitations in handling underspecifications, ambiguities, and multi-document protocol descriptions?

Key findings

• Hermes achieves 81–87% accuracy in synthesizing finite state machines from natural language 3GPP specifications, significantly outperforming state-of-the-art methods.
• The framework successfully identifies 3 new vulnerabilities in 4G and 5G protocols, including one stemming from a newly introduced message rejection mechanism in 5G.
• It detects 19 previously known attacks that exploit design flaws in 4G and 5G NAS and RRC protocols, validating its ability to uncover real-world flaws.
• Hermes uncovers 7 implementation deviations in commercial 4G baseband chips, demonstrating its practical utility in real-world systems.
• Security analysis on generated FSMs reveals that even imperfect models can be effectively validated and corrected through manual inspection, reducing effort compared to full manual modeling.
• The framework is effective despite the absence of a gold-standard formal model from 3GPP, offering a viable alternative for ongoing protocol security analysis.