[Paper Review] Analyzing the Gadgets Towards a Metric to Measure Gadget Quality

Andreas Follner, Alexandre Bartel+1 more|arXiv (Cornell University)|May 26, 2016

Security and Verification in Computing14 references4 citations

TL;DR

This paper proposes four novel metrics to evaluate gadget quality in binary code, addressing the lack of standardized evaluation for code-reuse attacks like ROP. Applied to Intel MPX-protected binaries, the metrics reveal a 17% increase in useful gadgets, indicating MPX may inadvertently increase exploitability despite its security intent.

ABSTRACT

Current low-level exploits often rely on code-reuse, whereby short sections of code (gadgets) are chained together into a coherent exploit that can be executed without the need to inject any code. Several protection mechanisms attempt to eliminate this attack vector by applying code transformations to reduce the number of available gadgets. Nevertheless, it has emerged that the residual gadgets can still be sufficient to conduct a successful attack. Crucially, the lack of a common metric for "gadget quality" hinders the effective comparison of current mitigations. This work proposes four metrics that assign scores to a set of gadgets, measuring quality, usefulness, and practicality. We apply these metrics to binaries produced when compiling programs for architectures implementing Intel's recent MPX CPU extensions. Our results demonstrate a 17% increase in useful gadgets in MPX binaries, and a decrease in side-effects and preconditions, making them better suited for ROP attacks.

Motivation & Objective

• To address the lack of a standardized metric for evaluating gadget quality in binary code, particularly for ROP-based exploits.
• To enable researchers to compare the security impact of binary transformations, such as compiler optimizations or exploit mitigation techniques.
• To evaluate whether mitigation techniques like Intel's MPX actually reduce or increase the attack surface by altering gadget quality.
• To provide a practical, tool-supported framework for assessing gadget usability across diverse exploit scenarios.
• To demonstrate that existing metrics based solely on gadget count reduction are insufficient for evaluating real-world exploit feasibility.

Proposed method

• Define four metrics—two based on real-world exploit scenarios (arbitrary code execution, information leakage) and two general-purpose metrics (gadget variety and usability)—to assess gadget quality.
• Design the metrics to evaluate key exploit characteristics: preconditions, side effects, and instruction count, reflecting practical exploit development challenges.
• Implement the metrics in GaLity, an open-source tool for automated gadget quality analysis on binary sets.
• Apply the metrics to a case study analyzing binaries compiled with Intel's MPX (Memory Protection Extensions), comparing gadget sets across eight representative x64 Windows binaries.
• Use static analysis to extract gadgets and score them based on exploit relevance, including conditional checks for register preservation and control-flow integrity.
• Quantify changes in gadget quality by comparing MPX-enabled binaries to their non-MPX counterparts using the defined metrics.

Experimental results

Research questions

• RQ1 To what extent do existing metrics for gadget reduction fail to capture the true exploitability of remaining gadgets?
• RQ2 How do different binary transformations, such as those from exploit mitigation techniques, affect the quality and practicality of available gadgets?
• RQ3 Does Intel's MPX, intended to prevent buffer overflows, actually increase the number and quality of usable ROP gadgets?
• RQ4 Can a standardized, multi-dimensional metric accurately reflect the real-world exploit potential of a gadget set beyond simple count reduction?
• RQ5 How do specific exploit scenarios (e.g., arbitrary code execution, information leakage) influence the evaluation of gadget quality?

Key findings

• MPX-enabled binaries exhibit a 17% increase in the number of useful gadgets compared to non-MPX binaries, indicating a measurable increase in exploit surface.
• Gadgets in MPX binaries show reduced side effects and fewer preconditions, making them more practical for ROP chaining despite the mitigation’s intent.
• The study reveals that existing metrics like DAIR (Dynamic Average Indirect Target Reduction) are insufficient, as they do not account for gadget quality or usability.
• Even though MPX reduces the number of indirect jumps, the remaining gadgets are of higher quality and more amenable to exploitation than in unmodified binaries.
• The results suggest that some mitigation techniques may unintentionally degrade security by improving gadget quality, highlighting the need for quality-aware evaluation.
• GaLity, the implemented tool, enables automated, reproducible evaluation of gadget sets across diverse binaries and transformation techniques.