[Paper Review] Smart Contract Vulnerabilities: Vulnerable Does Not Imply Exploited

Daniel Pérez, Benjamin Livshits|arXiv (Cornell University)|Feb 18, 2019

Blockchain Technology Applications and SecurityComputer Science35 references41 citations

TL;DR

The paper analyzes six academic vulnerability datasets (23,327 vulnerable contracts) and finds that only 1.98% were exploited, representing about 8,487 ETH (≈1.7M USD), i.e., 0.27% of the total Ether at stake, using a Datalog-based analysis over EVM traces.

ABSTRACT

In recent years, we have seen a great deal of both academic and practical interest in the topic of vulnerabilities in smart contracts, particularly those developed for the Ethereum blockchain. While most of the work has focused on detecting *vulnerable* contracts, in this paper, we focus on finding how many of these vulnerable contracts have actually been *exploited*. We survey the 23,327 vulnerable contracts reported by six recent academic projects and find that, despite the amounts at stake, only 1.98% of them have been exploited since deployment. This corresponds to at most 8,487 ETH (~1.7 million USD), or only 0.27% of the 3 million ETH (600 million USD) at stake. We explain these results by demonstrating that the funds are very concentrated in a small number of contracts which are *not exploitable* in practice.

Motivation & Objective

• Quantify how many reported vulnerable smart contracts have actually been exploited in the wild.
• Develop a scalable, formal method to detect exploits using EVM execution traces.
• Integrate data from multiple academic vulnerability datasets to assess the real risk of the affected Ether.

Proposed method

• Represent six vulnerability types as Datalog queries over Ethereum blockchain state and EVM traces.
• Collect and harmonize execution traces from over 20 million transactions to identify exploits.
• Replay transactions via EVM debugging to obtain bytecode-level execution traces for analysis.
• Encode traces into Datalog facts and apply rules to detect exploitable patterns for each vulnerability.
• Compute aggregate metrics (contracts, Ether at stake, exploited Ether) and analyze high-value contracts.

Experimental results

Research questions

• RQ1 What fraction of contracts labeled vulnerable by academic tools have actually been Exploited on Ethereum mainnet?
• RQ2 What are the distributions of Ether at stake among vulnerable contracts and Exploited ones?
• RQ3 How do differences between vulnerability tools affect agreement on vulnerability identification and exploitation?
• RQ4 Can a scalable, automated approach reliably detect real exploits from execution traces across large datasets?

Key findings

• Out of 23,327 vulnerable contracts worth 3,124,433 ETH, 463 contracts may have been exploited for 8,487 ETH (0.27% of the total at stake).
• Exploitation is concentrated in a small number of high-value contracts, with many Ether held by a few contracts and many reported vulnerabilities not exploitable in practice.
• A Datalog-based framework analyzing over 20 million transactions can automatically identify potential exploits from EVM traces.
• There is notable disagreement among tools on vulnerability detection, highlighting challenges in static analysis for EVM security.
• The study emphasizes that publicized exploits (e.g., TheDAO, Parity Wallet) may overstate the practical exploitation rate relative to the total exposure.