[论文解读] Empirical Vulnerability Analysis of Automated Smart Contracts Security Testing on Blockchains

Reza M. Parizi, Ali Dehghantanha+2 more|arXiv (Cornell University)|Sep 7, 2018

Blockchain Technology Applications and Security参考文献 35被引用 72

一句话总结

本文通过实证评估四个开源自动化安全测试工具在以太坊智能合约上的性能（Oyente、Mythril、Securify、SmartCheck），以比较它们在漏洞检测有效性和准确性方面的表现，数据集为十份易受攻击的 Solidity 合约。

ABSTRACT

The emerging blockchain technology supports decentralized computing paradigm shift and is a rapidly approaching phenomenon. While blockchain is thought primarily as the basis of Bitcoin, its application has grown far beyond cryptocurrencies due to the introduction of smart contracts. Smart contracts are self-enforcing pieces of software, which reside and run over a hosting blockchain. Using blockchain-based smart contracts for secure and transparent management to govern interactions (authentication, connection, and transaction) in Internet-enabled environments, mostly IoT, is a niche area of research and practice. However, writing trustworthy and safe smart contracts can be tremendously challenging because of the complicated semantics of underlying domain-specific languages and its testability. There have been high-profile incidents that indicate blockchain smart contracts could contain various code-security vulnerabilities, instigating financial harms. When it involves security of smart contracts, developers embracing the ability to write the contracts should be capable of testing their code, for diagnosing security vulnerabilities, before deploying them to the immutable environments on blockchains. However, there are only a handful of security testing tools for smart contracts. This implies that the existing research on automatic smart contracts security testing is not adequate and remains in a very stage of infancy. With a specific goal to more readily realize the application of blockchain smart contracts in security and privacy, we should first understand their vulnerabilities before widespread implementation. Accordingly, the goal of this paper is to carry out a far-reaching experimental assessment of current static smart contracts security testing tools, for the most widely used blockchain, the Ethereum and its domain-specific programming language, Solidity to provide the first...

研究动机与目标

• 评估在 Ethereum/Solidity 上四种开源静态智能合约安全测试工具的有效性。
• 衡量并比较这些工具在检测真实漏洞与避免误报方面的准确性。
• 提供实证知识以指导在区块链部署前对智能合约进行安全测试。
• 分析各工具在漏洞检测与准确性之间的权衡，以指导实践。

提出的方法

• 选择四个 FOSS 工具：Oyente、Mythril、Securify、SmartCheck。
• 使用十份公开可得的易受攻击的 Solidity 合约作为测试集。
• 对每份合约对每个工具运行并记录每种情形的 TP、FP、TN、FN。
• 使用公式 Eff_j = (sum_i TP_i/(TP_i+FN_i)/n) * 100 计算每个工具的有效性（召回率）。
• 计算准确性（Youden 类似指数）Acc_j = (Eff_j + (sum_i TN_i/(TN_i+FP_i)/n)*100) - 1。
• 应用随机区组设计：每份合约由所有四个工具进行测试；使用 ANOVA 和 LSD 事后检验进行分析。
• 假设检验采用 5% 的显著性水平。

实验结果

研究问题

• RQ1 RQ1：在漏洞检测方面，自动化智能合约安全测试工具的有效性有多高，哪一个是最有效的？
• RQ2 RQ2：这些工具在检测真实漏洞并避免误报方面的准确性分数是多少？
• RQ3 这四种工具在有效性和准确性方面的统计比较如何？
• RQ4 在有效性和准确性方面，工具之间的成对差异有多大？

主要发现

• SmartCheck 在四个工具中显示出最高的漏洞检测有效性。
• ANOVA 表明工具之间在漏洞检测有效性上存在统计显著差异（p = 0.0003）。
• LSD 事后检验显示 SmartCheck 在有效性方面与其他所有工具存在显著差异（p < 0.05）。
• Mythril 和 SmartCheck 通常具有更高的准确性得分，其中 Mythril 常常在各合约上达到最高准确性。
• 准确性 ANOVA 显示工具之间存在显著差异（p = 0.0002）；事后检验显示 Mythril 和 SmartCheck 在准确性方面与其他工具存在显著差异。
• 总体上观察到一种取舍：SmartCheck 在检测有效性方面表现出色，而 Mythril 和 SmartCheck 往往具有更高的准确性，提示在实际使用中工具的不同优势。