[논문 리뷰] Empirical Vulnerability Analysis of Automated Smart Contracts Security Testing on Blockchains

Reza M. Parizi, Ali Dehghantanha외 2명|arXiv (Cornell University)|2018. 09. 07.

Blockchain Technology Applications and Security참고 문헌 35인용 수 72

한 줄 요약

이 논문은 이더리움 스마트 컨트랙트에 대한 네 가지 오픈 소스 자동 보안 테스트 도구(Oyente, Mythril, Securify, SmartCheck)를 실증적으로 평가하여 ten vulnerable Solidity contracts를 사용해 그 취약점 탐지 효과와 정확성을 비교한다.

ABSTRACT

The emerging blockchain technology supports decentralized computing paradigm shift and is a rapidly approaching phenomenon. While blockchain is thought primarily as the basis of Bitcoin, its application has grown far beyond cryptocurrencies due to the introduction of smart contracts. Smart contracts are self-enforcing pieces of software, which reside and run over a hosting blockchain. Using blockchain-based smart contracts for secure and transparent management to govern interactions (authentication, connection, and transaction) in Internet-enabled environments, mostly IoT, is a niche area of research and practice. However, writing trustworthy and safe smart contracts can be tremendously challenging because of the complicated semantics of underlying domain-specific languages and its testability. There have been high-profile incidents that indicate blockchain smart contracts could contain various code-security vulnerabilities, instigating financial harms. When it involves security of smart contracts, developers embracing the ability to write the contracts should be capable of testing their code, for diagnosing security vulnerabilities, before deploying them to the immutable environments on blockchains. However, there are only a handful of security testing tools for smart contracts. This implies that the existing research on automatic smart contracts security testing is not adequate and remains in a very stage of infancy. With a specific goal to more readily realize the application of blockchain smart contracts in security and privacy, we should first understand their vulnerabilities before widespread implementation. Accordingly, the goal of this paper is to carry out a far-reaching experimental assessment of current static smart contracts security testing tools, for the most widely used blockchain, the Ethereum and its domain-specific programming language, Solidity to provide the first...

연구 동기 및 목표

• 이더리움/Solidity에서 네 가지 오픈 소스 정적 스마트 컨트랙트 보안 테스트 도구의 효과를 평가한다.
• 실제 취약점을 탐지하고 잘못된 경고를 피하는 데 있어 이 도구들의 정확성을 측정하고 비교한다.
• 블록체인에 배포하기 전에 안전한 스마트 컨트랙트 테스트를 이끄는 실증적 지식을 제공한다.
• 도구들 간의 취약점 탐지와 정확성 사이의 트레이드오프를 분석하여 실무에 정보를 제공한다.

제안 방법

• 4개의 FOSS 도구를 선택한다: Oyente, Mythril, Securify, SmartCheck.
• 테스트 세트로 공개적으로 사용 가능한 ten vulnerable Solidity contracts를 사용한다.
• 각 계약마다 각 도구를 실행하고 케이스별 TP, FP, TN, FN을 기록한다.
• Compute effectiveness (recall) per tool using the formula Eff_j = (sum_i TP_i/(TP_i+FN_i)/n) * 100.
• Compute accuracy (Youden-like index) Acc_j = (Eff_j + (sum_i TN_i/(TN_i+FP_i)/n)*100) - 1.
• 무작위 블록 설계를 적용한다: 각 계약은 네 도구 모두로 테스트되며; ANOVA 및 LSD 사후검정을 사용해 분석한다.
• 가설 검정에 5% 유의수준을 사용한다.

실험 결과

연구 질문

• RQ1 RQ1: How effective are the automated smart contract security testing tools at vulnerability detection, and which is most effective?
• RQ2 RQ2: What are the accuracy scores of the tools in detecting true vulnerabilities and avoiding false alarms?
• RQ3 How do tool effectiveness and accuracy compare statistically across the four tools?
• RQ4 What are the pairwise differences between tools in terms of effectiveness and accuracy?

주요 결과

• SmartCheck shows the highest vulnerability detection effectiveness among the four tools.
• ANOVA indicates a statistically significant difference in vulnerability detection effectiveness across tools (p = 0.0003).
• LSD post-hoc tests show SmartCheck differs significantly from all other tools in effectiveness (p < 0.05).
• Mythril and SmartCheck generally yield higher accuracy scores, with Mythril often achieving the highest accuracy across contracts.
• ANOVA for accuracy shows significant differences among tools (p = 0.0002); post-hoc tests reveal Mythril and SmartCheck differ significantly from others in accuracy.
• Overall, a trade-off is observed: SmartCheck excels in detection effectiveness, while Mythril and SmartCheck tend to have higher accuracy, suggesting different tool strengths for practical use.