[论文解读] GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis

Yuqiang Sun, Daoyuan Wu+6 more|arXiv (Cornell University)|Aug 7, 2023

Software Engineering Research参考文献 36被引用 14

一句话总结

GPTScan 将基于GPT的代码理解与静态分析相结合，以检测Solidity智能合约中的逻辑漏洞，在多个数据集上实现高精度和高召回，同时通过静态确认降低误报。

ABSTRACT

Smart contracts are prone to various vulnerabilities, leading to substantial financial losses over time. Current analysis tools mainly target vulnerabilities with fixed control or data-flow patterns, such as re-entrancy and integer overflow. However, a recent study on Web3 security bugs revealed that about 80% of these bugs cannot be audited by existing tools due to the lack of domain-specific property description and checking. Given recent advances in Large Language Models (LLMs), it is worth exploring how Generative Pre-training Transformer (GPT) could aid in detecting logicc vulnerabilities. In this paper, we propose GPTScan, the first tool combining GPT with static analysis for smart contract logic vulnerability detection. Instead of relying solely on GPT to identify vulnerabilities, which can lead to high false positives and is limited by GPT's pre-trained knowledge, we utilize GPT as a versatile code understanding tool. By breaking down each logic vulnerability type into scenarios and properties, GPTScan matches candidate vulnerabilities with GPT. To enhance accuracy, GPTScan further instructs GPT to intelligently recognize key variables and statements, which are then validated by static confirmation. Evaluation on diverse datasets with around 400 contract projects and 3K Solidity files shows that GPTScan achieves high precision (over 90%) for token contracts and acceptable precision (57.14%) for large projects like Web3Bugs. It effectively detects ground-truth logic vulnerabilities with a recall of over 70%, including 9 new vulnerabilities missed by human auditors. GPTScan is fast and cost-effective, taking an average of 14.39 seconds and 0.01 USD to scan per thousand lines of Solidity code. Moreover, static confirmation helps GPTScan reduce two-thirds of false positives.

研究动机与目标

• 解决大多数漏洞是逻辑性而非通过模式驱动工具捕获的差距。
• 利用 GPT 作为代码理解工具来识别代码级的漏洞情景和性质。
• 将基于 GPT 的匹配与静态确认结合，验证候选漏洞。
• 在各种真实世界数据集上评估 GPTScan 以评估精确度、召回率和成本效益。

提出的方法

• 将逻辑漏洞类型分解为用于 GPT 匹配的代码级场景和属性。
• 应用多维筛选在 GPT 分析之前缩小候选函数范围。
• 使用 GPT-3.5-turbo 结合场景-再属性匹配提示，识别可能的易漏洞函数。
• 提示 GPT 提取关键变量和语句以进行静态确认。
• 用四个静态分析模块验证 GPT 识别的候选项：数据流跟踪、值比较检查、顺序检查和函数调用参数检查。

实验结果

研究问题

• RQ1 以作为通用代码理解工具使用的 GPT，结合静态分析时，是否能有效识别逻辑漏洞？
• RQ2 如何通过多维筛选在保持检测准确性的同时降低大型 Solidity 项目中的 GPT 成本？
• RQ3 静态确认对减少 GPT 假阳性、提升精确度有何影响？
• RQ4 在代币合约与大型多文件项目上，GPTScan 在精确度和召回方面有何表现？

主要发现

• GPTScan 在代币合约（Top200 和 DefiHacks 数据集）上实现了高精度（超过 90%）。
• GPTScan 在像 Web3Bugs 这样的大型项目上实现了可接受的精度（57.14%）。
• 真实世界的逻辑漏洞在召回率超过 70% 的情况下被检测到，其中包括 9 个未被人工审计报告的新漏洞。
• 平均扫描时间为 14.39 秒，成本为每千行 Solidity 代码 0.01 美元；较大型项目成本更高且扫描更慢（约 0.018 美元与 ~20 秒）。
• 静态确认在 Web3Bugs 数据集中减少了原始假阳性的 65.84%。