[Paper Review] Software Vulnerability and Functionality Assessment using LLMs

Rasmus Ingemann Tuffveson Jensen, Vali Tawosi+1 more|arXiv (Cornell University)|Mar 13, 2024

Software Reliability and Analysis Research4 citations

TL;DR

This paper investigates the use of Large Language Models (LLMs) for automated code review, focusing on vulnerability detection and functional correctness validation. Using zero-shot and chain-of-thought prompting on HumanEval, MBPP, and SecurityEval datasets, proprietary LLMs like GPT-4 achieve over 95% accuracy in vulnerability detection and 88% in functionality validation, with 36.7% of generated vulnerability descriptions matching real CWE entries.

ABSTRACT

While code review is central to the software development process, it can be tedious and expensive to carry out. In this paper, we investigate whether and how Large Language Models (LLMs) can aid with code reviews. Our investigation focuses on two tasks that we argue are fundamental to good reviews: (i) flagging code with security vulnerabilities and (ii) performing software functionality validation, i.e., ensuring that code meets its intended functionality. To test performance on both tasks, we use zero-shot and chain-of-thought prompting to obtain final ``approve or reject'' recommendations. As data, we employ seminal code generation datasets (HumanEval and MBPP) along with expert-written code snippets with security vulnerabilities from the Common Weakness Enumeration (CWE). Our experiments consider a mixture of three proprietary models from OpenAI and smaller open-source LLMs. We find that the former outperforms the latter by a large margin. Motivated by promising results, we finally ask our models to provide detailed descriptions of security vulnerabilities. Results show that 36.7% of LLM-generated descriptions can be associated with true CWE vulnerabilities.

Motivation & Objective

• To evaluate whether LLMs can effectively flag security vulnerabilities in code without execution.
• To assess if LLMs can validate whether code meets its intended functionality based on docstrings.
• To investigate whether LLMs can perform both vulnerability detection and functionality validation simultaneously.
• To examine the quality of LLM-generated descriptions of security vulnerabilities in relation to real CWE entries.
• To compare performance across proprietary and open-source LLMs in code review tasks.

Proposed method

• Employed three code datasets: HumanEval (148 functions), MBPP (476 functions), and SecurityEval (36 vulnerable snippets), all with single functions and docstrings.
• Used zero-shot and chain-of-thought prompting to elicit 'approve or reject' decisions for vulnerability and functionality tasks.
• Evaluated a mix of proprietary models (GPT-3.5-turbo, GPT-4, text-davinci-003) and open-source models (Llama-2, Dolly-v2, Falcon, CodeLlama).
• Applied input perturbations and multiple runs to mitigate stochasticity and improve robustness of results.
• Measured performance using accuracy, F1 score, and matching rate of LLM-generated vulnerability descriptions to actual CWE identifiers.
• Constructed a combined dataset for evaluation, ensuring consistent function and docstring structure across all sources.

Experimental results

Research questions

• RQ1 RQ1: Can LLMs flag code with security vulnerabilities?
• RQ2 RQ2: Can LLMs perform software functionality validation without execution?
• RQ3 RQ3: Can LLMs simultaneously flag vulnerabilities and validate functionality?
• RQ4 RQ4: Can LLMs provide meaningful feedback on security vulnerabilities?

Key findings

• The proprietary model text-davinci-003 achieved 95.6% accuracy and 37.9% F1 score in detecting security vulnerabilities.
• GPT-4 achieved 88.7% accuracy and 88.2% F1 score in functionality validation, significantly outperforming open-source models.
• Chain-of-thought prompting improved GPT-4’s accuracy in combined tasks from 80.8% to 87.2% and F1 from 76.6% to 85.7%.
• Open-source models performed poorly, with F1 scores near or below random baseline levels across all tasks.
• GPT-4 generated vulnerability descriptions that matched real CWE entries 36.7% of the time, indicating meaningful semantic alignment.
• Model performance generally increased with model size across GPT, Llama, and Dolly families, with proprietary models significantly outperforming open-source ones.