[Paper Review] FLAG: Finding Line Anomalies (in code) with Generative AI

Baleegh Ahmad, Benjamin Tan+2 more|arXiv (Cornell University)|Jun 22, 2023

Software Engineering ResearchComputer Science3 citations

TL;DR

FLAG is a novel, language-agnostic framework that uses large language models (LLMs) to detect code anomalies by comparing each line of source code with LLM-generated alternatives. By identifying discrepancies—especially those with low similarity, low LLM confidence, or poor comment alignment—FLAG reduces the manual review search space to 12–17% of code while detecting 101 out of 121 known bugs across C, Python, and Verilog.

ABSTRACT

Code contains security and functional bugs. The process of identifying and localizing them is difficult and relies on human labor. In this work, we present a novel approach (FLAG) to assist human debuggers. FLAG is based on the lexical capabilities of generative AI, specifically, Large Language Models (LLMs). Here, we input a code file then extract and regenerate each line within that file for self-comparison. By comparing the original code with an LLM-generated alternative, we can flag notable differences as anomalies for further inspection, with features such as distance from comments and LLM confidence also aiding this classification. This reduces the inspection search space for the designer. Unlike other automated approaches in this area, FLAG is language-agnostic, can work on incomplete (and even non-compiling) code and requires no creation of security properties, functional tests or definition of rules. In this work, we explore the features that help LLMs in this classification and evaluate the performance of FLAG on known bugs. We use 121 benchmarks across C, Python and Verilog; with each benchmark containing a known security or functional weakness. We conduct the experiments using two state of the art LLMs in OpenAI's code-davinci-002 and gpt-3.5-turbo, but our approach may be used by other models. FLAG can identify 101 of the defects and helps reduce the search space to 12-17% of source code.

Motivation & Objective

• Address the challenge of identifying security and functional bugs in code during development, especially when programs are incomplete or non-compiling.
• Reduce the manual effort required for code review by narrowing the search space for potential defects.
• Develop a language-agnostic, rule-free approach to bug detection that relies on LLMs and developer intent encoded in code and comments.
• Explore the effectiveness of LLM-generated code comparisons as a novel mechanism for anomaly detection without requiring formal specifications or test cases.

Proposed method

• For each line in a source file, input the line along with its surrounding context (including comments) into a pre-trained LLM to generate an alternative line.
• Compare the original line with the LLM-generated line using multiple similarity metrics, including Levenshtein distance and LLM log probability (confidence).
• Classify a line as anomalous if it deviates significantly from the LLM’s expected continuation, especially when the LLM is uncertain (low log probability) or the line is distant from comments.
• Apply two classification criteria: C1 (based on distance and confidence) and C2 (which further filters out low-confidence suggestions), to balance detection and false positive rates.
• Evaluate the method across 121 benchmarks in C, Python, and Verilog using two state-of-the-art LLMs: code-davinci-002 and gpt-3.5-turbo.
• Use features such as comment proximity, Levenshtein distance, and LLM confidence to improve anomaly classification, with a focus on minimizing false positives without sacrificing true positive detection.

Experimental results

Research questions

• RQ1 Can LLM-generated code continuations effectively flag lines that deviate from developer intent, indicating potential bugs?
• RQ2 How effective is Levenshtein distance between original and generated code in identifying anomalous lines?
• RQ3 To what extent do comment proximity and LLM confidence improve the accuracy of anomaly detection?
• RQ4 How does FLAG perform across different programming languages (C, Python, Verilog) and LLMs (code-davinci-002, gpt-3.5-turbo)?
• RQ5 Can the framework reduce the manual code review search space while maintaining high detection rates for known security and functional bugs?

Key findings

• FLAG successfully detected 101 out of 121 known security and functional bugs across C, Python, and Verilog benchmarks using two state-of-the-art LLMs.
• On average, FLAG reduced the manual code review search space to 12–17% of the original source code, significantly improving developer efficiency.
• gpt-3.5-turbo showed higher detection capability than code-davinci-002 but also had a higher false positive rate (FPR of 0.172 vs. 0.121).
• Levenshtein distance between original and generated code was the most dominant feature for classifying anomalies, followed by comment proximity and LLM confidence.
• The framework performed best on C code and worst on Python, with functional bugs being detected more reliably than security-related bugs.
• Applying the C2 filtering criterion reduced false positives by eliminating low-confidence LLM suggestions, though it also reduced detection by an average of 3.75 defects.