[Paper Review] Evaluating AIGC Detectors on Code Content

Jian Wang, Shangqing Liu+2 more|arXiv (Cornell University)|Apr 11, 2023

Software Engineering ResearchComputer Science16 citations

TL;DR

The paper empirically evaluates six AIGC detectors (3 open-source, 3 commercial) on code-related content generated by ChatGPT and compares to natural language content, with a human study and dataset construction.

ABSTRACT

Artificial Intelligence Generated Content (AIGC) has garnered considerable attention for its impressive performance, with ChatGPT emerging as a leading AIGC model that produces high-quality responses across various applications, including software development and maintenance. Despite its potential, the misuse of ChatGPT poses significant concerns, especially in education and safetycritical domains. Numerous AIGC detectors have been developed and evaluated on natural language data. However, their performance on code-related content generated by ChatGPT remains unexplored. To fill this gap, in this paper, we present the first empirical study on evaluating existing AIGC detectors in the software domain. We created a comprehensive dataset including 492.5K samples comprising code-related content produced by ChatGPT, encompassing popular software activities like Q&A (115K), code summarization (126K), and code generation (226.5K). We evaluated six AIGC detectors, including three commercial and three open-source solutions, assessing their performance on this dataset. Additionally, we conducted a human study to understand human detection capabilities and compare them with the existing AIGC detectors. Our results indicate that AIGC detectors demonstrate lower performance on code-related data compared to natural language data. Fine-tuning can enhance detector performance, especially for content within the same domain; but generalization remains a challenge. The human evaluation reveals that detection by humans is quite challenging.

Motivation & Objective

• Assess the effectiveness of existing AIGC detectors on code-related content generated by ChatGPT.
• Compare detector performance on code content versus natural language content.
• Explore whether fine-tuning improves detector performance and generalization.
• Evaluate detector robustness to small content mutations.
• Compare human ability to detect AI-generated content with detectors.

Proposed method

• Construct two large datasets: CCD (code-related content) and NLCD (natural language content) with human- and ChatGPT-generated pairs.
• Evaluate six detectors (three open-source: GPT2-Detector, DetectGPT, RoBERTa-QA; three commercial: GPTZero, Writer, AITextClassifier) on CCD-Test and NLCD-Test.
• Fine-tune RoBERTa-QA on NLCD-Train and CCD-Train using domain-relevant subsets to assess improvements.
• Test robustness by applying code and text mutations and re-evaluating detectors.
• Conduct an online human study with 50 experienced developers to assess human detection performance.
• Use AUC as primary metric, with FPR and FNR as supplementary metrics.

Experimental results

Research questions

• RQ1 RQ1: How effective are existing detectors in detecting ChatGPT-generated code- and natural-language content?
• RQ2 RQ2: Can fine-tuning improve detector performance on code-related data?
• RQ3 RQ3: How robust are detectors when ChatGPT-generated data is slightly modified?
• RQ4 RQ4: How well can humans distinguish contents generated by ChatGPT compared to detectors?

Key findings

• Detectors show lower performance on code-related data than natural language data.
• Fine-tuning improves detector performance, but generalization across domains remains limited.
• Among detectors, various trade-offs exist between AUC, FPR, and FNR across datasets and languages.
• Robustness tests show detectors degrade under mutations; some detectors fare better than others depending on content type.
• Humans also find it challenging to detect ChatGPT-generated code content, mirroring detector difficulties.