[Paper Review] Origin Tracing and Detecting of LLMs

Linyang Li, Pengyu Wang+3 more|arXiv (Cornell University)|Apr 27, 2023

Topic Modeling6 citations

TL;DR

This paper introduces origin tracing for large language models (LLMs), proposing Sniffer—a model-wise feature-based method that detects text origins using contrastive features across LLMs. It works under both white-box and black-box settings, achieves high accuracy in tracing diverse LLMs (including distilled models like Alpaca), and provides a benchmark toolkit for future research.

ABSTRACT

The extraordinary performance of large language models (LLMs) heightens the importance of detecting whether the context is generated by an AI system. More importantly, while more and more companies and institutions release their LLMs, the origin can be hard to trace. Since LLMs are heading towards the time of AGI, similar to the origin tracing in anthropology, it is of great importance to trace the origin of LLMs. In this paper, we first raise the concern of the origin tracing of LLMs and propose an effective method to trace and detect AI-generated contexts. We introduce a novel algorithm that leverages the contrastive features between LLMs and extracts model-wise features to trace the text origins. Our proposed method works under both white-box and black-box settings therefore can be widely generalized to detect various LLMs.(e.g. can be generalized to detect GPT-3 models without the GPT-3 models). Also, our proposed method requires only limited data compared with the supervised learning methods and can be extended to trace new-coming model origins. We construct extensive experiments to examine whether we can trace the origins of given texts. We provide valuable observations based on the experimental results, such as the difficulty level of AI origin tracing, and the AI origin similarities, and call for ethical concerns of LLM providers. We are releasing all codes and data as a toolkit and benchmark for future AI origin tracing and detecting studies. \footnote{We are releasing all available resource at \url{https://github.com/OpenLMLab/}.}

Motivation & Objective

• Address the critical need to trace the origin of AI-generated text as LLMs become widespread and increasingly capable.
• Identify the limitations of existing detection methods that treat AI generation as a binary classification task without origin resolution.
• Develop a generalizable, data-efficient method to trace origins across diverse LLMs, including black-box and distilled models.
• Establish a benchmark and open-source toolkit to support reproducible research in LLM origin tracing and detection.

Proposed method

• Propose Sniffer, a detection framework that extracts model-wise contrastive features from LLM outputs to distinguish text origins.
• Use heuristic-based feature engineering to capture intrinsic model-specific discrepancies across different LLMs (e.g., GPT-2, GPT-J, LLaMA).
• Apply a lightweight linear classifier to map extracted features to specific model origins, enabling both white-box and black-box detection.
• Design a multi-level origin classification system to trace not only base models but also instruction-fine-tuned variants (e.g., Alpaca, Dolly).
• Construct SnifferBench, a diverse benchmark dataset containing texts from multiple LLMs and instruction types to evaluate origin tracing performance.
• Combine semantic and model-wise features in hybrid detection (Sniffer+RoBERTa) to improve robustness across model strengths and instruction variations.

Experimental results

Research questions

• RQ1 Can model-wise features effectively distinguish the origin of AI-generated text across diverse LLMs, even when the source models are not fully accessible?
• RQ2 How does the strength of the source LLM (e.g., GPT-2 vs. GPT-3.5) affect the difficulty of origin tracing?
• RQ3 To what extent can origin tracing detect distilled or fine-tuned models (e.g., Alpaca, Dolly) trained on outputs from stronger LLMs?
• RQ4 How do different instruction prompts influence the detectability of AI-generated text across models?
• RQ5 Can a hybrid approach combining semantic and model-wise features improve origin tracing performance compared to either method alone?

Key findings

• Sniffer achieves high accuracy in tracing origins of open-source LLMs (e.g., GPT-2, GPT-J, LLaMA) under both white-box and black-box settings.
• Texts generated by stronger LLMs (e.g., GPT-3.5) are harder to trace than those from weaker models like GPT-2, indicating increased detection difficulty with model capability.
• The alignment process (e.g., instruction tuning) has a stronger influence on generated text characteristics than the base model, as seen in Alpaca and Dolly being classified as ChatGPT-generated.
• Sniffer successfully separates GPT-J and GPT-Neo origins with high f1-score, demonstrating its ability to distinguish between closely related models.
• Supervised methods like RoBERTa-based detection fail to trace GPT-2-generated texts, highlighting the limitations of semantic-only features and the superiority of model-wise features.
• The hybrid Sniffer+RoBERTa approach outperforms both individual methods, proving that combining semantic and model-wise features enhances detection robustness.