[Paper Review] AI vs. Human -- Differentiation Analysis of Scientific Content Generation

Yongqiang Ma, Jiawei Liu+5 more|arXiv (Cornell University)|Jan 24, 2023

Topic Modeling76 citations

TL;DR

The paper builds a feature-based framework to distinguish AI-generated from human-written scientific abstracts, analyzes coherence/consistency/argument logistics, and evaluates detection methods including perplexity and a fine-tuned detector. It finds a writing-style gap and that AI-generated content often has fewer external factual inconsistencies but may harbor factual errors.

ABSTRACT

Recent neural language models have taken a significant step forward in producing remarkably controllable, fluent, and grammatical text. Although studies have found that AI-generated text is not distinguishable from human-written text for crowd-sourcing workers, there still exist errors in AI-generated text which are even subtler and harder to spot. We primarily focus on the scenario in which scientific AI writing assistant is deeply involved. First, we construct a feature description framework to distinguish between AI-generated text and human-written text from syntax, semantics, and pragmatics based on the human evaluation. Then we utilize the features, i.e., writing style, coherence, consistency, and argument logistics, from the proposed framework to analyze two types of content. Finally, we adopt several publicly available methods to investigate the gap of between AI-generated scientific text and human-written scientific text by AI-generated scientific text detection models. The results suggest that while AI has the potential to generate scientific content that is as accurate as human-written content, there is still a gap in terms of depth and overall quality. The AI-generated scientific content is more likely to contain errors in factual issues. We find that there exists a "writing style" gap between AI-generated scientific text and human-written scientific text. Based on the analysis result, we summarize a series of model-agnostic and distribution-agnostic features for detection tasks in other domains. Findings in this paper contribute to guiding the optimization of AI models to produce high-quality content and addressing related ethical and security concerns.

Motivation & Objective

• Develop a feature description framework to differentiate AI-generated and human-written scientific text across syntax, semantics, and pragmatics.
• Analyze AI vs. human scientific abstracts in CS and Bio domains using writing style, coherence, consistency, and argument logistics.
• Evaluate GPT-generated text detection methods, including feature-based and neural model approaches, with explainability.

Proposed method

• Generate abstracts using GPT-3/Text-Davinci-003 with optimized prompts containing scientific structure information.
• Construct a feature-based detection framework with four dimensions: Writing Style, Coherence, Consistency, and Argument Logistics.
• Fine-tune a GPT-2 output detector and compare with a RoBERTa/OpenAI detector baseline.
• Apply perplexity-based detection using SciBERT to set domain-specific thresholds (2.6 for abstracts, 4.6 for wiki items).
• Conduct human evaluation to assess humans' ability to identify AI-generated scientific text and analyze related factors.

Experimental results

Research questions

• RQ1 Can AI-generated scientific abstracts be reliably distinguished from human-written ones using syntax, semantics, and pragmatics features?
• RQ2 What is the contribution of writing style, coherence, consistency, and argument logistics to detection performance?
• RQ3 What gaps exist in depth/quality and factual accuracy between AI-generated and human-written scientific content?
• RQ4 How effective are perplexity and detector-based approaches in identifying AI-generated scientific text across domains?

Key findings

• There exists a significant writing-style gap (syntax) between AI-generated and human-written scientific text.
• Perplexity-based detection achieves high F1 on abstracts (94%); lower on wiki item descriptions (77%).
• Token-level and function-word features are strongly predictive in the syntax-based detector (logistic regression explaining up to 86.1% of the variation).
• AI-generated abstracts show high coherence to titles but lower internal consistency, with some factual references being incorrect or invented in case examples.
• A trained detection model outperforms humans at distinguishing AI-generated from human-written scientific text, supporting labeling AI-generated content in science.