[Paper Review] A First Look at GPT Apps: Landscape and Vulnerability

Zejun Zhang, Li Zhang+4 more|arXiv (Cornell University)|Feb 23, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This paper presents the first large-scale analysis of GPT application stores, introducing a TriLevel GPT Reversing (T-GR) framework to extract system prompts and internals from GPTs. It reveals that nearly 90% of system prompts are publicly accessible, leading to widespread plagiarism, with over 700 GPTs exhibiting semantically similar or duplicated content, undermining trust and quality in the ecosystem.

ABSTRACT

Following OpenAI's introduction of GPTs, a surge in GPT apps has led to the launch of dedicated LLM app stores. Nevertheless, given its debut, there is a lack of sufficient understanding of this new ecosystem. To fill this gap, this paper presents a first comprehensive longitudinal (5-month) study of the evolution, landscape, and vulnerability of the emerging LLM app ecosystem, focusing on two GPT app stores: extit{GPTStore.AI} and the official extit{OpenAI GPT Store}. Specifically, we develop two automated tools and a TriLevel configuration extraction strategy to efficiently gather metadata (\ie names, creators, descriptions, \etc) and user feedback for all GPT apps across these two stores, as well as configurations (\ie system prompts, knowledge files, and APIs) for the top 10,000 popular apps. Our extensive analysis reveals: (1) the user enthusiasm for GPT apps consistently rises, whereas creator interest plateaus within three months of GPTs' launch; (2) nearly 90\% system prompts can be easily accessed due to widespread failure to secure GPT app configurations, leading to considerable plagiarism and duplication among apps. Our findings highlight the necessity of enhancing the LLM app ecosystem by the app stores, creators, and users.

Motivation & Objective

• To map the landscape of GPT applications in official and unofficial GPT stores for the first time.
• To investigate vulnerabilities in GPT internals, particularly system prompts, and assess their exposure to reverse engineering.
• To analyze the prevalence of plagiarism and duplication among GPTs in the wild.
• To develop automated tools for web scraping and programmatic interaction with GPTs to enable large-scale analysis.
• To propose actionable recommendations for GPT creators and stores to improve security and reduce intellectual property leakage.

Proposed method

• Developed an automated web scraping tool to collect metadata from GPTStore.AI and the official OpenAI GPT Store over a two-month period.
• Built a programmatic interaction tool to enable controlled, repeated queries to GPTs for reverse engineering purposes.
• Proposed a novel TriLevel GPT Reversing (T-GR) strategy: Level I uses direct queries, Level II retries with refined prompts, and Level III involves cross-lingual GPT replication to recover missing internals.
• Employed BERT-based embedding models (bert-base-multilingual-uncased) to compute semantic similarity between GPT descriptions using cosine similarity.
• Used keyword-based search bots to identify duplicate and similar GPTs by name and description, with similarity threshold set at 0.99 cosine similarity.
• Collected and analyzed conversation count differences between similar GPTs to assess user confusion and potential manipulation risks.

Experimental results

Research questions

• RQ1 What is the current landscape of GPT applications in terms of growth, popularity, and creator activity across major GPT stores?
• RQ2 To what extent are GPT internals, particularly system prompts, exposed to public access and reverse engineering?
• RQ3 How prevalent is plagiarism and duplication among GPTs, and what impact does this have on user trust and selection?
• RQ4 What are the limitations of current GPT store interfaces in terms of navigation, categorization, and user experience?
• RQ5 Can automated tools effectively extract GPT internals and identify semantically similar or duplicated GPTs at scale?

Key findings

• Nearly 90% of the system prompts of the top 10,000 GPTs analyzed are directly accessible through simple queries, indicating severe protection failures.
• More than 700 GPTs (9% of 7,706 analyzed) have at least one semantically similar or duplicated counterpart, with 135 having identical names and descriptions.
• Over 75% of GPTs with semantically similar counterparts have conversation counts differing by less than 100, increasing user confusion and risk of selecting malicious or low-quality GPTs.
• Despite 40% of GPTs exposing API details, external API access remains difficult due to authentication requirements, limiting full reverse engineering of data pipelines.
• The current reliance on conversation count as a popularity metric is fragile and susceptible to manipulation, undermining its reliability for user decision-making.
• The study identifies a critical gap in GPT store design: lack of robust user guidance, categorization, and quality control mechanisms to mitigate the impact of duplication and plagiarism.