[Paper Review] Proxona: Supporting Creators' Sensemaking and Ideation with LLM-Powered Audience Personas

Yoonseo Choi, Eun Jeong Kang+3 more|arXiv (Cornell University)|Aug 20, 2024

Persona Design and Applications4 citations

TL;DR

Proxona is a large language model (LLM)-powered system that generates data-driven audience personas from creators' YouTube comments to enhance audience understanding and content ideation. By extracting dimensions and values from comments and clustering them into synthetic personas, Proxona enables creators to converse with these personas to gain actionable insights, resulting in more informed, audience-centered content decisions—validated through technical evaluations and a user study with 11 creators.

ABSTRACT

A content creator's success depends on understanding their audience, but existing tools fail to provide in-depth insights and actionable feedback necessary for effectively targeting their audience. We present Proxona, an LLM-powered system that transforms static audience comments into interactive, multi-dimensional personas, allowing creators to engage with them to gain insights, gather simulated feedback, and refine content. Proxona distills audience traits from comments, into dimensions (categories) and values (attributes), then clusters them into interactive personas representing audience segments. Technical evaluations show that Proxona generates diverse dimensions and values, enabling the creation of personas that sufficiently reflect the audience and support data grounded conversation. User evaluation with 11 creators confirmed that Proxona helped creators discover hidden audiences, gain persona-informed insights on early-stage content, and allowed them to confidently employ strategies when iteratively creating storylines. Proxona introduces a novel creator-audience interaction framework and fosters a persona-driven, co-creative process.

Motivation & Objective

• To address creators' difficulty in understanding audience motivations and preferences beyond basic demographics.
• To bridge the gap between quantitative platform analytics and qualitative audience feedback by transforming comments into actionable audience insights.
• To support creators in the early content planning phase by enabling natural language interaction with synthetic audience personas.
• To reduce reliance on trend-chasing by helping creators co-create content based on diverse audience perspectives.
• To provide a scalable, AI-augmented method for integrating audience feedback into the creative process before content production.

Proposed method

• Using LLMs to analyze YouTube comments and extract latent audience characteristics along two axes: dimensions (e.g., interests, expertise level) and values (specific attributes within dimensions).
• Applying a pipeline that filters comments by length, then uses few-shot prompting to classify comments into dimensions and extract values, minimizing hallucination risks.
• Clustering similar combinations of dimensions and values into distinct, representative audience personas using embedding-based similarity methods.
• Presenting personas as interactive, fictionalized audience profiles with contextualized traits, motivations, and experiences derived from real comments.
• Enabling creators to engage in natural language conversations with personas to solicit feedback, test content ideas, and refine storylines.
• Validating persona quality through technical evaluation (precision, distinctiveness, hallucination rate) and user study with 11 YouTube creators.

Figure 1. Exploration page where creators can explore their audience personas, associated dimensions and values , and proceeding conversations by asking natural language questions to personas. Creators can add a new persona by configuring dimensions and values , if necessary.

Experimental results

Research questions

• RQ1 How can LLMs be used to extract meaningful audience dimensions and values from unstructured video comments?
• RQ2 To what extent do LLM-generated personas reflect real audience diversity and preferences, and how can hallucination be minimized?
• RQ3 How do creators perceive and utilize audience personas in the context of content planning and ideation?
• RQ4 In what ways can persona-based interaction improve the quality and audience alignment of content strategies?
• RQ5 How can such a system be integrated into the real-world creative workflow of content creators?

Key findings

• Technical evaluation showed that Proxona’s pipeline generated relevant, distinct, and low-hallucination personas, with high precision in dimension and value extraction.
• User study with 11 creators demonstrated that Proxona enabled them to gain new insights about audience preferences and motivations not previously accessible through analytics alone.
• Creators reported increased confidence in content decisions and expressed that persona interactions helped them ideate more diverse and audience-aligned content.
• Participants reported that interacting with personas supported a more collaborative and iterative content development process, resembling co-creation with real audience members.
• The system was particularly effective in helping creators move beyond trend-chasing and create content that resonated with specific audience segments.
• Limitations were identified, including bias from length-based comment filtering and reduced utility for early-stage creators with minimal comment data.

Figure 2. Creation page where creators write a video storyline, proceed conversation about their plot, and request feedback on their written content.