[Paper Review] DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles

Nicole Sultanum, Arjun Srinivasan|arXiv (Cornell University)|Aug 8, 2023

Scientific Computing and Data Management4 citations

TL;DR

This paper introduces DataTales, a prototype system that leverages large language models (LLMs) to generate narrative content for data-driven articles by linking textual stories to interactive visualizations. Through a qualitative study with 11 data professionals, the authors identify key affordances—such as chart-based prompting, annotation guidance, and story summarization—while highlighting challenges like hallucinations, bias, and the need for external validation, offering design insights for LLM-integrated data storytelling tools.

ABSTRACT

Authoring data-driven articles is a complex process requiring authors to not only analyze data for insights but also craft a cohesive narrative that effectively communicates the insights. Text generation capabilities of contemporary large language models (LLMs) present an opportunity to assist the authoring of data-driven articles and expedite the writing process. In this work, we investigate the feasibility and perceived value of leveraging LLMs to support authors of data-driven articles. We designed a prototype system, DataTales, that leverages a LLM to generate textual narratives accompanying a given chart. Using DataTales as a design probe, we conducted a qualitative study with 11 professionals to evaluate the concept, from which we distilled affordances and opportunities to further integrate LLMs as valuable data-driven article authoring assistants.

Motivation & Objective

• To investigate the feasibility and perceived value of using large language models (LLMs) to assist in authoring data-driven articles.
• To explore how chart interaction can serve as a more intuitive alternative to direct prompting for conveying narrative intent to LLMs.
• To identify design affordances and challenges in integrating LLMs as collaborative writing assistants in data storytelling workflows.
• To understand how LLM-generated narratives can be improved through user annotations, summarization, and external validation mechanisms.
• To address ethical and reliability concerns such as hallucinations, confirmation bias, and data literacy gaps in LLM-assisted content generation.

Proposed method

• The authors developed DataTales, a web-based prototype that integrates a GPT-3.5-turbo LLM with D3.js-rendered charts and React-based UI components.
• The system allows users to generate narrative text based on a chart and optional annotations, with real-time interactivity linking text to visual elements.
• Users can edit generated text, regenerate stories, or select from multiple story variants, enabling iterative refinement of narrative content.
• The system supports user annotations to guide focus and context, such as highlighting data points or drawing trend lines to signal importance.
• A qualitative study with 11 data professionals was conducted using DataTales as a design probe to elicit feedback on usability, usefulness, and limitations.
• Findings were synthesized into 13 takeaways (T1–T13) addressing features like summarization, external data recommendations, and error mitigation through citations and feedback loops.

Figure 1: \datatales workflow overview. Given a chart and an optional set of annotations, the system generates textual narratives that are interactively linked to the chart and can be further edited by authors.

Experimental results

Research questions

• RQ1 How can LLMs be effectively leveraged to support the authoring of data-driven articles in a way that aligns with user narrative intent?
• RQ2 What are the perceived benefits and challenges of using LLMs for generating narratives linked to data visualizations in professional settings?
• RQ3 How do chart interactions and user annotations influence the quality and relevance of LLM-generated stories?
• RQ4 What design features are needed to mitigate risks such as hallucinations, bias, and misinformation in LLM-generated content?
• RQ5 How can LLMs be integrated with existing data literacy and fact-checking practices to support responsible authoring?

Key findings

• Participants found the concept of LLM-assisted data storytelling valuable, especially when using chart interaction and annotations to guide narrative generation.
• The system was perceived as helpful for idea generation and drafting, particularly in reducing cognitive load during early-stage story development.
• Many participants reported that generated stories were often too verbose, leading to difficulties in reading and reviewing, prompting calls for concise, bullet-point summaries.
• A significant number of hallucinations and factual inaccuracies were observed, including incorrect reasoning and labeling, underscoring the need for external validation and source citations.
• Participants emphasized the importance of distinguishing between data-derived statements and LLM-invented content, suggesting visual cues like color-coding to signal provenance.
• There was strong interest in features that recommend external datasets, comparisons, and follow-up questions to extend exploratory analysis and support deeper insight discovery.

Figure 2: Example of an incorrect statement generated by the LLM (contrary to the text, the chart shows that Florida does not have a higher number of people over the age of 80 compared to California). The text $\rightarrow$ chart linking feature helps verify the statement and identify the erroneous