[Paper Review] Datastringer: easy dataset monitoring for journalists

Matt Shearer, Basile Simon+1 more|arXiv (Cornell University)|Feb 8, 2024

Big Data Technologies and ApplicationsDecision Sciences3 citations

TL;DR

Datastringer is a journalist-focused software tool that enables non-technical users to monitor constantly updated datasets by defining custom alert criteria, using a JavaScript-based, reusable architecture to ensure scalability and accessibility. The system delivers timely story alerts when predefined data conditions are met, serving as a proof of concept for accessible data monitoring in investigative journalism.

ABSTRACT

We created a software enabling journalists to define a set of criteria they would like to see applied regularly to a constantly-updated dataset, sending them an alert when these criteria are met, thus signaling them that there may be a story to write. The main challenges were to keep the product scalable and powerful, while making sure that it could be used by journalists who would not possess all the technical knowledge to exploit it fully. In order to do so, we had to choose Javascript as our main language, as well as designing the code in such a way that it would allow re-usability and further improvements. This project is a proof of concept being tested in a real-life environment, and will be developed towards more and more accessibility.

Motivation & Objective

• Address the challenge of enabling non-technical journalists to monitor dynamic datasets for story opportunities.
• Develop a scalable and extensible system that requires minimal technical expertise to use.
• Create a proof-of-concept tool that integrates real-time data checks with actionable alerting for newsrooms.
• Ensure reusability and future extensibility through modular, maintainable code design in JavaScript.
• Bridge the gap between data availability and journalistic storytelling by automating detection of noteworthy data changes.

Proposed method

• Implement a rule-based monitoring engine that evaluates datasets against user-defined criteria.
• Use JavaScript as the primary development language to ensure broad compatibility and ease of deployment.
• Design a modular codebase to support reusability and future enhancements by third parties.
• Integrate periodic data polling or streaming to track dataset updates in near real time.
• Trigger alerts when predefined conditions—such as threshold breaches or anomalies—are detected.
• Provide a simple, intuitive interface for journalists to input and manage monitoring rules without coding knowledge.

Experimental results

Research questions

• RQ1 How can a data monitoring system be designed to be both powerful and accessible to non-technical journalists?
• RQ2 What architectural choices enable scalability and reusability while minimizing technical barriers?
• RQ3 Can a rule-based alerting system effectively detect newsworthiness in continuously updated datasets?
• RQ4 How can real-time data evaluation be balanced with performance and maintainability in a journalistic context?
• RQ5 To what extent can a non-technical user define and manage complex data monitoring rules without programming expertise?

Key findings

• The system successfully enables journalists to define and deploy data monitoring rules without requiring programming knowledge.
• The use of JavaScript and a modular architecture ensures the tool is scalable and maintainable for future development.
• The tool functions as a working proof of concept in a real-world environment, demonstrating feasibility for newsroom integration.
• Alerts are triggered reliably when user-defined data conditions are met, signaling potential news stories.
• The design prioritizes usability and extensibility, supporting future enhancements and broader adoption.
• The project establishes a foundation for accessible data monitoring tools tailored to investigative journalism workflows.