[Paper Review] A Fourth Wave of Open Data? Exploring the Spectrum of Scenarios for Open Data and Generative AI

Hannah Chafetz, Sampriti Saxena+1 more|arXiv (Cornell University)|May 7, 2024

Big Data and Business Intelligence6 citations

TL;DR

The paper proposes a Spectrum of Scenarios framework to map how open data and generative AI intersect, outlining scenarios from data pertaining to open data readiness to open-ended exploration, and identifies five key areas to advance data quality and governance.

ABSTRACT

Since late 2022, generative AI has taken the world by storm, with widespread use of tools including ChatGPT, Gemini, and Claude. Generative AI and large language model (LLM) applications are transforming how individuals find and access data and knowledge. However, the intricate relationship between open data and generative AI, and the vast potential it holds for driving innovation in this field remain underexplored areas. This white paper seeks to unpack the relationship between open data and generative AI and explore possible components of a new Fourth Wave of Open Data: Is open data becoming AI ready? Is open data moving towards a data commons approach? Is generative AI making open data more conversational? Will generative AI improve open data quality and provenance? Towards this end, we provide a new Spectrum of Scenarios framework. This framework outlines a range of scenarios in which open data and generative AI could intersect and what is required from a data quality and provenance perspective to make open data ready for those specific scenarios. These scenarios include: pertaining, adaptation, inference and insight generation, data augmentation, and open-ended exploration. Through this process, we found that in order for data holders to embrace generative AI to improve open data access and develop greater insights from open data, they first must make progress around five key areas: enhance transparency and documentation, uphold quality and integrity, promote interoperability and standards, improve accessibility and useability, and address ethical considerations.

Motivation & Objective

• Motivate the exploration of how open data interacts with generative AI in a rapidly evolving AI landscape.
• Propose a Spectrum of Scenarios framework to categorize possible intersections between open data and generative AI.
• Identify data quality, provenance, and governance prerequisites for each scenario.
• Highlight organizational and ethical considerations to help data holders embrace AI-enabled open data access and insights.

Proposed method

• Develop a qualitative framework (Spectrum of Scenarios) to map open data and generative AI intersections.
• Define and classify scenarios: pertaining, adaptation, inference and insight generation, data augmentation, and open-ended exploration.
• Analyze data quality and provenance requirements for each scenario within the framework.
• Synthesize areas where improvements in openness (transparency, documentation), quality, interoperability, accessibility, and ethics are needed.

Experimental results

Research questions

• RQ1 What are the possible ways open data and generative AI can intersect in practice?
• RQ2 What data quality and provenance requirements are needed to support each intersection scenario?
• RQ3 What organizational practices and ethical considerations are necessary to enable AI-enabled open data access and insights?

Key findings

• A Spectrum of Scenarios framework outlines five intersection categories: pertaining, adaptation, inference and insight generation, data augmentation, and open-ended exploration.
• Progress for data holders to leverage generative AI hinges on improving transparency and documentation.
• Improvements are needed in data quality and integrity, interoperability and standards, accessibility and usability, and ethical considerations to realize AI-enabled open data benefits.
• The paper argues for approaching open data readiness through a structured framework rather than ad hoc AI adoption.