[Paper Review] Planetary computing for data-driven environmental policy-making

Patrick Ferris, Michael Dales+9 more|arXiv (Cornell University)|Mar 8, 2023

Scientific Computing and Data Management4 citations

TL;DR

The paper proposes a planetary computing framework to enable non-expert users—such as ecologists, policymakers, and journalists—to process petabyte-scale remote sensing data for environmental decision-making. By integrating secure, composable data pipelines with provenance tracking, differential privacy, and containerized reproducibility, the system supports scalable, transparent, and auditable analysis essential for combating climate and biodiversity crises.

ABSTRACT

We make a case for "planetary computing" -- infrastructure to handle the ingestion, transformation, analysis and publication of global data products for furthering environmental science and enabling better informed policy-making. We draw on our experiences as a team of computer scientists working with environmental scientists on forest carbon and biodiversity preservation, and classify existing solutions by their flexibility in scalably processing geospatial data, and also how well they support building trust in the results via traceability and reproducibility. We identify research gaps in the intersection of computing and environmental science around how to handle continuously changing datasets that are often collected across decades and require careful access control rather than being fully open access.

Motivation & Objective

• Address the urgent need for scalable, trustworthy data systems to support environmental policy-making amid climate and biodiversity crises.
• Enable non-computer science experts—such as ecologists, policymakers, and journalists—to perform large-scale spatial-temporal analysis without systems expertise.
• Ensure data privacy and security while maintaining transparency and auditability, especially for sensitive ecological data.
• Develop a federated, open, and extensible computing infrastructure that resists capture by single entities and supports long-term sustainability.
• Integrate advanced systems research in provenance, privacy, and reproducibility to prevent misuse and ensure public trust in environmental data insights.

Proposed method

• Design a planetary computing engine with a visual or domain-specific language interface (e.g., R/Python) to abstract low-level systems complexity.
• Implement a dataflow pipeline architecture that ingests, cleans, and normalizes multi-source data (satellite, drone, ground sensors) with versioned provenance tracking.
• Use content-addressable storage (e.g., IPFS, Tezos) and Docker containers to enable deterministic, reproducible, and portable execution of data pipelines.
• Integrate differential privacy and decentralized information flow control to protect sensitive data while allowing eventual transparency.
• Support hybrid data access via 'embargoed' initial access with eventual openness, enabling auditing without exposing raw data prematurely.
• Enable secure, composable pipelines using DIFC (Data-Integrity-First Computation) labels to propagate privacy and integrity constraints across public and private data sources.

Figure 1. Ideal dataflow pipeline for a planetary computing engine

Experimental results

Research questions

• RQ1 How can planetary-scale environmental data be processed in a way that is both accessible to non-CS experts and reproducible for scientific scrutiny?
• RQ2 What systems architecture enables secure, privacy-preserving computation on sensitive ecological data while still allowing for public auditability?
• RQ3 How can data provenance and lineage be tracked end-to-end across petabyte-scale datasets and machine learning pipelines to ensure trust in policy decisions?
• RQ4 What mechanisms can ensure that planetary computing systems remain open, federated, and resilient to capture by any single organization or cloud provider?
• RQ5 How can the integration of remote sensing, AI inference, and ground-truth data be orchestrated in a scalable, composable, and interactive manner for real-time environmental monitoring?

Key findings

• Existing systems for processing global environmental data are incomplete, not user-friendly for non-experts, and lack sufficient reliability and trust for high-stakes policy decisions.
• The proposed planetary computing framework enables end-to-end data processing pipelines that support reproducibility through containerized, content-addressed execution with full provenance tracking.
• By using DIFC labels and secure data composition, the system allows private data to be combined with public datasets in a way that preserves privacy and enables verifiable computation.
• The integration of differential privacy and eventual openness mechanisms helps prevent adversarial manipulation while maintaining transparency over time.
• The system supports interactive, scalable analysis via web-based execution (e.g., WebGPU, WebAssembly), reducing dependency on proprietary cloud platforms and enabling broader access for researchers in the Global South.
• The framework enables the creation of standalone, reproducible scientific artifacts—such as Docker images with embedded data and code—that can be independently verified, enhancing trust in environmental assessments.