[Paper Review] Novel approach for ranking DEMs: Copernicus DEM improves one arc second open global topography

Conrad Bielski, Carlos M. López-Vázquez+10 more|arXiv (Cornell University)|Feb 16, 2023

Remote Sensing and LiDAR Applications11 citations

TL;DR

The paper introduces a DEM inter-comparison framework using randomized complete block design (RCBD) to rank six 1" global DEMs across diverse sites, finding CopDEM 1" (and FABDEM) as the overall best.

ABSTRACT

We present a practical approach to inter-compare a range of candidate digital elevation models (DEMs) based on pre-defined criteria and statistically sound ranking approach. The presented approach integrates the randomized complete block design (RCBD) into a novel framework for DEMs comparison. The method presented provides a flexible, statistically sound and customizable tool for evaluating the quality of any raster - in this case a DEM - by means of a ranking approach, which takes into account a confidence level, and can use both quantitative and qualitative criteria. The users can design their own criteria for the quality evaluation in relation to their specific needs. The application of the RCBD method to rank six 1" global DEMs, considering a wide set of study sites, covering different morphological and landcover settings, highlights the potentialities of the approach. We used a suite of criteria relating to the differences in the elevation, slope, and roughness distributions compared to reference DEMs aggregated from 1-5 m lidar-derived DEMs. Results confirmed significant superiority of CopDEM 1" and its derivative FABDEM as the overall best 1" global DEMs. They are slightly better than ALOS, and clearly outperform NASADEM and SRTM, which are in turn much better than ASTER.

Motivation & Objective

• Provide a flexible, statistically sound tool to inter-compare digital elevation models (DEMs) using pre-defined criteria.
• Integrate randomized complete block design (RCBD) to rank DEMs with confidence measures.
• Evaluate six global 1" DEMs across 24 test areas with diverse morphologies and land covers.
• Quantify DEM differences using elevation, slope, and roughness metrics and derive robust statistics.
• Enable user-defined criteria and per-land-type analysis to tailor DEM selection to specific applications.

Proposed method

• Construct a GIS/DEMIX database aggregating per-tile statistics and an opinions table that enables ties via predefined tolerances.
• Compute pixel-by-pixel differences for elevation, slope, and roughness (ELVD, SLPD, RUFD) against high-resolution reference DEMs.
• Derive statistics from difference distributions (STD, AVD, RMSE, MAE, LE90) to form non-negative, rankable metrics.
• Apply an RCBD framework to produce rankings (lower is better) with statistical support across tiles and criteria.
• Transform reference DEMs to match global DEMs’ grids and datums, while maintaining high fidelity through careful aggregation and datum shifts.
• Provide a Jupyter notebook workflow to reproduce rankings, graphics, and confidence assessments.

Figure 1: A) Location of the 24 test areas made up of 236 DEMIX tiles. B) Distribution of DEMIX tiles over Las Palmas Island. The names of the test areas shown on the map A: 01 – Norway [9 tiles], 02 – Oxford [4], 03 – Caen [6], 04 – Valonne [9], 05 – Vanoise [4], 06 – Trentino [1], 74 – Pyrenees [2

Experimental results

Research questions

• RQ1 Which global 1" DEMs perform best under a multi-criteria, site-diverse evaluation?
• RQ2 How do CopDEM and FABDEM compare to other global DEMs (e.g., ALOS, NASADEM, SRTM, ASTER) across elevation, slope, and roughness criteria?
• RQ3 Can RCBD ranking with mixed quantitative and qualitative criteria deliver statistically supported DEM rankings and confidence measures?
• RQ4 How do land-cover and slope classes affect DEM performance across tiles and criteria?

Key findings

• CopDEM 1" and its derivative FABDEM are the overall best 1" global DEMs.
• They are slightly better than ALOS and clearly outperform NASADEM and SRTM, which are in turn much better than ASTER.
• The RCBD-based ranking reveals performance differences across diverse terrains and land-cover settings.
• CopDEM and FABDEM show superior performance across multiple metrics (elevation, slope, roughness) relative to other candidates.
• The framework supports per-tile, per-land-type filtering and can incorporate user-defined tolerances to determine ties.
• The approach demonstrates the value of a transparent, statistically grounded DEM inter-comparison framework for open global topography.

Figure 2: Tile classifications for the 24 test areas in our sample, for one landcover classification and three geomorphometric landform classifications. Areas are arranged from north to south. Legend for colors is available in Supplementary Figure 1.