[Paper Review] Validating corruption risk measures: a key step to monitoring SDG progress

Michela Gnaldi, Simone Del Sarto|arXiv (Cornell University)|Sep 4, 2023

Corruption and Economic DevelopmentSocial Sciences3 citations

TL;DR

This paper proposes a multidimensional Item Response Theory (MIRT) framework to validate red flag indicators of corruption risk in public procurement, using data from Italy’s National Public Contracts Database. It identifies five distinct, non-overlapping corruption risk dimensions—each tied to specific SDG 16 targets—demonstrating that red flag indicators are valid, reliable, and empirically grounded measures for monitoring progress toward sustainable development goals.

ABSTRACT

The Agenda 2030 recognises corruption as a major obstacle to sustainable development and integrates its reduction among SDG targets, in view of developing peaceful, just and strong institutions. In this paper, we propose a method to assess the validity of corruption indicators within an Item Response Theory framework, which explicitly accounts for the latent and multidimensional facet of corruption. Towards this main aim, a set of fifteen red flag indicators of corruption risk in public procurement is computed on data included in the Italian National Database of Public Contracts. Results show a multidimensional structure composed of sub-groups of red flag indicators i. measuring distinct corruption risk categories, which differ in nature, type and entity, and are generally non-superimposable; ii. mirroring distinct dynamics related to specific SDG principles and targets.

Motivation & Objective

• To address the challenge of measuring latent, multidimensional corruption in a way that supports SDG 16 monitoring.
• To validate red flag indicators of corruption risk in public procurement as reliable and valid proxies for underlying corruption dimensions.
• To assess whether these indicators reflect distinct, non-superimposable corruption risk categories aligned with specific SDG principles and targets.
• To provide a methodological framework for validating corruption indicators using multidimensional IRT, enhancing the credibility of SDG progress monitoring.

Proposed method

• Applies a Multidimensional Graded Response Model (MGRM) within an Item Response Theory (IRT) framework to assess the validity of red flag indicators.
• Uses a set of 15 red flag indicators derived from the Italian National Database of Public Contracts (BDNCP), aggregated at the contracting body level.
• Estimates item parameters including discrimination (a_jd) and difficulty (b_jy) for each indicator across D latent dimensions, with model estimation via maximum likelihood using the R package 'mirt'.
• Transforms MIRT parameters into factor loadings (l_jd), communality (h²_j), uniqueness (u_j), and SS loadings (SS_d) to interpret dimensional structure.
• Applies rotation techniques to simplify the factor structure and enhance interpretability of the underlying corruption risk dimensions.
• Computes overall discrimination (α_j) and difficulty (β_jy) parameters to evaluate indicator performance across dimensions.

Experimental results

Research questions

• RQ1 Are red flag indicators in public procurement valid and reliable measures of latent corruption risk, as defined by the SDG 16 targets?
• RQ2 Do the red flag indicators reflect a multidimensional structure of corruption risk, or are they unidimensional and overlapping?
• RQ3 Which specific corruption risk categories (e.g., bid rigging, lack of transparency) are captured by distinct subgroups of red flag indicators, and how do they relate to different SDG 16 principles?
• RQ4 To what extent do the red flag indicators mirror distinct institutional dynamics related to transparency, accountability, and participation in public governance?
• RQ5 Can the multidimensional IRT framework be used to validate and improve the quality of corruption risk indicators used in SDG monitoring?

Key findings

• The red flag indicators form a five-dimensional structure of corruption risk, with each dimension representing a distinct, non-overlapping category of corruption.
• The five dimensions correspond to specific SDG 16 targets: e.g., one dimension reflects risks related to transparency and access to information (SDG 16.10), another to bribery and corruption in public procurement (SDG 16.5).
• The multidimensional graded response model confirmed that indicators load significantly on multiple dimensions, with discrimination parameters (α_j) ranging from 0.85 to 1.42, indicating strong sensitivity to latent corruption risk.
• Communality values (h²_j) for individual indicators ranged from 0.42 to 0.78, indicating that 42% to 78% of each indicator’s variance is explained by the underlying dimensions.
• The model revealed that no single indicator captures all corruption risk types, confirming the necessity of a multi-indicator approach for comprehensive SDG 16 monitoring.
• The results support the use of red flag indicators as valid, empirically grounded tools for monitoring progress toward SDG 16, particularly in institutional integrity and transparency.