[Paper Review] Calibrated Multivariate Distributional Regression with Pre-Rank Regularization

Aya Laajil, Elnura Zhalieva+2 more|arXiv (Cornell University)|Jan 30, 2026

Forecasting Techniques and Applications0 citations

TL;DR

The paper introduces a differentiable pre-rank based regularizer to enforce multivariate calibration during training of distributional regression models, including a novel PCA-based pre-rank, and shows improved multivariate calibration on 18 real-world datasets without harming predictive accuracy.

ABSTRACT

The goal of probabilistic prediction is to issue predictive distributions that are as informative as possible, subject to being calibrated. Despite substantial progress in the univariate setting, achieving multivariate calibration remains challenging. Recent work has introduced pre-rank functions, scalar projections of multivariate forecasts and observations, as flexible diagnostics for assessing specific aspects of multivariate calibration, but their use has largely been limited to post-hoc evaluation. We propose a regularization-based calibration method that enforces multivariate calibration during training of multivariate distributional regression models using pre-rank functions. We further introduce a novel PCA-based pre-rank that projects predictions onto principal directions of the predictive distribution. Through simulation studies and experiments on 18 real-world multi-output regression datasets, we show that the proposed approach substantially improves multivariate pre-rank calibration without compromising predictive accuracy, and that the PCA pre-rank reveals dependence-structure misspecifications that are not detected by existing pre-ranks.

Motivation & Objective

• Motivate the need for calibrated multivariate predictive distributions beyond marginal calibration.
• Develop a differentiable regularizer that enforces calibration with respect to chosen pre-rank functions during training.
• Introduce a PCA-based pre-rank to probe and enforce calibration along principal directions of the predictive distribution.
• Demonstrate through simulations and 18 real-world datasets that regularization improves multivariate calibration without reducing predictive performance.

Proposed method

• Define pre-rank functions to reduce multivariate calibration to scalar checks via projected PITs.
• Propose a differentiable regularizer based on a kernel-smoothed empirical CDF (PCE-KDE) of projected PITs to penalize non-uniformity.
• Augment the training objective with a regularization term that penalizes deviations from uniformity of Z_rho across a fixed grid of levels.
• Introduce a new PCA-based pre-rank that projects Y onto the leading principal components of the predictive distribution to detect dependence-structure misspecifications.
• Adapt existing post-hoc calibration notions (HDR, copula) into training-time pre-rank functions for broader calibration checks.
• Provide an end-to-end training algorithm (Algorithm 1) with a specific PCA pre-rank (Algorithm 2) for efficient computation.

Figure 1 : Marginal calibration is insufficient for multivariate predictive distributions. (Bottom) The model is well-calibrated marginally (per-dimension densities and PIT curves) yet fails to capture dependence, as shown by the mismatch in sample geometry and the strongly miscalibrated Dependency

Experimental results

Research questions

• RQ1 How can multivariate calibration be enforced during training of distributional regression models?
• RQ2 Which pre-rank functions best detect different forms of miscalibration in multivariate forecasts?
• RQ3 Does pre-rank based regularization improve calibration without harming predictive accuracy across diverse datasets?
• RQ4 Does the PCA-based pre-rank reveal dependence-structure misspecifications not detected by other pre-ranks?
• RQ5 How do calibrated models perform on real-world multi-output regression tasks compared to unregularized baselines?

Key findings

• Regularization substantially reduces Probabilistic Calibration Error (PCE) for the chosen pre-ranks.
• Regularized models show improved calibration across pre-ranks while maintaining similar or better predictive performance (NLL and energy score).
• The PCA pre-rank detects dependence-structure misspecifications that other pre-ranks miss, highlighting its complementary value.
• Across 18 real-world datasets, pre-rank regularization consistently improves multivariate calibration without degrading predictive accuracy.
• The approach is flexible: different pre-ranks can be used to target different calibration aspects, including marginals, dependence, and HDR/copula-based checks.

(a) Marginal