[Paper Review] Tight Prediction Intervals Using Expanded Interval Minimization

Dongqi Su, Ying Yin Ting+1 more|arXiv (Cornell University)|Jun 28, 2018

Neural Networks and Applications11 references17 citations

TL;DR

This paper introduces Expanded Interval Minimization (EIM), a novel neural network loss function that directly predicts tight, coverage-constrained prediction intervals without assuming symmetric error distributions. By scaling bounds in mini-batches to meet target coverage and minimizing width, EIM achieves 1.37x tighter intervals on average than prior methods across two real-world datasets, outperforming MLE, ensemble, and quantile regression techniques especially at lower coverage levels.

ABSTRACT

Prediction intervals are a valuable way of quantifying uncertainty in regression problems. Good prediction intervals should be both correct, containing the actual value between the lower and upper bound at least a target percentage of the time; and tight, having a small mean width of the bounds. Many prior techniques for generating prediction intervals make assumptions on the distribution of error, which causes them to work poorly for problems with asymmetric distributions. This paper presents Expanded Interval Minimization (EIM), a novel loss function for generating prediction intervals using neural networks. This loss function uses minibatch statistics to estimate the coverage and optimize the width of the prediction intervals. It does not make the same assumptions on the distributions of data and error as prior work. We compare to three published techniques and show EIM produces on average 1.37x tighter prediction intervals and in the worst case 1.06x tighter intervals across two large real-world datasets and varying coverage levels.

Motivation & Objective

• To address the limitation of existing prediction interval methods that assume symmetric error distributions, which leads to suboptimal performance on real-world asymmetric data.
• To develop a method that produces tighter prediction intervals while maintaining correct coverage, without relying on distributional assumptions like Gaussian noise.
• To enable flexible, sample-specific prediction interval quantiles that adapt to data density, rather than enforcing symmetric bounds around a mean.
• To reduce training complexity compared to ensemble methods while improving interval tightness over existing neural network-based approaches.

Proposed method

• EIM trains a single neural network to output lower and upper prediction bounds directly, without separate models for mean and variance.
• The loss function first scales the predicted bounds in each minibatch to cover the target percentage of the data (e.g., 90%) using empirical quantiles.
• After scaling for coverage, the loss minimizes the width of the scaled bounds, optimizing for tightness while preserving correctness.
• The method uses minibatch statistics to estimate coverage, making it distribution-agnostic and suitable for asymmetric data.
• Unlike maximum likelihood or ensemble methods, EIM does not assume independent model and noise variance components or require multiple models.
• The approach allows for varying prediction interval quantiles per sample, enabling more adaptive and data-driven interval shapes.

Experimental results

Research questions

• RQ1 Can a neural network-based method generate tighter prediction intervals than existing techniques without assuming symmetric error distributions?
• RQ2 How does the performance of EIM compare to MLE, ensemble, and quantile regression across different coverage levels and real-world datasets?
• RQ3 Does EIM’s ability to adapt bounds based on data density lead to better interval tightness, especially in asymmetric distributions?
• RQ4 How does the training complexity and inference efficiency of EIM compare to ensemble and quantile regression methods?

Key findings

• EIM produces prediction intervals that are, on average, 1.37x tighter than the next best method across two large real-world datasets and varying coverage levels.
• For a 70% coverage target, EIM achieves intervals 1.26x tighter than the next best technique, improving to 1.06x tighter at 90% coverage.
• The performance gap between EIM and other methods is largest at lower coverage targets (e.g., 70%), where EIM leverages greater flexibility in interval shape.
• EIM’s predicted distributions are more skewed toward high-density regions of the data, unlike symmetric methods that expand equally from the mean.
• EIM outperforms quantile regression even though the latter requires exhaustive grid search over two hyperparameters, which EIM avoids.
• EIM is the fastest and simplest method, requiring only a single neural network, while ensemble methods require training 200 models and are significantly more complex.