[Paper Review] Improvement of the cross-entropy method in high dimension for failure probability estimation through a one-dimensional projection without gradient estimation

Maxime El-Masri, Jérôme Morio+1 more|arXiv (Cornell University)|Dec 21, 2020

Probabilistic and Robust Engineering Design36 references4 citations

TL;DR

This paper proposes a one-dimensional projection-based improvement to the cross-entropy (CE) method for rare event probability estimation in high-dimensional problems. By updating the Gaussian covariance matrix only along the direction of the current mean vector estimate, the method avoids the covariance matrix collapse issue in high dimensions without requiring gradient estimation. It significantly improves CE performance, especially in high dimensions, while maintaining the same computational budget and never degrading performance.

ABSTRACT

Rare event probability estimation is an important topic in reliability analysis. Stochastic methods, such as importance sampling, have been developed to estimate such probabilities but they often fail in high dimension. In this paper, we propose a new cross-entropy-based importance sampling algorithm to improve rare event probability estimation in high dimension. We focus on the cross-entropy method with Gaussian auxiliary distributions and we suggest to update the Gaussian covariance matrix only in a one-dimensional subspace. For that purpose, the main idea is to consider the projection in the one-dimensional subspace spanned by the sample mean vector, which gives an influential direction for the variance estimation. This approach does not require any additional simulation budget compared to the basic cross-entropy algorithm and we show on different numerical test cases that it greatly improves its performance in high dimension.

Motivation & Objective

• Address the failure of the standard cross-entropy (CE) method in high-dimensional reliability problems due to covariance matrix degeneracy.
• Improve rare event probability estimation in high dimensions using a computationally efficient, gradient-free approach.
• Maintain the same simulation budget as the basic CE method while enhancing convergence and accuracy.
• Provide a robust alternative to full covariance matrix updates in CE by focusing on the most influential direction in the parameter space.
• Ensure the method is applicable to unimodal target failure distributions, a common assumption in reliability analysis.

Proposed method

• Project the high-dimensional parameter update onto a one-dimensional subspace spanned by the current mean vector estimate (m*).
• Update only the covariance terms in the direction of m* at each iteration, rather than the full covariance matrix.
• Use the sample mean vector as the primary direction because it corresponds to the most influential direction for variance reduction.
• Preserve the standard CE algorithm’s structure but restrict the covariance update to a single dominant direction to reduce estimation noise.
• Leverage the light-tailed nature of the Gaussian distribution to ensure stable variance estimation along m*.
• Avoid gradient estimation entirely, relying only on sample means and weights from the importance sampling framework.

Experimental results

Research questions

• RQ1 Can a one-dimensional projection of the covariance update direction significantly improve the convergence and accuracy of the CE method in high-dimensional failure probability estimation?
• RQ2 Does restricting covariance updates to the direction of the current mean vector prevent the covariance matrix degeneracy that plagues standard CE in high dimensions?
• RQ3 Can this method maintain or improve performance compared to full covariance updates (e.g., CEd) while avoiding the need for gradient computation?
• RQ4 Is the proposed method robust across different high-dimensional test cases, especially when all input variables are influential?
• RQ5 Does the method preserve the theoretical properties of CE while reducing computational complexity and estimation variance in high dimensions?

Key findings

• The proposed CE-m* method avoids the covariance matrix collapse issue common in standard CE for high-dimensional problems.
• In high dimensions (e.g., n > 200), CE-m* consistently outperforms standard CE and CEd, achieving accurate estimates where CE fails to converge.
• For dimensions up to 100, CE-m* performs comparably to CEd in terms of accuracy and coefficient of variation.
• On the parabolic test function (n = 100), CE-m* achieved a mean estimate of 2.9 × 10⁻⁴ with a coefficient of variation of 7.2%, significantly outperforming standard CE.
• The method never degrades CE performance and often provides orders-of-magnitude better convergence in high-dimensional settings.
• Empirical results show that estimating variance only along the m* direction yields more reliable and informative updates than full or diagonal covariance updates in high dimensions.