[Paper Review] Fast Clustering Analysis of Inhomogeneous Megapixel CMB maps

István Szapudi, S. Prunet+1 more|arXiv (Cornell University)|Jul 19, 2001

Medical Image Segmentation Techniques3 citations

TL;DR

This paper presents a fast, scalable algorithm for estimating the angular power spectrum $C_\ell$ from inhomogeneous megapixel CMB maps using weighted pixel-space correlation functions and fast spherical harmonic transforms. The method achieves $\sim N^{1.61}$ scaling, extracting $C_\ell$ from a MAP-like map (512 resolution, ~3 million pixels) in under 5 minutes on a 500MHz CPU, with results closely matching theoretical expectations and enabling efficient Monte Carlo noise covariance estimation.

ABSTRACT

Szapudi et al (2001) introduced the method of estimating angular power spectrum of the CMB sky via heuristically weighted correlation functions. Part of the new technique is that all (co)variances are evaluated by massive Monte Carlo simulations, therefore a fast way to measure correlation functions in a high resolution map is essential. This letter presents a new algorithm to calculate pixel space correlation functions via fast spherical harmonics transforms. Our present implementation of the idea extracts correlations from a MAP-like CMB map (HEALPix resolution of 512, i.e. $ \simeq 3 imes 10^6$ pixels) in about 5 minutes on a 500MHz computer, including $C_\ell$ inversion; the analysis of one Planck-like map takes less then one hour. We use heuristic window and noise weighting in pixel space, and include the possibility of additional signal weighting as well, either in $\ell$ or pixel space. We apply the new code to an ensemble of MAP simulations, to test the response of our method to the inhomogenous sky coverage/noise of MAP. We show that the resulting $C_\ell$'s are very close to the theoretical expectations. The HEALPix based implementation of the method, SpICE (Spatially Inhomogenous Correlation Estimator) will be available to the public from the authors.

Motivation & Objective

• Address the computational bottleneck in analyzing megapixel CMB maps from missions like MAP and Planck, which exceed the capabilities of standard maximum likelihood methods.
• Develop a fast, scalable alternative to optimal $C_\ell$ estimation that remains accurate despite inhomogeneous sky coverage and noise.
• Enable efficient power spectrum estimation using heuristic weighting and fast spherical harmonic transforms, suitable for real-world data with complex noise and geometry.
• Facilitate Monte Carlo-based estimation of noise and signal covariance matrices to improve accuracy in inhomogeneous conditions.
• Provide a general-purpose, publicly available tool (SpICE) for cosmological power spectrum estimation applicable beyond CMB.

Proposed method

• Estimate the two-point correlation function in pixel space using a weighted estimator with heuristic edge correction and noise weighting.
• Utilize fast spherical harmonic transforms (SHT) to accelerate the computation of correlation functions and $C_\ell$ estimation.
• Compute noise covariance matrices via Monte Carlo simulations of noise realizations, enabling accurate error estimation without assuming diagonal noise.
• Apply Gauss-Legendre quadrature to integrate the weighted correlation function and recover $C_\ell$ coefficients.
• Incorporate signal weighting in $\ell$-space or pixel space, allowing for semi-heuristic optimization of the estimator.
• Implement the method using HEALPix for spherical sky geometry, enabling efficient handling of irregular sky cuts and holes.

Experimental results

Research questions

• RQ1 Can a fast, scalable method estimate $C_\ell$ from megapixel CMB maps with inhomogeneous noise and sky coverage without relying on optimal, computationally expensive likelihood methods?
• RQ2 How accurately can a heuristic, correlation-function-based estimator reproduce theoretical $C_\ell$ values when applied to realistic, inhomogeneous data such as from the MAP mission?
• RQ3 What is the computational scaling of a correlation-function-based $C_\ell$ estimator when accelerated via fast spherical harmonic transforms and Monte Carlo noise modeling?
• RQ4 To what extent can heuristic weighting and Monte Carlo noise covariance estimation maintain accuracy compared to optimal methods in the presence of complex noise patterns?
• RQ5 Can the method be generalized to other cosmological power spectrum and higher-order function estimations (e.g., bispectrum, cross-correlations) in both CMB and large-scale structure data?

Key findings

• The method achieves a measured computational scaling of $\sim N^{1.61}$, significantly faster than standard $N^3$ or $N^2$ methods, making it feasible for megapixel maps.
• A MAP-like CMB map (HEALPix resolution 512, ~3×10^6 pixels) can be processed in less than 5 minutes on a 500MHz CPU, including $C_\ell$ inversion.
• The estimated $C_\ell$ values from an ensemble of MAP simulations agree closely with theoretical expectations, validating the method’s accuracy.
• Noise covariance is accurately estimated via Monte Carlo simulations of noise realizations, enabling robust error estimation even for non-diagonal noise matrices.
• The method remains robust to irregular sky coverage, such as cut-out holes around bright sources, with minimal impact on performance or speed.
• The HEALPix-based implementation, named SpICE (Spatially Inhomogeneous Correlation Estimator), is publicly available and supports future extensions to non-Gaussianity, cross-correlations, and polarization.