[Paper Review] Constraints on Yield Parameters in Extended Maximum Likelihood Fits

T. M. Karbach, Maximilian Schlupp|arXiv (Cornell University)|Oct 26, 2012

Particle physics theoretical and experimental studies3 citations

TL;DR

This paper addresses the correct implementation of constraints on yield parameters in extended maximum likelihood fits, particularly within the RooFit framework. It proposes generating pseudo-experiments by fluctuating constraints during toy Monte Carlo generation, ensuring unbiased pulls even with constrained yields, and identifies critical flaws in RooFit's Constrain() and ExternalConstraints() methods that distort error and pull distributions.

ABSTRACT

The method of extended maximum likelihood is a well known concept of parameter estimation. One can implement external knowledge on the unknown parameters by multiplying the likelihood by constraint terms. In this note, we emphasize that this is also true for yield parameters in an extended maximum likelihood fit, which is widely used in the particle physics community. We recommend a way to generate pseudo-experiments in presence of constraint terms on yield parameters, and point to pitfalls inside the RooFit framework.

Motivation & Objective

• To clarify how external constraints on yield parameters can be properly incorporated into extended maximum likelihood fits.
• To address the challenge of generating unbiased pseudo-experiments when yield parameters are constrained.
• To identify and explain critical flaws in RooFit's RooMCStudy tool when using constraint terms in pseudo-experiment generation.
• To ensure that pull statistics remain unbiased and follow a unit Gaussian distribution even under constrained yield parameters.
• To provide a recommended method for generating pseudo-experiments that correctly accounts for constraint fluctuations in the generation step.

Proposed method

• Use a two-step generation process: first fluctuate the constrained yield parameters using their constraint PDFs (e.g., Gaussian), then generate the total number of events using a Poisson distribution.
• Apply the standard pull definition, $ p = (\hat{\theta} - \theta_{\text{true}})/\sigma_{\hat{\theta}} $, to assess bias and coverage.
• Ensure that the constraint PDFs are included in the generation phase, not just in the fitting phase, to avoid double-fluctuation issues.
• Use the ExternalConstraints() method in RooFit instead of Constrain() to avoid incorrect double-fluctuation of yield parameters.
• Verify that pull distributions remain unit Gaussian when constraints are applied, indicating unbiased estimation.
• Compare results from different constraint widths to assess bias and error inflation in pull and error distributions.

Experimental results

Research questions

• RQ1 How can constraints on yield parameters be correctly implemented in extended maximum likelihood fits to ensure unbiased parameter estimation?
• RQ2 What is the correct procedure for generating pseudo-experiments when yield parameters are subject to external constraints?
• RQ3 Why do RooFit's Constrain() and ExternalConstraints() methods in RooMCStudy produce inconsistent results when generating toy datasets with constrained yields?
• RQ4 How do constraint widths affect the bias and width of pull and error distributions in pseudo-experiment studies?
• RQ5 Under what conditions do pull distributions deviate from unit Gaussianity in constrained extended maximum likelihood fits?

Key findings

• Using RooFit's Constrain() method leads to incorrect double-fluctuation of yield parameters: the constraint is applied both during generation and fitting, distorting the total yield distribution and causing a bias of $ \mu_p = 0.3 $ in the pull distribution.
• The Constrain() method produces wider error and central value distributions compared to the unconstrained case, even when constraints are wide, due to redundant fluctuation of the same parameter.
• When the total number of events is explicitly specified in generateAndFit(), the pull distribution becomes dependent on constraint width, leading to incorrect coverage for wide constraints.
• The ExternalConstraints() method avoids double-fluctuation and correctly generates yields as Poisson-distributed, but computes pulls using a fixed constraint, resulting in a pull distribution that is too narrow when constraints are wide.
• Only when constraints are fluctuated during generation and the standard pull definition is used does the pull distribution remain unit Gaussian, confirming unbiased estimation.
• Deviations in the fitted total yield $ N_f - N' $ can reach up to $ \approx 10 $ events (or $ \approx 1.4\% $) under strong constraints, highlighting the need for careful generation procedures.