[Paper Review] Impact of model choice on LR assessment in case of rare haplotype match (frequentist approach)

Giulia Cereda|arXiv (Cornell University)|Feb 13, 2015

Forensic and Genetic Research52 references4 citations

TL;DR

This paper proposes two frequentist methods—the discrete Laplace and generalized Good-Turing—for estimating the likelihood ratio (LR) in the rare Y-STR haplotype match problem, where the matching profile is absent from the reference database. It emphasizes that the LR should be treated as 'a' LR rather than 'the' LR due to model and estimation uncertainty, and advocates for quantifying estimation error. The generalized Good-Turing method offers higher precision through data reduction, though at the cost of some information loss, while the discrete Laplace method preserves more data but yields less precise estimates.

ABSTRACT

The likelihood ratio (LR) measures the relative weight of forensic data regarding two hypotheses. Several levels of uncertainty arise if frequentist methods are chosen for its assessment: the assumed population model only approximates the true one and its parameters are estimated through a database. Moreover, it may be wise to discard part of data, especially that only indirectly related to the hypotheses. Different reductions define different LRs. Therefore, it is more sensible to talk about "a" LR instead of "the" LR, and the error involved in the estimation should be quantified. Two frequentist methods are proposed in the light of these points for the `rare type match problem', that is when a match between the perpetrator's and the suspect's DNA profile, never observed before in the database of reference, is to be evaluated.

Motivation & Objective

• To address the challenge of likelihood ratio (LR) estimation in the rare haplotype match problem, particularly for Y-STR profiles not present in existing databases.
• To argue that frequentist LR estimation must account for multiple levels of uncertainty: model choice, parameter estimation, and data reduction.
• To advocate for treating the LR as 'a' LR rather than 'the' LR, and to require explicit quantification of estimation error.
• To compare two frequentist methods—discrete Laplace and generalized Good-Turing—on their performance in terms of precision and information retention.
• To provide a coherent framework for frequentist LR assessment that avoids hybrid Bayesian-frequentist approaches and ensures methodological transparency.

Proposed method

• Applies the discrete Laplace method, a parametric frequentist approach, to model the frequency of rare Y-STR haplotypes using a Laplace prior on the logarithmic scale of population frequencies.
• Employs the generalized Good-Turing estimator, a nonparametric method, to estimate the probability of unobserved haplotypes by leveraging the frequency of frequencies in the database.
• Introduces data reduction by separating the evidence into E (relevant to hypotheses) and B (irrelevant to hypotheses), allowing for more focused estimation.
• Quantifies estimation error through simulation-based evaluation of the mean squared error (MSE) of the estimated LR under both methods.
• Compares the two methods by evaluating their performance in terms of precision (MSE) and information loss, using the theoretical LR = 1/f as a benchmark in a panmictic population.
• Uses simulation studies with small databases and rare haplotypes to assess the behavior of both methods under realistic forensic conditions.

Experimental results

Research questions

• RQ1 How can the likelihood ratio be consistently estimated in the rare Y-STR haplotype match problem when the matching profile is absent from the reference database?
• RQ2 To what extent does data reduction affect the precision and reliability of frequentist likelihood ratio estimates?
• RQ3 How do the discrete Laplace and generalized Good-Turing methods compare in terms of estimation error and information retention?
• RQ4 Is it justifiable to treat the likelihood ratio as 'a' LR rather than 'the' LR when multiple sources of uncertainty exist in model choice and parameter estimation?
• RQ5 What is the trade-off between estimation precision and information loss when applying different data reduction strategies in frequentist LR assessment?

Key findings

• The generalized Good-Turing method achieves significantly lower estimation error (measured in log10 scale) compared to the discrete Laplace method, especially when data reduction is applied.
• On average, the generalized Good-Turing method results in a loss of approximately 0.5 in log10(LR) strength compared to the theoretical LR = 1/f, representing a modest disadvantage for the prosecution.
• The discrete Laplace method preserves more data and thus maintains higher discriminatory power between hypotheses, but yields less precise estimates due to higher variance.
• Data reduction improves estimation precision, but excessive reduction leads to information loss, which reduces the method’s ability to distinguish between the prosecution and defense hypotheses.
• The discrete Laplace method is more universally applicable, as it does not require excluding any experimental runs, unlike the generalized Good-Turing method, which excludes 121 cases where N₂ = 0.
• The study concludes that there is no universally optimal method; the choice must be context-specific, balancing precision, information retention, and model assumptions.