[Paper Review] Comparison of the likelihood ratios of two diagnostic tests subject to a paired design: confidence intervals and sample size

José Antonio Roldán Nofuentes, Saad Bouh Sidaty-Regad|arXiv (Cornell University)|Jul 31, 2024

Reliability and Agreement in MeasurementDecision Sciences3 citations

TL;DR

This paper proposes six approximate confidence intervals for comparing the likelihood ratios of two paired binary diagnostic tests, evaluates their coverage probabilities and average lengths via simulation, and introduces a sample size calculation method to achieve desired precision in estimating the ratio of likelihood ratios. The approach is validated using coronary artery disease data, offering practical tools for diagnostic test comparison in paired designs.

ABSTRACT

Positive and negative likelihood ratios are parameters which are used to assess and compare the effectiveness of binary diagnostic tests. Both parameters only depend on the sensitivity and specificity of the diagnostic test and are equivalent to a relative risk. This article studies the comparison of the likelihood ratios of two binary diagnostic tests subject to a paired design through confidence intervals. Six approximate confidence intervals are presented for the ratio of the likelihood ratios, and simulation experiments are carried out to study the coverage probabilities and the average lengths of the intervals considered, and some general rules of application are proposed. A method is also proposed to determine the sample size necessary to estimate the ratio between the likelihood ratios with a determined precision. The results were applied to the diagnosis of coronary artery disease.

Motivation & Objective

• To develop and evaluate confidence intervals for the ratio of positive and negative likelihood ratios in paired diagnostic test designs.
• To assess the performance of six approximate confidence interval methods through simulation in terms of coverage probability and interval length.
• To propose a sample size calculation procedure ensuring desired precision in estimating the ratio of likelihood ratios.
• To provide practical, data-driven guidelines for applying these methods in real-world diagnostic studies.
• To demonstrate the method's utility through an application to coronary artery disease diagnosis.

Proposed method

• Six approximate confidence interval methods are derived for the ratio of likelihood ratios under a paired design, using asymptotic distribution theory and variance stabilizing transformations.
• Simulation experiments are conducted to evaluate the coverage probability and average length of each confidence interval across various sample sizes and parameter configurations.
• The ratio of likelihood ratios is modeled using the delta method and Fieller's theorem to account for the correlation between paired tests.
• A sample size formula is derived based on the desired margin of error and confidence level for the ratio of likelihood ratios, incorporating estimated sensitivity and specificity.
• The performance of each method is compared using empirical coverage and average interval length, with recommendations based on simulation results.
• The method is applied to a real dataset on coronary artery disease to illustrate practical implementation and interpretation.

Experimental results

Research questions

• RQ1 Which of the six proposed confidence interval methods for the ratio of likelihood ratios performs best in terms of coverage probability and interval length under paired designs?
• RQ2 How does the performance of these confidence intervals vary with different sample sizes and underlying sensitivity/specificity values?
• RQ3 What is the optimal sample size required to estimate the ratio of likelihood ratios with a specified margin of error and confidence level?
• RQ4 How do the proposed confidence intervals compare to existing methods in terms of statistical efficiency and accuracy?
• RQ5 Can the proposed method be reliably applied to real-world diagnostic test comparisons, such as in coronary artery disease?

Key findings

• The simulation results show that the modified score-based confidence interval method exhibits the most accurate coverage probability, closely matching the nominal confidence level across various scenarios.
• The average length of the confidence intervals varied significantly among methods, with the score-based approach providing a favorable balance between precision and coverage.
• The proposed sample size formula effectively controls the margin of error for the ratio of likelihood ratios, with coverage close to the nominal level when the estimated sensitivity and specificity are accurate.
• The method was successfully applied to a coronary artery disease dataset, demonstrating its practical utility in clinical diagnostic research.
• General application rules are proposed, recommending the use of the score-based interval for most scenarios due to its robust performance.
• The study confirms that paired designs require specialized methods for likelihood ratio comparison, as standard independent-sample approaches may lead to biased inference.