[Paper Review] Lower bounds for identifying subset members with subset queries

Emanuel Knill|arXiv (Cornell University)|Jan 22, 1995

SARS-CoV-2 detection and testing15 references60 citations

TL;DR

This paper establishes tight lower bounds on the number of queries required in two-stage group testing algorithms to identify a random subset $ P $ of $ n $ objects, where each object is included in $ P $ independently with probability $ p/n $. It shows that if the first stage uses $ o(\log n \log n / \log \log n) $ queries, the expected number of individual tests in the second stage is $ n^{1-o(1)} $, while algorithms with $ O(\log n \log n / \log \log n) $ first-stage queries can achieve $ \Exp(q_2) = o(1) $, resolving a conjecture on non-adaptive group testing efficiency.

ABSTRACT

An instance of a group testing problem is a set of objects $\cO$ and an unknown subset $P$ of $\cO$. The task is to determine $P$ by using queries of the type ``does $P$ intersect $Q$'', where $Q$ is a subset of $\cO$. This problem occurs in areas such as fault detection, multiaccess communications, optimal search, blood testing and chromosome mapping. Consider the two stage algorithm for solving a group testing problem. In the first stage a predetermined set of queries are asked in parallel and in the second stage, $P$ is determined by testing individual objects. Let $n=\cardof{\cO}$. Suppose that $P$ is generated by independently adding each $x\in \cO$ to $P$ with probability $p/n$. Let $q_1$ ($q_2$) be the number of queries asked in the first (second) stage of this algorithm. We show that if $q_1=o(\log(n)\log(n)/\log\log(n))$, then $\Exp(q_2) = n^{1-o(1)}$, while there exist algorithms with $q_1 = O(\log(n)\log(n)/\log\log(n))$ and $\Exp(q_2) = o(1)$. The proof involves a relaxation technique which can be used with arbitrary distributions. The best previously known bound is $q_1+\Exp(q_2) = \Omega(p\log(n))$. For general group testing algorithms, our results imply that if the average number of queries over the course of $n^\gamma$ ($\gamma>0$) independent experiments is $O(n^{1-\epsilon})$, then with high probability $\Omega(\log(n)\log(n)/\log\log(n))$ non-singleton subsets are queried. This settles a conjecture of Bill Bruno and David Torney and has important consequences for the use of group testing in screening DNA libraries and other applications where it is more cost effective to use non-adaptive algorithms and/or too expensive to prepare a subset $Q$ for its first test.

Motivation & Objective

• To determine the minimum number of queries required in two-stage group testing when $ P $ is a random subset of $ n $ objects, with each object included independently with probability $ p/n $.
• To analyze the trade-off between the number of queries in the first stage and the expected number of individual tests in the second stage.
• To resolve a conjecture by Bruno and Torney regarding the query complexity of non-adaptive group testing in applications like DNA library screening.
• To establish fundamental limits on the efficiency of non-adaptive group testing algorithms under probabilistic subset generation.

Proposed method

• The authors introduce a relaxation technique to analyze the query complexity of group testing under arbitrary distributions, enabling analysis of random subset models.
• They model the group testing problem as a two-stage process: a non-adaptive first stage with $ q_1 $ queries, followed by adaptive individual testing in the second stage with $ q_2 $ queries.
• The analysis uses probabilistic methods to bound the expected number of second-stage queries, $ \Exp(q_2) $, based on the number of first-stage queries $ q_1 $.
• The key technical contribution is a lower bound showing that $ q_1 = o(\log n \log n / \log \log n) $ implies $ \Exp(q_2) = n^{1-o(1)} $, even under optimal query design.
• The method applies to general distributions and is used to derive a general lower bound on the number of non-singleton subsets queried in non-adaptive algorithms.

Experimental results

Research questions

• RQ1 What is the minimum number of first-stage queries required to ensure that the expected number of second-stage individual tests is sub-polynomial?
• RQ2 Can non-adaptive group testing algorithms achieve sub-polynomial total query cost when $ P $ is a sparse random subset?
• RQ3 What is the fundamental limit on the number of non-singleton subsets that must be queried in non-adaptive group testing under general distributions?
• RQ4 Does the conjecture by Bruno and Torney—that $ \Omega(\log n \log n / \log \log n) $ queries are necessary—hold under probabilistic subset models?

Key findings

• If the first stage uses $ o(\log n \log n / \log \log n) $ queries, then the expected number of second-stage individual tests is $ n^{1-o(1)} $, indicating poor scalability.
• There exist algorithms with $ q_1 = O(\log n \log n / \log \log n) $ first-stage queries and $ \Exp(q_2) = o(1) $, showing this bound is tight.
• The result resolves the conjecture of Bruno and Torney by proving that $ \Omega(\log n \log n / \log \log n) $ non-singleton subsets must be queried in non-adaptive group testing with high probability.
• The analysis establishes a new lower bound of $ q_1 + \Exp(q_2) = \Omega(p \log n) $, improving upon the previously known bound.
• The relaxation technique used in the proof is general and applies to arbitrary distributions, making the results widely applicable beyond the specific probabilistic model considered.