[论文解读] Limitations of Mean-Based Algorithms for Trace Reconstruction at Small Distance

Elena Grigorescu, Madhu Sudan+1 more|arXiv (Cornell University)|Nov 27, 2020

Algorithms and Data Compression参考文献 25被引用 9

一句话总结

本文证明，尽管均值算法在迹重建中具有强大能力，但在编辑距离为4时，仍无法用亚多项式数量的迹来区分某些二进制字符串对。作者证明了存在显式构造的、编辑距离为4的字符串对，其均值算法需要指数级数量的迹才能区分，这一局限性与数论中的困难Prouhet-Tarry-Escott问题相关联。

ABSTRACT

Trace reconstruction considers the task of recovering an unknown string $x \in \{0,1\}^n$ given a number of independent "traces", i.e., subsequences of $x$ obtained by randomly and independently deleting every symbol of $x$ with some probability $p$. The information-theoretic limit of the number of traces needed to recover a string of length $n$ is still unknown. This limit is essentially the same as the number of traces needed to determine, given strings $x$ and $y$ and traces of one of them, which string is the source. The most-studied class of algorithms for the worst-case version of the problem are "mean-based" algorithms. These are a restricted class of distinguishers that only use the mean value of each coordinate on the given samples. In this work we study limitations of mean-based algorithms on strings at small Hamming or edit distance. We show that, on the one hand, distinguishing strings that are nearby in Hamming distance is "easy" for such distinguishers. On the other hand, we show that distinguishing strings that are nearby in edit distance is "hard" for mean-based algorithms. Along the way, we also describe a connection to the famous Prouhet-Tarry-Escott (PTE) problem, which shows a barrier to finding explicit hard-to-distinguish strings: namely such strings would imply explicit short solutions to the PTE problem, a well-known difficult problem in number theory. Furthermore, we show that the converse is also true, thus, finding explicit solutions to the PTE problem is equivalent to the problem of finding explicit strings that are hard-to-distinguish by mean-based algorithms. Our techniques rely on complex analysis arguments that involve careful trigonometric estimates, and algebraic techniques that include applications of Descartes' rule of signs for polynomials over the reals.

研究动机与目标

• 研究均值算法在小汉明距离和编辑距离下的迹重建中区分二进制字符串的局限性。
• 确定在编辑距离较近（例如距离为4）的字符串是否能被均值算法高效区分。
• 建立难以区分的字符串对与数论中Prouhet-Tarry-Escott（PTE）问题之间的联系。
• 通过代数与复分析技术，提供编辑距离为4的、对均值算法具有挑战性的二进制字符串的显式构造。

提出的方法

• 使用复分析与三角估计，分析与迹分布相关的生成函数的行为。
• 应用笛卡尔符号法则，对由期望迹值之差导出的多项式中的符号变化次数进行上界估计。
• 将迹重建问题分解为块结构，以显式建模插入与删除操作。
• 通过利用多项式系数分析所得的结构约束，构造编辑距离为4的显式字符串。
• 将寻找难以区分字符串的问题约化为求解Prouhet-Tarry-Escott问题的短解。
• 采用仅依赖于各比特位置在迹中经验期望的均值算法框架。

实验结果

研究问题

• RQ1 均值算法能否高效区分编辑距离较近的二进制字符串对？
• RQ2 是否存在编辑距离较小（例如4）的二进制字符串对，其对均值算法而言难以区分？
• RQ3 此类困难字符串对的存在性与数论中Prouhet-Tarry-Escott问题之间有何关联？
• RQ4 从编辑距离2到4的转变是否标志着均值区分器样本复杂度的显著阈值？
• RQ5 能否通过代数与分析工具（如多项式符号变化与复分析）刻画均值算法的局限性？

主要发现

• 存在显式构造的编辑距离为4的二进制字符串对，使得任意均值算法均需超多项式数量的迹才能区分它们。
• 区分编辑距离为4的字符串的样本复杂度至少为 $\exp(\Omega(n^{1/3}))$ 量级，其下界由关联多项式中符号变化次数的平方根决定。
• 对于编辑距离为2的字符串，均值算法仅需 $n^{O(1)}$ 数量的迹即可区分，表明编辑距离为2与4之间存在显著的复杂度阈值。
• 此类困难字符串对的存在性等价于Prouhet-Tarry-Escott问题存在短而显式的解，这是数论中的一个著名开放问题。
• 生成函数差值中符号变化次数的上界为 $3d - 1$，其中 $d$ 为字符串分解中的块数，由此得到 $w=1$ 处零点的重数上界为 $3d$。
• $\ell_1$-距离在两字符串的期望迹向量之间下界为 $\frac{q}{e} \left( \frac{q}{n} \right)^{3d}$，当 $d$ 较大时，该下界意味着样本复杂度为超多项式。