[Paper Review] Counting Small Permutation Patterns

Chaim Even‐Zohar, Calvin Leng|arXiv (Cornell University)|Nov 4, 2019

Bayesian Methods and Mixture Models4 citations

TL;DR

This paper introduces corner tree formulas, an algebraic framework that enables nearly linear-time counting of small permutation patterns—specifically all 2- and 3-patterns, eight 4-patterns, and 23 independent linear combinations of 4-patterns—using multidimensional sum trees and divide-and-conquer strategies. The key contribution is a near-optimal $ frac{1}{2}$-time algorithm for counting one remaining 4-pattern, achieving $ ilde{O}(n^{3/2})$ complexity, which improves upon prior methods for nonparametric independence tests.

ABSTRACT

A sample of n generic points in the xy-plane defines a permutation that relates their ranks along the two axes. Every subset of k points similarly defines a pattern, which occurs in that permutation. The number of occurrences of small patterns in a large permutation arises in many areas, including nonparametric statistics. It is therefore desirable to count them more efficiently than the straightforward ~O(n^k) time algorithm. This work proposes new algorithms for counting patterns. We show that all patterns of order 2 and 3, as well as eight patterns of order 4, can be counted in nearly linear time. To that end, we develop an algebraic framework that we call corner tree formulas. Our approach generalizes the existing methods and allows a systematic study of their scope. Using the machinery of corner trees, we find twenty-three independent linear combinations of order-4 patterns, that can be computed in time ~O(n). We also describe an algorithm that counts one of the remaining 4-patterns, and hence all 4-patterns, in time ~O(n^(3/2)). As a practical application, we provide a nearly linear time computation of a statistic by Yanagimoto (1970), Bergsma and Dassios (2010). This statistic yields a natural and strongly consistent variant of Hoeffding's test for independence of X and Y, given a random sample as above. This improves upon the so far most efficient ~O(n^2) algorithm.

Motivation & Objective

• To develop efficient algorithms for counting small permutation patterns, which are critical in nonparametric statistics and combinatorics.
• To overcome the $ ilde{O}(n^k)$ bottleneck of naive pattern counting for $k$-patterns in permutations of size $n$.
• To generalize existing methods using an algebraic framework that systematically captures the scope of efficient pattern counting.
• To enable faster computation of rank-based statistical tests, such as Hoeffding’s and Yanagimoto’s independence tests, by accelerating pattern counting.
• To explore the computational limits of pattern counting and identify which linear combinations of patterns admit efficient algorithms.

Proposed method

• The authors introduce 'corner tree formulas'—an algebraic framework that models pattern counts as sums over tree-structured configurations of relative orderings in permutations.
• They use two-dimensional sum trees to efficiently maintain and query pairs of elements across horizontal strips, enabling fast counting of complex 4-patterns like 3241 and 3214.
• The algorithm partitions the permutation into $m = ilde{O}( binom{n}{k})$ strips and balances computation and space via a tradeoff between $ ilde{O}(n^{2-a})$ time and $ ilde{O}(n^{1+a})$ space for $a o 0.5$.
• For the 3241 pattern, they combine bottom-up scanning with box-sum queries on a sparse 2D sum tree to count valid merges of 321 and 34 pairs in $ ilde{O}(n^{3/2})$ time.
• The method generalizes to linear combinations of patterns by expressing them as sums over corner trees, allowing efficient evaluation via dynamic programming on tree structures.
• The approach is implemented in a Python package, `cornertree.py`, including a `ProductTree` class for efficient 2D range queries.

Experimental results

Research questions

• RQ1 Can pattern counts or linear combinations of them be computed in $ ilde{O}(n)$ time if they are not spanned by corner trees?
• RQ2 What is the computational complexity of computing the full 4-profile of a permutation?
• RQ3 What is the dimension of the space of pattern combinations spanned by $k$-vertex corner trees, and does it span all $k$-patterns?
• RQ4 Are there other tree-like structures beyond corner trees that yield efficient pattern counting algorithms?
• RQ5 What is the smallest $d(k)$ such that the $k$-profile can be computed in $ ilde{O}(n^{d(k)})$ time for general $k$?

Key findings

• All 2- and 3-patterns, as well as eight specific 4-patterns, can be counted in $ ilde{O}(n)$ time using corner tree formulas.
• The paper identifies twenty-three independent linear combinations of 4-patterns that can be computed in $ ilde{O}(n)$ time, extending the scope of efficient counting.
• A novel algorithm counts the 3241 pattern in $ ilde{O}(n^{3/2})$ time, which implies that all 4-patterns can be counted in the same time complexity.
• The method improves the state-of-the-art from $ ilde{O}(n^2)$ to $ ilde{O}(n^{3/2})$ for computing Yanagimoto’s independence test statistic.
• The authors compute the space of pattern combinations spanned by $k$-vertex corner trees for $k \leq 7$, showing that the number of such trees grows as $O(11.07^k)$, which is sub-exponential compared to $k!$.
• The framework enables a systematic study of efficient pattern counting and suggests that further improvements may be possible via refined case division or alternative tree structures.