[Paper Review] Learning symmetric k-juntas in time n^o(k)

Mihail N. Kolountzakis, Evangelos Markakis+1 more|ArXiv.org|Apr 12, 2005

Machine Learning and Algorithms18 references3 citations

TL;DR

This paper presents the first $ n^{o(k)} $-time algorithm for learning symmetric $ k $-juntas under the uniform distribution, by proving that every non-constant, non-parity symmetric Boolean function on $ k $ variables has a non-zero Fourier coefficient of order at most $ O(k / \log k) $. The result leverages number theory, combinatorics, and Fourier analysis to establish a tight bound on the lowest non-zero Fourier degree, enabling efficient learning via Fourier sampling.

ABSTRACT

We give an algorithm for learning symmetric k-juntas (boolean functions of $n$ boolean variables which depend only on an unknown set of $k$ of these variables) in the PAC model under the uniform distribution, which runs in time n^{O(k/\log k)}. Our bound is obtained by proving the following result: Every symmetric boolean function on k variables, except for the parity and the constant functions, has a non-zero Fourier coefficient of order at least 1 and at most O(k/\log k). This improves the previously best known bound of (3/31)k, and provides the first n^{o(k)} time algorithm for learning symmetric juntas.

Motivation & Objective

• To close the gap between the trivial $ n^k $-time exhaustive search and efficient learning for symmetric $ k $-juntas.
• To determine the minimal degree $ t $ such that every non-constant, non-parity symmetric Boolean function on $ k $ variables has a non-zero Fourier coefficient of degree at most $ t $.
• To develop a novel Fourier-based learning algorithm that runs in time $ n^{O(k/\log k)} $, achieving $ n^{o(k)} $ complexity.
• To establish a tight bound on the lowest non-zero Fourier coefficient for symmetric functions, improving upon the prior $ 3k/31 $ bound.

Proposed method

• Reduce the problem to analyzing the vanishing of Fourier coefficients via Diophantine equations involving binomial coefficients.
• Prove that a certain integer-valued polynomial $ P $ is constant on $ \{0, 1, \dots, k\} $ by showing it is constant on two symmetric intervals $ \{0, \dots, t\} \cup \{k-t, \dots, k\} $ using modular arithmetic and the Siegel-Walfisz theorem.
• Extend the constancy of $ P $ from small intervals to the entire domain $ \{0, \dots, k\} $ using repeated applications of Lucas’ Theorem.
• Use a probabilistic equivalence between Fourier coefficient vanishing and equality of moments under uniform and function-induced measures to eliminate redundant case analysis.
• Apply the Fourier sampling framework: query the function on random inputs, estimate low-degree Fourier coefficients, and identify the relevant variables using the $ O(k/\log k) $ bound.
• Leverage the fact that symmetric functions depend only on Hamming weight, allowing representation by a $ (k+1) $-bit string indicating function values at each weight.

Experimental results

Research questions

• RQ1 What is the smallest $ t $ such that every non-constant, non-parity symmetric Boolean function on $ k $ variables has a non-zero Fourier coefficient of degree at most $ t $?
• RQ2 Can the Fourier-based learning algorithm for symmetric $ k $-juntas be made to run in $ n^{o(k)} $ time?
• RQ3 Is it possible to improve the $ O(k/\log k) $ bound on the lowest non-zero Fourier coefficient for symmetric functions?
• RQ4 Can the equivalence between vanishing Fourier coefficients and moment equality under different measures be used to simplify analysis in learning problems?

Key findings

• Every non-constant, non-parity symmetric Boolean function on $ k $ variables has a non-zero Fourier coefficient of degree at most $ O(k / \log k) $, improving the prior bound of $ 3k/31 $.
• The first $ n^{o(k)} $-time algorithm for learning symmetric $ k $-juntas is achieved, with runtime $ n^{O(k / \log k)} \cdot \mathrm{poly}(2^k, n, \log(1/\delta)) $.
• The proof establishes that a certain integer-valued polynomial $ P $ is constant on $ \{0, 1, \dots, k\} $, which implies the Fourier coefficient bound.
• The use of the Siegel-Walfisz theorem ensures sufficient prime density in arithmetic progressions to enable modular analysis of $ P $.
• Lucas’ Theorem is applied iteratively to extend constancy from small intervals to the entire domain, completing the proof.
• The equivalence between vanishing Fourier coefficients and moment equality under uniform and function-induced measures simplifies the analysis and reduces case distinctions.