[Paper Review] Fast Generation of Big Random Binary Trees

William B. Langdon|arXiv (Cornell University)|Jan 13, 2020

Evolutionary Algorithms and Applications16 references4 citations

TL;DR

This paper presents a highly optimized C++ implementation, random_tree(), that generates large random binary trees in linear time and space, achieving over 18 million nodes per second on a 3.60GHz CPU. It uses Knuth's shuffle to randomize a sequence of nodes and leaves, then maps it to a valid binary tree via lattice path reordering, enabling efficient generation of trees up to a billion nodes for genetic programming experiments.

ABSTRACT

random_tree() is a linear time and space C++ implementation able to create trees of up to a billion nodes for genetic programming and genetic improvement experiments. A 3.60GHz CPU can generate more than 18 million random nodes for GP program trees per second.

Motivation & Objective

• To address the inefficiency of existing random tree generation methods when scaling to very large trees in genetic programming (GP) experiments.
• To enable the creation of massive random trees (up to a billion nodes) for testing GP systems under extreme evolutionary runs.
• To optimize tree generation for modern parallel hardware by improving performance for large-scale GP and genetic improvement workloads.
• To ensure uniform sampling of tree space while maintaining computational efficiency for both initial population and subtree mutation operations.

Proposed method

• Uses Knuth's Fisher-Yates shuffle to randomize a deterministic list of n+1 leaves and n binary functions, creating a uniformly random permutation.
• Applies lattice path reordering to transform the random sequence into a valid binary tree structure that never violates stack depth constraints.
• Employs Iba's linear-time algorithm to map the re-ordered sequence into a full binary tree with n internal nodes and n+1 leaves.
• Generates the final GP program by sequentially assigning nodes using the GPquick SETNODE macro, selecting leaves or binary functions at random.
• Optimizes performance by separating tree construction and labeling passes, though notes these could be merged for better cache efficiency.
• Uses the Park-Miller PRNG for pseudo-random number generation, acknowledging limitations due to seed space constraints.

Experimental results

Research questions

• RQ1 Can random binary trees of up to a billion nodes be generated efficiently enough to support large-scale genetic programming experiments?
• RQ2 How can uniform sampling of tree space be achieved at scale without incurring prohibitive computational cost?
• RQ3 What performance gains can be achieved by rearchitecting a linear-time tree generation algorithm for modern high-speed processors?
• RQ4 To what extent does the depth of large random binary trees conform to theoretical expectations from Flajolet and Odlyzko's analysis?

Key findings

• The new random_tree() implementation generates over 18 million random nodes per second on a 3.60GHz CPU, enabling rapid creation of billion-node trees.
• The algorithm scales linearly with tree size, as confirmed by benchmarking on an Intel i7-4790 CPU using log-log plots.
• The average depth of large random binary trees closely matches the theoretical Flajolet limit, approximating √(2π|size|), with less than 2% error for trees above 32,000 nodes.
• The implementation is limited by 32-bit integer size, causing segmentation faults when attempting to generate trees near 2 billion nodes.
• Despite using a PRNG with limited seed space, the generated trees appear sufficiently random for typical GP applications.
• The method enables efficient testing of GP systems on massive trees by replacing evolved programs with randomly generated ones that mimic their statistical properties.