[Paper Review] A Comparison of Random Task Graph Generation Methods for Scheduling Problems

Louis-Claude Canon, Mohamad El Sayah+1 more|arXiv (Cornell University)|Feb 15, 2019

Scheduling and Optimization AlgorithmsEngineering29 references21 citations

TL;DR

This paper compares random task graph generation methods for scheduling problems, focusing on properties like mass—quantifying decomposability—to avoid biased, trivial instances. It establishes sub-exponential generic time complexity for uniformly generated decomposable instances and evaluates how generation methods impact heuristic performance, revealing critical flaws in existing approaches like Erdős-Rényi and layer-by-layer methods.

ABSTRACT

How to generate instances with relevant properties and without bias remains an open problem of critical importance for a fair comparison of heuristics. In the context of scheduling with precedence constraints, the instance consists of a task graph that determines a partial order on task executions. To avoid selecting instances among a set populated mainly with trivial ones, we rely on properties that quantify the characteristics specific to difficult instances. Among numerous identified such properties, the mass measures how much a task graph can be decomposed into smaller ones. This property, together with an in-depth analysis of existing random task graph generation methods, establishes the sub-exponential generic time complexity of the studied problem. Empirical observations on the impact of existing generation methods on scheduling heuristics concludes our study.

Motivation & Objective

• To address bias in random task graph generation that compromises fair evaluation of scheduling heuristics.
• To identify and analyze key DAG properties—especially mass—that characterize difficult, non-trivial scheduling instances.
• To formally analyze and empirically assess existing random DAG generation methods in terms of these properties.
• To evaluate the impact of generation methods on heuristic performance (MinMin, HEFT, HCPT) under unit-cost scheduling.
• To guide the design of uniform, non-biased generation methods that target hard, meaningful scheduling instances.

Proposed method

• Identifies 34 DAG properties and selects 8 for in-depth analysis, with mass as the central measure of decomposability.
• Uses formal analysis to show that uniformly generated decomposable instances have sub-exponential generic time complexity.
• Empirically evaluates five generation methods: Erdős-Rényi, layer-by-layer, random orders, fan-in/fan-out, and expansion/contraction.
• Measures properties such as length, width, number of edges, critical path, and shape standard deviation across generated graphs.
• Implements and compares scheduling heuristics (MinMin, HEFT, HCPT) on generated instances to assess performance differences.
• Uses R and Python code to generate instances and analyze results, with all data and code publicly available.

Experimental results

Research questions

• RQ1 Which DAG properties best characterize hard, non-trivial scheduling instances, and how do they affect heuristic performance?
• RQ2 How do existing random DAG generation methods perform in terms of key structural properties like mass, width, and critical path length?
• RQ3 What is the generic time complexity of scheduling problems when instances are uniformly sampled from a class defined by structural constraints?
• RQ4 Why do certain generation methods (e.g., Erdős-Rényi) fail to differentiate heuristic performance despite known differences in algorithmic behavior?
• RQ5 How can generation methods be improved to avoid bias while maintaining diversity and targeting meaningful instance classes?

Key findings

• The mass property quantifies how much a DAG can be decomposed into independent subgraphs, and it plays a critical role in determining instance difficulty.
• Uniformly generated instances with non-zero mass exhibit sub-exponential generic time complexity, indicating that decomposability affects problem hardness.
• The random orders method produces high variance in heuristic performance but lacks uniformity and is limited by discrete parameter K, reducing diversity.
• Erdős-Rényi and layer-by-layer methods fail to distinguish between MinMin and HEFT, despite HEFT being theoretically superior, indicating poor sensitivity to DAG structure.
• HCPT and MinMin show significant performance degradation on specific pathological instances: absolute makespan differences from HEFT exceed 45 and 9 respectively under n=100 tasks and |P|≤10.
• Instances with high width and large shape standard deviation (e.g., long chains with forks) expose weaknesses in MinMin and HCPT, highlighting the need for better generation control.