[Paper Review] On the Optimal Linear Contraction Order of Tree Tensor Networks, and Beyond

Mihail Stoian, Milbradt, Richard+1 more|arXiv (Cornell University)|Sep 25, 2022

Parallel Computing and Optimization Techniques4 citations

TL;DR

This paper presents TensorIKKBZ, a polynomial-time algorithm that finds the optimal linear contraction order for tree tensor networks by adapting database join ordering techniques. It proves the contraction cost function satisfies the adjacent sequence interchange property, enabling globally optimal solutions via local comparisons, and extends the method to general tensor networks via a heuristic framework with strong empirical performance.

ABSTRACT

The contraction cost of a tensor network depends on the contraction order. However, the optimal contraction ordering problem is known to be NP-hard. We show that the linear contraction ordering problem for tree tensor networks admits a polynomial-time algorithm, by drawing connections to database join ordering. The result relies on the adjacent sequence interchange property of the contraction cost, which enables a global decision of the contraction order based on local comparisons. Based on that, we specify a modified version of the IKKBZ database join ordering algorithm to find the optimal tree tensor network linear contraction order. Finally, we extend our algorithm as a heuristic to general contraction orders and arbitrary tensor network topologies.

Motivation & Objective

• To develop a polynomial-time algorithm for optimal linear contraction orders in tree tensor networks, which is NP-hard in general.
• To establish a formal connection between tensor network contraction and database join ordering, particularly the IKKBZ algorithm.
• To prove that the tensor network contraction cost function satisfies the adjacent sequence interchange (ASI) property, enabling global optimization via local decisions.
• To extend the optimal linear contraction framework into a heuristic for general contraction orders and arbitrary tensor network topologies.
• To evaluate the method empirically against state-of-the-art hypergraph partitioning-based optimizers, demonstrating superior performance and efficiency.

Proposed method

• Adapt the IKKBZ database join ordering algorithm to tensor network contraction by reformulating the contraction cost as analogous to join selectivity and cardinality.
• Prove that the tensor contraction cost function satisfies the adjacent sequence interchange (ASI) property, which allows optimal ordering through local comparisons.
• Define the contraction cost as the product of leg sizes involved in each contraction, ensuring it aligns with the computational cost of naive tensor multiplication.
• Construct a modified version of the IKKBZ algorithm—TensorIKKBZ—specifically for tree tensor networks to compute the optimal linear contraction sequence in polynomial time.
• Use TensorIKKBZ’s output as a seed for LinDP, a recent join-ordering optimizer that builds optimal contraction trees respecting a given tensor permutation.
• Apply a maximum spanning tree heuristic to generalize the approach to arbitrary tensor networks, where exact optimization is infeasible.

Experimental results

Research questions

• RQ1 Can the optimal linear contraction order for tree tensor networks be computed in polynomial time?
• RQ2 Does the tensor network contraction cost function satisfy the adjacent sequence interchange (ASI) property required for greedy optimization?
• RQ3 Can the database join ordering technique IKKBZ be adapted to achieve optimal linear contraction orders in tree tensor networks?
• RQ4 How does the performance of the proposed optimal linear contraction algorithm compare to state-of-the-art hypergraph partitioning-based optimizers in practice?
• RQ5 Can the optimal linear contraction order serve as a high-quality seed for heuristic optimization of general tensor network topologies?

Key findings

• TensorIKKBZ computes the optimal linear contraction order for tree tensor networks in polynomial time, proving that the contraction cost function satisfies the adjacent sequence interchange (ASI) property.
• For Fork-Tensor Product State (FTPS) networks, TensorIKKBZ achieves contraction costs comparable to the state-of-the-art hyperoptimizer ctg.Hyper-Par, despite being restricted to linear orders.
• LinDP, which uses TensorIKKBZ as a seed, consistently outperforms ctg.Hyper-Par by nearly an order of magnitude in contraction cost across both FTPS and Hierarchical Tucker (HT) networks.
• Both TensorIKKBZ and LinDP are significantly faster than ctg.Hyper-Par due to efficient C++ implementation and favorable time complexity, even with parallelization.
• The optimal linear contraction order from TensorIKKBZ provides a robust starting point for heuristic optimization, enabling near-optimal results in general tensor networks.
• The method is publicly available and integrated into opt_einsum and cotengra, demonstrating practical applicability and extensibility.