[Paper Review] Hyper-optimized approximate contraction of tensor networks with arbitrary geometry

Johnnie Gray, Garnet Kin‐Lic Chan|arXiv (Cornell University)|Jun 14, 2022

Elasticity and Material Modeling75 references4 citations

TL;DR

This paper introduces a hyper-optimized framework for approximate tensor network contraction on arbitrary graphs, using iterative optimization over contraction trees that include both pairwise contractions and singular value decomposition-based bond compressions. By minimizing memory or computational cost during hyper-optimization, the method achieves significantly lower error and cost than hand-crafted or existing general algorithms, enabling efficient simulation of large-scale tensor networks including frustrated 3D lattices and random regular graphs.

ABSTRACT

Tensor network contraction is central to problems ranging from many-body physics to computer science. We describe how to approximate tensor network contraction through bond compression on arbitrary graphs. In particular, we introduce a hyper-optimization over the compression and contraction strategy itself to minimize error and cost. We demonstrate that our protocol outperforms both hand-crafted contraction strategies in the literature as well as recently proposed general contraction algorithms on a variety of synthetic and physical problems on regular lattices and random regular graphs. We further showcase the power of the approach by demonstrating approximate contraction of tensor networks for frustrated three-dimensional lattice partition functions, dimer counting on random regular graphs, and to access the hardness transition of random tensor network models, in graphs with many thousands of tensors.

Motivation & Objective

• To develop a general-purpose framework for approximate tensor network contraction applicable to arbitrary graph geometries, not limited to regular lattices.
• To reduce the exponential cost of exact tensor network contraction by introducing a hyper-optimization over contraction and compression sequences.
• To improve upon existing approximate contraction strategies in accuracy and efficiency, especially for physically relevant models like frustrated 3D lattices and random tensor networks.
• To demonstrate that minimizing a cost function (memory or computational cost) during hyper-optimization simultaneously reduces contraction error, enabling a trade-off between cost and accuracy.
• To explore the computational hardness transition in random tensor networks and identify regimes where approximate contraction remains effective despite complex tensor entries.

Proposed method

• The method represents approximate contraction as a directed, ordered computational tree where each node corresponds to a contraction or compression step.
• It employs a hyper-optimization loop that iteratively samples contraction trees using heuristic parameters θ and maximum bond dimension χ, guided by a cost function M (peak memory) or C (computational cost).
• Compression is performed via singular value decomposition (SVD) on bonds between tensors, with the compression order and timing optimized as part of the tree structure.
• The framework supports arbitrary graph topologies, including regular lattices, random regular graphs, and complex 3D geometries such as the pyrochlore and diamond lattices.
• The optimization process uses parametrized heuristics (e.g., Greedy algorithm) to explore the space of possible contraction trees, with hyper-parameters tuned to minimize the cost function.
• The approach is applied to both synthetic models (e.g., random tensor networks with uniform entries in [λ,1]) and physical systems (e.g., partition functions on frustrated lattices and dimer counting on random graphs).

Experimental results

Research questions

• RQ1 Can hyper-optimization over contraction trees with compression steps yield more accurate and efficient approximate tensor network contractions than hand-crafted or general-purpose algorithms on arbitrary graphs?
• RQ2 Does minimizing a cost function (memory or computational cost) during hyper-optimization lead to a simultaneous reduction in contraction error across diverse tensor network geometries?
• RQ3 What is the computational phase transition in approximate contraction of random tensor networks, and how does it relate to the sign structure and magnitude of tensor entries?
• RQ4 Can the framework efficiently contract large-scale tensor networks with tens of thousands of tensors, including those with long-range interactions or frustration?
• RQ5 To what extent can approximate contraction reduce the exponential cost of exact contraction in physically relevant models such as 3D frustrated lattices and random regular graphs?

Key findings

• The hyper-optimized approximate contraction protocol outperforms both hand-crafted contraction strategies and recent general algorithms on regular lattices and random regular graphs, achieving lower error and cost.
• On the pyrochlore lattice, the method achieves an exponential advantage over exact contraction for non-hard tensor entries, demonstrating practical scalability.
• For 3-regular random graphs, approximate contraction retains exponential scaling but with a significantly reduced pre-factor, enabling efficient computation on large networks.
• The framework successfully enables approximate contraction of 3D frustrated lattice partition functions and dimer counting on random regular graphs with up to several thousand tensors.
• A hardness transition is observed in random tensor networks: when tensor entries are uniformly distributed in [λ,1], the relative error ΔZ increases sharply as λ decreases, especially when Z values become negative or have large magnitude variations.
• The method maintains low error even in hard regimes when large bond dimensions χ are used, suggesting it approaches exact contraction in such cases, and highlights the role of tensor sign structure in contraction difficulty.