[Paper Review] Transformer-based Machine Learning for Fast SAT Solvers and Logic Synthesis

Shi Feng, Chonghan Lee+4 more|arXiv (Cornell University)|Jul 15, 2021

Formal Methods in Verification25 references4 citations

TL;DR

This paper proposes TRSAT, a Transformer-based one-shot model for solving MaxSAT problems in logic synthesis using self-attention and cross-attention mechanisms on bipartite graphs of variables and clauses. It achieves 97% completion rate on general SAT and 88.7% on DES circuits with significant speedup over heuristic and machine learning baselines by leveraging parallel message passing and iterative clause satisfaction.

ABSTRACT

CNF-based SAT and MaxSAT solvers are central to logic synthesis and verification systems. The increasing popularity of these constraint problems in electronic design automation encourages studies on different SAT problems and their properties for further computational efficiency. There has been both theoretical and practical success of modern Conflict-driven clause learning SAT solvers, which allows solving very large industrial instances in a relatively short amount of time. Recently, machine learning approaches provide a new dimension to solving this challenging problem. Neural symbolic models could serve as generic solvers that can be specialized for specific domains based on data without any changes to the structure of the model. In this work, we propose a one-shot model derived from the Transformer architecture to solve the MaxSAT problem, which is the optimization version of SAT where the goal is to satisfy the maximum number of clauses. Our model has a scale-free structure which could process varying size of instances. We use meta-path and self-attention mechanism to capture interactions among homogeneous nodes. We adopt cross-attention mechanisms on the bipartite graph to capture interactions among heterogeneous nodes. We further apply an iterative algorithm to our model to satisfy additional clauses, enabling a solution approaching that of an exact-SAT problem. The attention mechanisms leverage the parallelism for speedup. Our evaluation indicates improved speedup compared to heuristic approaches and improved completion rate compared to machine learning approaches.

Motivation & Objective

• To develop a scalable, one-shot machine learning model for solving MaxSAT problems in logic synthesis without iterative retraining.
• To improve upon heuristic SAT solvers and existing ML-based approaches by combining attention mechanisms with graph neural networks for better accuracy and speed.
• To enable parallel, scale-free processing of CNF formulas of arbitrary size using meta-path-based self-attention and cross-attention on bipartite graphs.
• To enhance solution quality by integrating an iterative refinement step that approaches exact SAT solutions.
• To evaluate the model’s performance on real-world logic synthesis benchmarks and compare it with state-of-the-art heuristic and deep learning solvers.

Proposed method

• Adapts the Transformer architecture to bipartite graphs of variables and clauses, treating them as heterogeneous nodes.
• Uses meta-paths to define self-attention between homogeneous nodes—e.g., variables sharing the same clause or clauses sharing the same variable.
• Applies cross-attention between heterogeneous nodes (variables and clauses) to propagate information across the bipartite structure.
• Employs a message-passing mechanism with learned queries, keys, and values to update node representations in parallel.
• Introduces an iterative refinement step that re-evaluates and satisfies additional clauses to improve solution quality.
• Trains the model end-to-end on synthetic and real-world CNF instances to approximate MaxSAT solutions with high completion rates.

Experimental results

Research questions

• RQ1 Can a Transformer-based model achieve high completion rates on MaxSAT problems while maintaining one-shot inference and scalability to arbitrary instance sizes?
• RQ2 How does attention-based message passing on bipartite graphs compare to sequential or RNN-based approaches in solving SAT problems?
• RQ3 To what extent can self-attention and cross-attention mechanisms capture complex variable-clause interactions in CNF formulas?
• RQ4 Can the model’s predicted partial solutions accelerate exact SAT solvers by reducing backtracking in heuristic solvers?
• RQ5 How does the model’s performance scale across diverse logic synthesis benchmarks with varying variable and clause distributions?

Key findings

• The TRSAT model achieves a 97% completion rate on general SAT problems and 88.7% on DES circuit instances, outperforming both heuristic and machine learning baselines.
• The model demonstrates significant speedup over PDP and WalkSAT, with average inference speeds consistently higher across all tested graph structures.
• Unlike RL-based models such as RLSAT, TRSAT shows stable training performance without oscillation, even when generalizing to larger graph scales.
• The parallel message-passing mechanism reduces the number of required iterations, enabling faster convergence due to richer initial feature propagation.
• The model maintains high performance across diverse CNF instances, including random 3-SAT and complex logic circuits, without architectural changes.
• The integration of iterative refinement improves solution quality, bringing the model closer to exact SAT solutions while preserving speed.