[论文解读] Symbol-Equivariant Recurrent Reasoning Models

Richard Freinschlag, Timo Bertram+3 more|arXiv (Cornell University)|Mar 2, 2026

Explainable Artificial Intelligence (XAI)被引用 0

一句话总结

SE-RRMs 在递归推理模型中强制符号置换对称性，在 Sudoku 推断上取得强劲 extrapolation，并在较少参数和较少数据增强的情况下实现与 ARC-AGI 竞争的结果。

ABSTRACT

Reasoning problems such as Sudoku and ARC-AGI remain challenging for neural networks. The structured problem solving architecture family of Recurrent Reasoning Models (RRMs), including Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), offer a compact alternative to large language models, but currently handle symbol symmetries only implicitly via costly data augmentation. We introduce Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs), which enforce permutation equivariance at the architectural level through symbol-equivariant layers, guaranteeing identical solutions under symbol or color permutations. SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize from just training on 9x9 to smaller 4x4 and larger 16x16 and 25x25 instances, to which existing RRMs cannot extrapolate. On ARC-AGI-1 and ARC-AGI-2, SE-RRMs achieve competitive performance with substantially less data augmentation and only 2 million parameters, demonstrating that explicitly encoding symmetry improves the robustness and scalability of neural reasoning. Code is available at https://github.com/ml-jku/SE-RRM.

研究动机与目标

• Motivate structured reasoning problems and the need for robust, data-efficient architectures.
• Introduce SE-RRMs that encode symbol equivariance architecturally.
• Show improved generalization to different grid sizes and competitive ARC-AGI performance with a small parameter count.

提出的方法

• Extend vanilla RRMs by introducing a symbol dimension and symbol-equivariant layers.
• Use a symbol embedding scheme where the same embedding is shared across symbols except for special tokens.
• In SE-RRM, replace the output mapping with a direct logit mapping over the symbol dimension to produce predictions.
• Incorporate task-type information via a broadcasted task-type embedding to maintain equivariance.
• Employ two Transformer-style self-attention blocks along position and symbol dimensions within SE-RRM blocks.
• Train with deep supervision and a fixed-point iterative scheme, using RoPE2d for positional encoding.

实验结果

研究问题

• RQ1 Does enforcing symbol equivariance at the architectural level improve learning efficiency and generalization on structured problems?
• RQ2 Can SE-RRMs extrapolate to Sudoku grid sizes and symbol sets unseen during training, compared to vanilla RRMs?
• RQ3 How does SE-RRM perform on ARC-AGI tasks relative to data augmentation and parameter count?
• RQ4 What are the computational trade-offs of adding a symbol dimension to RRM blocks?
• RQ5 To what extent can SE-RRMs reduce reliance on data augmentation while maintaining or improving performance?

主要发现

• SE-RRM 超越 HRM、TRM 和 GPT-OSS-20B 在 9x9 Sudoku 的全解率（FSR）及网格点精度（GPA）。
• SE-RRM 能推广至 4x4、16x16 和 25x25 的 Sudoku 题目，先前的 RRMs 无法外推；SE-RRM 在 4x4 上的 FSR 为 95.46%（92.01-97.08），GPA 为 99.15%（98.67-99.41），在 9x9 上为 GPA 93.73%（93.66-93.81）和 97.58%（97.54-97.63）；对更大网格的外推显示其他方法数据有限或无数据。
• SE-RRM 仅使用 200 万参数，显著少于许多基线，并且在 ARC-AGI 任务上对数据增强的需求较低，同时取得竞争性结果。
• 在 9x9 Sudoku 上，SE-RRM 对 4x4 的 GPA 为 51.95%？（表中对 16x16 的行显示 0（0-1.75），对 25x25 为 31.49%，注：表格细节指外推结果）
• SE-RRM 表明对称性显式编码有助于提高神经推理在结构化任务中的鲁棒性和可扩展性。