[论文解读] Beyond Hungarian: Match-Free Supervision for End-to-End Object Detection

Shoumeng Qiu, Xinrun Li+1 more|arXiv (Cornell University)|Mar 9, 2026

Advanced Neural Network Applications被引用 0

一句话总结

提出基于无匹配训练的 DETR 方案，使用基于跨注意力的查询选择（CAQS）模块学习隐式的查询到对象对应关系，实现显著的训练加速和 COCO 性能提升，尤其是对大对象。

ABSTRACT

Recent DEtection TRansformer (DETR) based frameworks have achieved remarkable success in end-to-end object detection. However, the reliance on the Hungarian algorithm for bipartite matching between queries and ground truths introduces computational overhead and complicates the training dynamics. In this paper, we propose a novel matching-free training scheme for DETR-based detectors that eliminates the need for explicit heuristic matching. At the core of our approach is a dedicated Cross-Attention-based Query Selection (CAQS) module. Instead of discrete assignment, we utilize encoded ground-truth information to probe the decoder queries through a cross-attention mechanism. By minimizing the weighted error between the queried results and the ground truths, the model autonomously learns the implicit correspondences between object queries and specific targets. This learned relationship further provides supervision signals for the learning of queries. Experimental results demonstrate that our proposed method bypasses the traditional matching process, significantly enhancing training efficiency, reducing the matching latency by over 50\%, effectively eliminating the discrete matching bottleneck through differentiable correspondence learning, and also achieving superior performance compared to existing state-of-the-art methods.

研究动机与目标

• Motivate removing the costly Hungarian bipartite matching in DETR-based detectors.
• Develop a differentiable, end-to-end supervision mechanism that learns query-object correspondences.
• Design a GT-Probe module and sparse correspondence generation to provide dense-to-sparse supervision.
• Show that match-free training improves training efficiency and detection accuracy on COCO.

提出的方法

• Introduce the GT-Probe Module (GTPM) that encodes ground-truths and predicted queries, using cross-attention to produce a dense correspondence matrix A between GTs and queries.
• Apply Sparse Correspondence Generation (SCG) to convert A into a sparse, normalized assignment matrix  for stable supervision.
• Construct a Broadcast Cost matrix C from classification and geometric terms to quantify supervision signals for all GT-query pairs.
• Define Correspondence Weight Loss Lw as the element-wise product of A and C to guide the GT-Probe learning.
• Define Sparse Query Loss Lq by gating C with  to focus supervision on a selected subset of queries.
• Train with total loss Ltotal = α Lw + β Lq, balancing assignment learning and query refinement.

实验结果

研究问题

• RQ1 Detr 基检测器在不显式 Hungarian 匹配的情况下是否仍能保持或提升准确性？
• RQ2 可微分的、由 GT 驱动的对应学习是否能提升训练效率和对大量查询库的扩展性？
• RQ3 提出的基于 CAQS 的监督对不同尺度目标的检测性能有何影响，尤其是大对象？
• RQ4 SCG 的稀疏化与归一化策略对定位和整体 AP 的影响如何？

主要发现

Detector	Setting	Epoch	AP	AP50	AP75	APS	APM	APL
Deformable DETR	Baseline	20	25.4	43.4	26.3	11.2	28.5	37.1
Ours	20 epochs (ours)	20	26.1	43.5	27.1	10.7	29.1	41.3

• 所提出的方法在 20 轮训练时达到 26.1 AP，基线（Deformable DETR）为 25.4 AP，提升 +0.7 AP。
• 在无匹配方法下 AP75 提升至 27.1，比基线提高 +0.8。
• 对于大对象，AP L 从 37.1 提升到 41.3，增加 +4.2，表现出对大对象的显著改进。
• 训练时延从启用 Hungarian 基线的 53 ms 降至无匹配的 25 ms，整轮训练速度提升 >50%。
• 方法对 α 值的 AP L 提供鲁棒提升，在 α=1 时达到最佳结果，归一化稀疏性（归一为和为 1）得到 26.1 AP。
• 对小对象的 AP S 略有下降（10.7 vs 11.2），表明对极小目标存在潜在挑战。