[论文解读] T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation

Xingzu Zhan, Chen Xie+3 more|arXiv (Cornell University)|Feb 1, 2026

Human Motion and Animation被引用 0

一句话总结

通过将运动周期性与显著性耦合并引入周期性差分跨模态对齐模块，稳定长文本到动作生成并提高对改写的鲁棒性，在 HumanML3D 与 KIT-ML 上实现极低 FID 与强对齐。

ABSTRACT

Text-to-motion generation, which converts motion language descriptions into coherent 3D human motion sequences, has attracted increasing attention in fields, such as avatar animation and humanoid robotic interaction. Though existing models have achieved significant fidelity, they still suffer from two core limitations: (i) They treat motion periodicity and keyframe saliency as independent factors, overlooking their coupling and causing generation drift in long sequences. (ii) They are fragile to semantically equivalent paraphrases, where minor synonym substitutions distort textual embeddings, propagating through the decoder and producing unstable or erroneous motions. In this work, we propose T2M Mamba to address these limitations by (i) proposing Periodicity-Saliency Aware Mamba, which utilizes novel algorithms for keyframe weight estimation via enhanced Density Peaks Clustering and motion periodicity estimation via FFT-accelerated autocorrelation to capture coupled dynamics with minimal computational overhead, and (ii) constructing a Periodic Differential Cross-modal Alignment Module (PDCAM) to enhance robust alignment of textual and motion embeddings. Extensive experiments on HumanML3D and KIT-ML datasets have been conducted, confirming the effectiveness of our approach, achieving an FID of 0.068 and consistent gains on all other metrics.

研究动机与目标

• 激发并解决长时间序列文本到动作生成中的不稳定性与漂移问题。
• 建模关键帧显著性与动作周期性之间的耦合以防止历史遗忘。
• 通过跨模态对齐增强提高对改写引起的嵌入漂移的鲁棒性。
• 提出高效算法，尽量降低关键帧检测和周期性估计的额外开销。

提出的方法

• 在运动片段内通过增强的密度峰值聚类检测关键帧显著性以分配自适应关键帧权重。
• 通过FFT加速的自相关、谱熵与显著性标准逐片段估计运动周期性。
• 将关键帧权重和相位编码整合到 Periodicity-Saliency Aware Mamba 中以强化关键帧和节律性。
• 开发 Periodic Differential Cross-modal Alignment Module (PDCAM) 以在时间尺度不匹配下鲁棒对齐文本和动作嵌入。
• 使用相位旋转的查询切片和微分注意力来强调跨模态判别信息，同时减小对改写的扰动影响。

Figure 1: The overview of the proposed T2M Mamba. (a) T2M Mamba. Our T2M Mamba consisting of N basic blocks aims to predict clean motion sequence (b) Inference Process. Starting from Gaussian noise, the model iteratively denoises to generate a clean motion sequence $M^{0}$ semantically aligned with

实验结果

研究问题

• RQ1 如何将关键帧显著性与动作周期性耦合以降低对历史的遗忘在长序列中的影响？
• RQ2 相位编码的周期信息是否能提升文本到动作生成的稳定性与节律感？
• RQ3 PDCAM 的跨模态对齐是否能稳健处理语义改写扰动？
• RQ4 在现有文本到动作模型中加入周期性-显著性耦合的性能提升与计算成本是多少？

主要发现

• T2M Mamba 在 HumanML3D/KIT-ML 基准上实现极低的 FID 0.068，并在各指标上持续提升。
• 消融实验表明移除关键帧权重（M）或相位编码（phi）会降低 FID 与 R-Top3，证实它们的互补作用。
• PDCAM 相比标准微分注意力显著提升跨模态对齐，提升 R-Top3 并降低 MM Dist。
• 联合使用 M 与 phi 能带来最佳的稳定性和运动保真度，在测试深度中 6 层的 Mamba 达到最优表现。
• 改写鲁棒性实验表明模型在文本微小变动下仍能保持稳定运动，解决了此前对改写的敏感性问题。

Figure 2: Illustration of our Periodicity-Saliency Aware Mamba. $\odot$ denotes dot product.