[论文解读] Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing

Paras Sharma|arXiv (Cornell University)|Feb 25, 2026

Neural Networks and Reservoir Computing被引用 0

一句话总结

PDNA 通过可学习的振荡脉冲和自注意力来增强闭式形式连续时间网络在连续时间序列处理中对时间 gaps 的鲁棒性，在 sMNIST 上显示出显著的多缺口增益。

ABSTRACT

We introduce PDNA (Pulse-Driven Neural Architecture), a method for augmenting continuous-time recurrent networks with learnable oscillatory dynamics that maintain internal state evolution independently of external input. Built on Closed-form Continuous-time (CfC) networks, PDNA adds two components: (1) a pulse module that generates structured oscillations $A \cdot \sin(ωt + φ(h))$ with learnable frequencies and state-dependent phase, and (2) a self-attend module that applies recurrent self-attention to the hidden state. Through a controlled ablation study on sequential MNIST (sMNIST) with five random seeds, we evaluate gap robustness -- the ability to maintain performance when portions of the input sequence are removed at test time. Our key finding is that structured oscillatory dynamics significantly improve robustness to input interruptions: the self-attend variant achieves a statistically significant 2.78 percentage point multi-gap advantage over baseline ($p = 0.041$), while the pulse variant shows a 4.62 pp advantage with large effect size (Cohen's $d = 0.87$). A noise control (random perturbation of equal magnitude) provides no benefit, confirming that the advantage is structural rather than merely dynamic. These results provide evidence that continuous-time models can benefit from biologically-inspired internal oscillatory mechanisms for temporal robustness.

研究动机与目标

• Motivate robustness to missing or interrupted inputs in continuous-time sequence models.
• Introduce PDNA, a biologically-inspired augmentation that adds structured oscillations and self-attention to CfC networks.
• Systematically evaluate gap robustness using a novel gapped evaluation protocol on sMNIST.
• Provide ablation evidence showing the structural benefit of oscillations beyond random noise.

提出的方法

• BaseCfC backbone provides closed-form continuous-time dynamics.
• Pulse module adds structured oscillations A·sin(ω t + φ(h)) with learnable per-dimension A and ω, and state-dependent phase φ(h).
• Self-attend module applies recurrent self-attention via W_self·σ(h) with learned gate β.
• PDNA combines pulse and self-attend as additive residuals to the CfC state, trained end-to-end.
• Ablation variants isolate each component to assess contribution.
• Gapped evaluation protocol tests performance when 0–30% of input is removed at test time, including a multi-gap setup.

Figure 2 : Accuracy under increasing gap severity on sMNIST (5 seeds, mean $\pm$ std bands). Pulse-augmented variants (C, E) degrade more gracefully than baseline, particularly on the multi-gap condition where scattered interruptions test recovery ability.

实验结果

研究问题

• RQ1 Does introducing structured oscillatory dynamics improve temporal robustness to input gaps in continuous-time sequence models?
• RQ2 How do pulse and self-attend modules contribute individually and jointly to gap robustness?
• RQ3 Is the observed robustness due to structured oscillations rather than random perturbations?
• RQ4 What are the learnable parameters (frequencies, amplitudes, phase) learned by the pulse and how do they behave?
• RQ5 How does PDNA affect standard (ungapped) performance and computational/resource overhead?

主要发现

Variant	sMNIST (test accuracy %)	Gap 0%	Gap 5%	Gap 15%	Gap 30%	Multi-gap Accuracy	Degradation (%)
A. Baseline CfC	97.82 ±0.12	97.82	94.88	48.35	28.51	88.24	69.31 ± 5.02
B. CfC + Noise	97.78 ±0.20	97.78	94.60	49.56	29.78	88.01	68.00 ± 4.78
C. CfC + Pulse	97.96 ±0.14	97.96	95.82	48.27	29.58	92.86	68.38 ± 3.57
D. CfC + SelfAttend	97.89 ±0.21	97.89	95.49	52.24	28.46	91.02	69.43 ± 2.17
E. Full PDNA	97.93 ±0.16	97.93	95.28	49.43	29.71	91.96	68.21 ± 3.05

• Structured oscillatory dynamics significantly improve gap robustness compared with baseline CfC and random-noise controls.
• On multi-gap evaluation, the pulse variant achieves 92.86% accuracy vs baseline 88.24% (Δ=+4.62 pp, Cohen’s d≈0.87).
• The self-attend variant reaches 91.02% multi-gap accuracy with statistical significance over baseline (p=0.041, d≈1.33).
• Noise perturbation provides no benefit, supporting that benefits are structural rather than due to any non-zero dynamics during gaps (gap-5%: +1.22 pp over noise, p=0.013).
• Learned pulse parameters show α grows from 0.01 to ~0.66 and frequencies span two orders of magnitude, indicating active utilization of oscillatory dynamics (ω mean≈2.17, median≈1.02).
• Full PDNA achieves 91.96% multi-gap accuracy with low variance (~±1.54%).
• Computational overhead is modest (≈38% more parameters, ≈5% wall-time).

(a) Learned frequency distribution $\omega$ .