[Paper Review] The role of delay-times in delay-based Photonic Reservoir Computing

Tim Hülser, Felix Köster+2 more|arXiv (Cornell University)|Dec 22, 2021

Neural Networks and Reservoir Computing4 citations

TL;DR

This paper challenges the misconception that delay-times in photonic reservoir computers must align with the input clock-cycle, demonstrating instead that arbitrary delay-times significantly enhance performance. By tuning delays in multi-delay systems—particularly through self-feedback and coupling delays—it achieves unprecedented memory capacity and a 65% reduction in NARMA10 error (to NRMSE ≈ 0.1), proving delay-times are critical, tunable parameters for task-specific optimization in delay-based reservoir computing.

ABSTRACT

Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However,there is still an misconception about the relationship between the delay-time and the input clock-cycle which has noticeable consequences for the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that there is no predefined relationship between these two times-scales. Further, we discuss ways to improve the computing performance of a reservoir formed by delay-coupled oscillators and show the crucial impact of delay-time tuning in those multi-delay systems.

Motivation & Objective

• To correct the widespread misconception that delay-times in delay-based reservoir computing must be synchronized with the input clock-cycle.
• To demonstrate that delay-times are not predetermined but can be tuned to optimize computing performance.
• To investigate how multiple delays—especially self-feedback and coupling delays—enhance memory capacity and task-specific performance.
• To show that resonant or desynchronized delay configurations are suboptimal for tasks like NARMA10, and that optimal tuning yields superior results.

Proposed method

• The study employs a time-multiplexed reservoir computing framework, modeling the reservoir as a dynamical system with time-delayed feedback rather than a network.
• It uses a system of two delay-coupled oscillators with independently tunable self-delay and coupling delay to explore memory capacity and task performance.
• The linear memory capacity is calculated using the standard reservoir computing formalism, with input terms analyzed over varying delay-times.
• The NARMA10 task is used as a benchmark, with performance measured via normalized root mean square error (NRMSE) across different delay configurations.
• The authors simulate and analyze the system’s response across a range of delay-times, identifying resonant and non-resonant regimes.
• They compare single-delay and multi-delay systems, showing that a second delay can fill memory capacity gaps and enable independent tuning of short- and long-term memory.

Experimental results

Research questions

• RQ1 Is there a predefined relationship between the input clock-cycle and the delay-time in delay-based reservoir computing?
• RQ2 Can tuning delay-times beyond resonance or desynchronization improve computing performance?
• RQ3 How does introducing a second delay (self-feedback or coupling) affect the memory capacity and task performance in multi-delay reservoir systems?
• RQ4 What role do resonances between delay-times and the input clock-cycle play in determining reservoir performance?
• RQ5 Can delay-times be independently tuned to optimize both short- and long-term memory for complex tasks like NARMA10?

Key findings

• The NARMA10 error (NRMSE) drops to approximately 0.1 when the self-delay is tuned to around 1000–2000 time units, representing a significant performance improvement over standard configurations.
• Performance degrades sharply when the self-delay exceeds 2000 time units, due to loss of memory for the earliest required input terms in the NARMA10 time series.
• The memory capacity corresponding to the critical input terms $u_{-i}u_{-i-9}$ increases with self-delay up to an optimal point, after which it declines due to memory gaps.
• Adding a second delay—especially as a self-feedback term—enables independent tuning of short- and long-term memory, filling gaps present in single-delay systems.
• Resonance between delay-times and the input clock-cycle is generally detrimental to performance, contradicting common assumptions in the literature.
• The optimal delay configuration for NARMA10 is neither resonant nor desynchronized, but lies in between, demonstrating that arbitrary delay-times can yield superior results.