[Paper Review] Toward bio-inspired information processing with networks of nano-scale switching elements

Zoran Konkoli, Göran Wendin|arXiv (Cornell University)|Nov 25, 2013

Advanced Memory and Neural Computing46 references3 citations

TL;DR

This paper proposes reservoir computing (RC) as a bio-inspired paradigm for scalable information processing using networks of nanoscale switching elements, such as memristors, leveraging their nonlinear, history-dependent dynamics to create a dynamic reservoir that mixes and transforms input signals. The key contribution is demonstrating that such networks can serve as high-capacity computational substrates without training the internal connections, relying instead on a trainable readout layer.

ABSTRACT

Unconventional computing explores multi-scale platforms connecting molecular-scale devices into networks for the development of scalable neuromorphic architectures, often based on new materials and components with new functionalities. We review some work investigating the functionalities of locally connected networks of different types of switching elements as computational substrates. In particular, we discuss reservoir computing with networks of nonlinear nanoscale components. In usual neuromorphic paradigms, the network synaptic weights are adjusted as a result of a training/learning process. In reservoir computing, the non-linear network acts as a dynamical system mixing and spreading the input signals over a large state space, and only a readout layer is trained. We illustrate the most important concepts with a few examples, featuring memristor networks with time-dependent and history dependent resistances.

Motivation & Objective

• To investigate the computational potential of locally connected networks of nanoscale switching elements as unconventional computing substrates.
• To explore reservoir computing (RC) as a paradigm that avoids training internal synaptic weights, instead training only a readout layer.
• To assess the computational capacity of memristor networks with time- and history-dependent resistances as dynamical systems.
• To identify measurable network-level properties—such as harmonic generation or correlation functions—that indicate high-quality reservoir behavior.
• To bridge neuromorphic computing with molecular-scale components by demonstrating feasibility in nanofabricated networks of organic transistors and functionalized nanoparticles.

Proposed method

• Utilizes networks of nonlinear nanoscale switching elements, particularly memristors with history-dependent resistance, as the core computational substrate.
• Applies reservoir computing principles: input signals are fed into a fixed, nonlinear dynamical system (the reservoir), which transforms them into a high-dimensional state space.
• Employs a trainable readout layer to extract relevant information from the reservoir’s state, avoiding the need to train internal connections.
• Analyzes the reservoir’s quality via nonlinear frequency response functions, testing whether outputs deviate from linear combinations of delayed inputs.
• Proposes measuring inter-node voltage differences or correlation functions (e.g., ⟨V_int,j(t′)V_int,i(t)⟩) as experimental proxies for reservoir state dynamics.
• Evaluates computational capacity through theoretical frameworks linking criticality, phase transitions, and chaotic dynamics in complex systems.

Experimental results

Research questions

• RQ1 Can networks of nanoscale switching elements with memory and nonlinearity serve as effective computational substrates for information processing?
• RQ2 To what extent does the nonlinear frequency response of a memristor network indicate high-quality reservoir behavior?
• RQ3 How can the computational capacity of such networks be quantified and optimized without training internal connections?
• RQ4 What measurable physical quantities (e.g., voltage correlations, harmonic generation) can serve as proxies for reservoir state quality?
• RQ5 Can reservoir computing with nanoscale components overcome limitations of conventional digital and neuromorphic computing in terms of scalability and energy efficiency?

Key findings

• Memristor networks with time- and history-dependent resistances exhibit strong nonlinear dynamics suitable for reservoir computing, enabling complex signal mixing and transformation.
• The reservoir’s ability to generate nonlinear frequency responses beyond linear combinations of delayed inputs indicates high-quality dynamic state generation.
• Harmonic generation and higher-order correlation functions between nodes are proposed as experimental indicators of reservoir quality and computational capacity.
• Theoretical analysis suggests that proximity to critical behavior and chaotic dynamics may enhance computational capacity in such networks.
• Reservoir computing offers a robust alternative to traditional neuromorphic architectures by decoupling the complex training of synaptic weights from the reservoir dynamics.
• Experimental feasibility is supported by nanofabricated systems such as organic transistors (NOMFET) and self-assembled gold nanoparticle networks with molecular switches.