An organized view of reservoir computing: a perspective on theory and technology development

Reservoir computing is an unconventional computing paradigm that uses system complexity and dynamics as a computational medium. Currently, it is the leading computational paradigm in the fields of unconventional in materia computing. This review briefly outlines the theory behind the term ‘reservoir computing,’ presents the basis for the evaluation of reservoirs, and presents a cultural reference of reservoir computing in a haiku. The summary highlights recent advances in physical reservoir computing and points out the importance of the drive, usually neglected in physical implementations of reservoir computing. However, drive signals may further simplify the training of reservoirs’ readout layer training, thus contributing to improved performance of reservoir computer performance.


Introduction
The current development of artificial intelligence (AI) is possible due to enormous progress in digital computing.New processor architectures, improved fabrication technologies, and progress in thermal management enables massive parallel computation, which the general audience perceives as intelligent.Current excitement about Chat GPT and other similar systems is the best illustration of the chances-and also threats-related AI approach.In principle, most of the currently enjoyed AI tools are based on a machine-learning approach; the quality and performance of these systems is related to their size and training protocols.
Training of an artificial neural network is a tedious and energy/time-consuming process, in which synaptic weights of all connections within the network must be modified according to the desired output.This implies, in simple cases, solving sets of millions of linear equations, optimizing the network architecture and activation function of nodes, and repeating these steps until the output meets the expected one.Naturally, the larger the network, the higher the cost of training, both in terms of energy and time.10][11] There are two main issues limiting computing efficiency: the von Neumann bottleneck and the informational "black hole" problem.The problem with central processing unitmemory information transfer can be solved by the in-memory computing approach, using, for example, memristors or other similar devices. 6,12)The "black hole" problem is related to a big data issue-humankind and all widely used information technology devices produce and store so much data that it can never be accessed and processed in a reasonable way, so most of the data stored are hidden behind the informational black hole event horizon. 13)This situation was already envisioned in 1928 by American writer H.P. Lovecraft in his novelette The Call of Cthulhu: 14) "The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents.We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age." In recent years, significant advancements have been witnessed in the development of artificial neural networks (ANNs), playing a pivotal role across diverse applications, including object detection, security, natural language processing, autonomous driving, and so on.ANNs are broadly classified into feedforward neural networks (FNNs), exemplified by the convolutional neural network, adept at handling static spatial patterns, and recurrent neural networks (RNNs) as typical temporal neural networks, designed for processing temporal signals by encapsulating historical information within internal states to facilitate short-term memory.The training of RNNs poses challenges attributable to the complexities of exploding or vanishing gradients inherent in recurrent structures.
To address this issue, reservoir computing (RC) emerged as a machine-learning framework and evolved through the amalgamation of specific recurrent neural network models, encompassing liquid state machines (LSMs), echo state networks (ESNs), and delay-feedback reservoirs. 15)RC originated from ESNs by Jaeger 16) in 2001 and the LSM by Maass et al. 17) in 2002.An ESN employs sparsely connected sigmoidal-response neurons in its random network architecture, while an LSM utilizes spiking neurons.Despite architectural differences, both methods leverage reservoirs of neurons to effectively address time-series problems, leading to their classification under RC. 18)The concept of an LSM (or echo state machine) helps us to both understand and to harness the role of stochasticity and dynamics for computation.
RC simplifies training by concentrating on training only the weights connected to the output layer (also called the readout layer).][21] BPTT"s efficacy lies in its versatility across network architectures, employing automatic differentiation, and taking advantage of graphics processing unit acceleration.Moreover, BPTT is implemented to train longshort-term memory, gated recurrent units, and unitary RNNs, allowing adjustment of parameters like forward-pass time steps and the number of previous time steps for backpropagation.Vlachas et al. 22) compared the efficiency of RNNs with RC and BPTT for forecasting spatio-temporal dynamics in complex systems.RC excels with full-state dynamics, outperforming BPTT in predictive accuracy and requiring less training time.However, for reduced-order data, BPTT-trained RNNs demonstrate superior stability and forecasting abilities. 19)t is not possible to decrease the total cost of computation below a certain limit; however, smart interplay and trade-off between cost and performance is possible. 23)In classical neural networks, the cost of training is related to the size of the network (more precisely, the number of synaptic connections within the network).On the other hand, the concept of RC, 24) which implies an unknown (and untrainable) inner architecture, followed by a relatively simple trainable perceptron (linear FNN) is a tempting alternative. 25)It seems simple, but the application of randomly arranged unknown nodes of a network is not a trivial task.This does not violate the "no free lunch" theorem due to preserved complexity and utilizing reservoir dynamics for computation instead of training an ANN. 26)35) From a more trivial side, the application of RC may increase sensitivity and selectivity of chemosensors.[36][37][38][39][40] Physical RC brings a completely new perspective to information processing.It utilizes the internal dynamics (spontaneous or stimulated) 41) of physical systems as well as their nonlinear responsiveness [42][43][44] for computation.There is a plethora of systems that can be used in this context, provided that they present significant dynamic properties. In thewords of Zoran Konkoli, "even rocks can compute," but the complexity of computation, which can be performed in reasonable time with a such system, is rather low.45,46) The field of in materia neuromorphic computing, despite tremendous activity of numerous research groups, is still in its infancy.Replication of a complex biological information-processing circuitry of even the simplest neural system is out of reach for current technology.There are, however, many successful constructs mimicking neutrons and synapses, and processes like sensory integration and nociception.[47][48][49][50][51][52][53][54][55] Physical neuromorphic computing is slowly coming into reality. 56,57)urthermore, RC seems to be a perfect tool to understand the relation between the connectome (the connectivity map between all neurons in the nervous system) and the cognitive abilities of the neural system.58,59) Whereas the vision of a full understanding of the human brain is still far-fetched, the development of new tools will be helpful for a better understanding of ANNs and also physical neuromimetic systems.[60][61][62]

Reservoir computing in dynamic systems
In principle, any dynamic system can be considered a computational engine; however, its performance strongly depends on the complexity of its dynamics.The reservoir can be understood as a function F acting on an input data space Ω and transforming it into the output space Ω′: W  W F: ' .

( )
This transformation ability of the phase space fulfills the definition of a filter.The reservoir system (the reservoir itself and a trainable output layer) removes the unwanted part of the input data, thus generating the output.In the most general case, the input set u, which is a bounded subset of ℝ, is mapped into Ì , n   i.e. the filtration of the input data is carried out by increasing the dimensionality of the input dataset.Finally, a readout layer (e.g.trained linear perceptron) takes states of the selected nodes of the reservoir and generates the output.This is also called the output layer, by analogy to ANNs.These two terms can be used interchangeably.The term "output layer" is usually used in neural network domains-as the final layer, with a unique set of weights, which contributes to the final result prediction.In RC, the final layer is denoted as the "readout layer"-it translates the reservoir output (by means of linear transformation) into one of the desired class.The purpose of both layers is the same: mapping high-dimensional output into a much lower number of categories.The difference between these definitions is the fact that the readout layer can be an external part from the reservoir itself, whereas the output layer of a neural network consists typically of the same type of nodes and connections.In effect, the reservoir separates the input data into distinct categories that do not overlap by temporarily increasing the dimensionality of the data space (Fig. 1). 63)Therefore, the target projection space subset can be considered as a Tikhonov space-a topological space the elements of which can be separated by a continuous function, and a distance between elements of the space can be defined. 64)Therefore, the distance between categories is considered as an abstract measure of the performance of the reservoir.Formally, the dynamic mapping of the input space in the reservoir states can be described by Eqs. ( 2) and (3): where f is a nonlinear activation function (e.g.Heaviside, or hyperbolic tangent functions), u n is the input signal, and q n is the external drive.The matrices W in and W drive denote the weight matrices responsible for mapping the input and the drive signal into the reservoir.Matrix W denotes the connection weight matrix controlling the internal dynamics of the reservoir.The scaling parameters α, β, γ, and δ describe the leakage of the previous state of the reservoir to the current one and the sensitivity to the input, drive, and reservoir history, respectively.The computation output R in the simplest case may be given by the trained perceptron, taking a subspace of the reservoir state as input (4): where y n is a well-defined subset of x n , i.e.Ì y x n n , and W out is the trained weight matrix.
More details of the formal description of reservoirs are given in dedicated papers. 39,65,66)Recently, more advanced approaches to RC move the nonlinear activation function from the reservoir to the output layer. 67)This is an important advancement, as it significantly accelerates with numerical calculations in the case of software implementation of RC.This is, however, not so relevant for physical systems, where the nonlinearity of the reservoir is embedded in the properties of the computations medium. 68)In this case, however, management of delayed feedback (and the introduction of multiple delays into a single physical system) is of crucial importance. 66,69)e performance of the reservoir can be preliminarily evaluated by spectral analysis of weight matrices.This can be done on various numerical models in which the dimensionality of the reservoir is known; however, it cannot be directly performed on physical reservoir systems.The spectral radii of the input and reservoir matrices measure the performance of the reservoir.Values that are too high [especially the spectral radius of W, r W ( )] result in chaotic dynamics of the system, whereas low values result in quickly fading oscillations.The same concern the input matrix-a too high spectral radius results in extreme separation and lack of categorization, as a pair of very similar input vectors is mapped into two distant points of the reservoir space.
Physical RC can be simply described as an echo state property of the physical system.In the most simplistic realization, any input applied to the system results an in out-of-equilibrium state, which evolves to an initial (or significantly different) equilibrium state.This case corresponds to reservoirs with very a low spectral radius of ], which, despite high sensitivity to the input, cannot present complex dynamic patterns [Fig.2(a)].In a more complex case, the reservoir is a dynamic system (a) (b) (c) Fig. 1.A simple dataset in which linear separation by a perceptron is possible (a), compared with a complex data space that cannot be linearly separated (b).The transformation of the higher-dimensional input data space into the space by the reservoir enables linear separation of the dataset into the desired categories (c).Adapted from Ref. 63.
In simple terms, RC can be related to a famous haiku by Matsuo Bashō: 古池や蛙飛びこむ水の音 (hiragana: ふるいけやかごずとびこむみずのおと; romaji: furuike ya kawazu tobikomu mizu no oto; English: "An old pond, frog jumps in, splash!").In this beautiful piece of Japanese poetry, an old pond (古池や) symbolizes the reservoir itself, in which the internal dynamics (水の音) can be disturbed by an input or external stimulus, depicted as a jumping frog (蛙飛びこむ).A graphical illustration of this process, as well as of Bashō"s haiku, is shown in Fig. 3.
Even with this simplistic description, the real power of RC in the classification of unknown inputs may be observed.Following Bashō's analogy, each object thrown into the water (of different masses/shapes) would induce different patterns on the surface.This can be understood as a translation of key properties of objects under consideration into a new observable present in a different phase space and filtration of any irrelevant information (e.g.color in this particular case).The computation power of any reservoir is determined by its complexity: the simplest systems (such as a rock, which was quoted at the beginning of this paper) cannot perform much computation, but highly complex dynamic systems may have high memory volume and high computation capability. 44,70)ime-and space-multiplexing may further boost their power. 71,72)

Physical reservoir computing
If the reservoir parameters are properly set, RC brings forth several notable advantages.Training in RC is simplified, primarily focusing on the readout part, resulting in swift, stable learning while requiring fewer parameter adjustments than conventional methods.The framework excels in multitasking scenarios, ensuring that tasks can be performed concurrently without interference or the risk of forgetting previous learning.
Additionally, RC provides flexibility in choosing reservoirs, allowing the utilization of diverse dynamical systems.This versatility allows users to tailor the system to their specific needs and preferences.In the context of conventional RC, a stable software framework within RNNs is characterized by interconnected nodes. 73)RNNs have the potential to form a reservoir computer, where the weights of the recurrent network are initialized randomly and remain untrained.Meanwhile, the weights of a simple output layer undergo adjustments to train the network for a specific desired output.In contrast, a physical reservoir is a tangible structure, completely separate from neural networks.The former encapsulates a conceptual framework within the software domain, while the latter is associated with tangible real-world infrastructure, serving as a potential candidate for an unconventional computing paradigm.Appeltant and colleagues present a new architecture that minimizes the typically necessary multitude of elements to a single nonlinear node featuring delayed feedback that minimizes the number of vital elements in a conventional reservoir. 74)Research has actively explored electronic circuits for RC systems, with the aim of low-cost machine-learning devices.Although existing ANNs and neuromorphic circuits can serve as electronic reservoirs, efforts have focused on simpler configurations to enhance energy efficiency, computational speed, and robustness to hardware imperfections.
Achieving a cohesive perspective involves classifying physical reservoir architectures into three types: a single nonlinear node with delayed feedback, network structures, and an excitable continuous medium. 74)However, formulating a comprehensive design guide for each physical reservoir type remains challenging due to the intricate influence of factors like system architecture, physical attributes, and signal-processing methods on computational performance in physical RC (PRC) systems.
Implementing an entire RNN physically involves mapping the network's architecture onto a physical system using electronic or photonic components to emulate the synaptic connections and dynamics of the neurons.While technically possible, this approach presents significant challenges due to the complexity and scalability of replicating the intricate connectivity and dynamic behavior of neural networks.Specialized hardware and precise engineering may be required, making it a resource-intensive task.
An alternative approach is to focus on implementing a single node or a small subset of nodes physically, while the rest are simulated in software.This reduces the complexity of the physical implementation while still harnessing the benefits of PRC.Although more feasible than replicating an entire RNN, challenges remain in achieving accurate emulation of the node's behavior.Ensuring that the physical node exhibits the desired dynamic properties and interactions can be technically demanding.
In both cases, physical constraints, such as noise, nonidealities, and limitations in precision, need to be considered.Despite these challenges, PRC offers the advantage of utilizing the intrinsic dynamics of physical systems for computation, potentially providing unique capabilities compared to traditional digital implementations.Successful implementation requires careful consideration of the physical constraints and desired dynamic properties of the neural network.The available physical reservoirs currently include diverse implementations, such as single-node reservoirs using analog circuits, RC systems with field-programmable gate arrays, designs for RC devices using very large-scale integrated circuits, ionic liquids, 75,76) soft robotics, 77) sensory devices, 78) photonics, 79) spintronics, 80) nanoscale materials and devices, 81) and quantum reservoirs. 82)ertain physical reservoirs show potential to expedite data processing, in contrast to software-based RC.This hardwarecentric approach addresses power consumption concerns in machine-learning devices and the RC framework proving adept at cost-effective computation.Edge computing emerges as a paradigm advantageous for real-time tasks, and the significance of machine-learning hardware, including RC hardware, is anticipated to rise, facilitating efficient edge computing for data processing.Despite this, optimizing physical RC systems presents challenges dictated by practical constraints.To effectively perform tasks, a reservoir must have key characteristics.First, it is required to transform nonlinear input signals into a high-dimensional state space using numerous interconnected reservoir nodes with recurrent nonlinear dynamics.Traditionally, hundreds or thousands of such nodes are used for optimal performance.Second, the reservoir's dynamics should exhibit fading memory, influenced by recent inputs rather than those from the distant past, which is crucial for processing temporal sequences.
Well-known models of nonlinear dynamical systems, such as delayed dynamical systems, cellular automata, and coupled oscillators, are generally considered software-based models.They are mathematical abstractions implemented through software or mathematical algorithms.The singlenode reservoir by Appeltant et al. was implemented and tested through software simulations, showcasing innovation in architectural design and the exploitation of dynamic properties in a software environment. 74)reakthroughs in tasks such as pattern recognition leverage abundant data and high-performance computing devices.Current ANNs lack an ideal hardware platform that fully implements physical neurons and synaptic links, hindering speed and energy efficiency.Ongoing efforts seek novel hardware substrates for ANN concepts that match the efficiency of the human brain in learning and information processing.
Since 2011, electronic nonlinear delay systems, optoelectronic systems, and all-optical nonlinear delay systems have demonstrated fully implemented analog reservoirs, enabling the physical realization of ANNs with a large number of neurons. 79,83,84)Photonic delay systems have revolutionized hardware implementation, aligning with the fundamental principles of RC and offering significant degrees of freedom for information processing in analog systems. 79)he implementation of RC into physical substrates required exploring spatial dimensions for the RNN component.A demonstration using water tank waves showed limited processing performance, while a numerical demonstration with semiconductor optical amplifiers followed. 79)However, progress in spatio-temporal systems was slow, and RC implementation based on delay systems gained popularity more rapidly.
Nonlinear delay systems have garnered attention for their dual impact on dynamic behavior-either stabilizing or destabilizing systems through parameters like feedback strength. 74)Previously considered a nuisance, these systems are now recognized as valuable resources, as seen in configurations like semiconductor lasers with delayed feedback.This article explores the use of the rich dynamics of delay systems for processing time-dependent signals, and introduces modifications to RC. RC, inspired by the brain"s information processing, excels in challenging tasks like chaotic time-series prediction and speech recognition by mimicking neuronal networks.Its hardware realization, such as fixed connections, enables training difficulties to be overcome, allowing generalization in processing unseen inputs or classifying them based on learned patterns.
PRC offers potential as a computing system if it outperforms traditional computers in terms of speed or energy efficiency.Micro-mechanical oscillators with PRC, especially, could serve dual purposes in sensing and computing, creating innovative devices for distributed sensing or control applications. 85)Dion et al. present a microfabricated silicon beam with nonlinear dynamics, coupled with a feedback mechanism, demonstrating a micro-electrical-mechanical system reservoir computer trained to process bit streams and classify spoken words.This small and energy-efficient computing device has the potential to function as both a sensor and a computer, encoding information in the mechanical domain.

Spintronic reservoir computing
In the context of spintronics, various materials and devices play crucial roles in the advancement of computing capabilities.Ferromagnetic metallic films, governed by magnetic anisotropy energy, exhibit nonvolatility, making them suitable for magnetic random access memory devices. 80)This property is used in PRC within spintronics, particularly through magnetic tunnel junction devices, which detect changes in magnetization via the tunneling magnetoresistance effect.These devices, composed of ferromagnetic and dielectric thin films, provide electrically controllable, nonvolatile, and highdensity memory.Their precessional magnetization dynamics offer short-term memory effects and nonlinear characteristics, making them valuable for computational tasks.
Spintronics-based reservoirs, especially those that incorporate skyrmions, present opportunities for integration into existing complementary metal-oxide-semiconductor (CMOS) devices with efficient low-power responses and tunable properties. 86)Msiska et al. introduced a nanosecondscale multichannel skyrmion reservoir for pattern recognition, achieving outstanding accuracy in spoken digit speech recognition. 87)Similarly, Liu et al. proposed a stackable reservoir system using ferroelectric α-In 2 Se 3 devices, demonstrating impressive memory capacity and computing capability for advanced neuromorphic computing with emerging two-dimensional materials. 88)Nakane et al. delved into spin-wave-based RC, emphasizing its potential for edgecomputing applications (Fig. 4). 89)Their exploration involved spin-wave dynamics in a continuous magnetic garnet film with a stripe domain structure, showcasing the interplay between spin-wave behavior and RC capabilities.Spatially arranged electrodes detect spin-vector outputs with diverse nonlinear characteristics, achieving remarkable performance in temporal exclusive-OR problems.
Within the realm of spintronics, a broad spectrum of materials and devices contribute to the advancement of computational capabilities.From ferromagnetic metallic films with magnetic anisotropy energy to emerging technologies like skyrmion reservoirs and ferroelectric devices, the diversity of materials in spintronics has expanded from traditional solid-state components to innovative liquid-based systems.For example, the utilization of a Fe 3 O 4 water-based ferrofluid (FF) by Crepaldi et al. 90) introduces a dynamic element into this landscape, drawing parallels with solid-state memristors (Fig. 5).This FF exhibits intricate behavior influenced by Brownian motions and electrical polarizability of surfactant molecules, offering both fading memory and long-term plasticity.Despite these advantages, challenges in maintaining its dynamics over time are addressed through a specific "reset" sequence, which may also be considered as a kind of system programming or training.This tailored approach mitigates variations and maintains stability in the FF"s behavior, thereby contributing to the exploration of novel computing elements in the evolving field of spintronics.These diverse materials, from solids to liquids, showcase the versatility and potential of spintronics in shaping the future of computational technology.

Ionic liquids
Ionic liquids (ILs), renowned for their expansive potential window and tunable properties, represent a compelling avenue for systematic investigations in various scientific domains.In the realm of spintronics, Matsuo et al. 76) have introduced a groundbreaking physical reservoir device that harnesses faradaic currents arising from the redox reactions of metal ions within ionic liquids.Through the application of triangular voltage pulses, which symbolize binary sequences, the study systematically evaluates the impact of faradaic current on short-term memory and parity check tasks.The findings not only underscore the advantages of faradaic current for short-term memory, but also emphasize its role in nonlinear conversion within physical reservoir devices.This research provides valuable information for the design and control of such systems, as illustrated in Fig. 6.
In a related study by Sato et al., 75) the efficiency of information processing in machine learning for the classification of electrocardiogram signal waves was enhanced using 1-butyl-3-methylimidazolium bis(trifluoromethylsulfonyl) amide ([BMIM][Tf 2 N]) containing 0.4 M Cu(Tf 2 N) 2 .The evaluation processes for short-term memory (STM) and nonlinear auto-regressive moving average (NARMA2) tasks within IL-based physical reservoir devices (IL-PRDs) are meticulously depicted in Fig. 7.This work not only contributes to the understanding of information-processing mechanisms in IL-based reservoir devices but also exemplifies the potential of ILs, such as [BMIM][Tf2N], to optimize machine-learning tasks through their unique electrochemical properties.

Memristors
Memristors, known for their resistive switching properties, have made significant strides in integrating ANNs within RC systems, using their dynamic and nonlinear characteristics.Challenges persist in fine-tuning reservoir states and minimizing additional read operations for enhanced system performance and speed.Reservoir computers comprise a dynamic reservoir and a static readout.Tong et al. 91) applied two parallel memristive devices as a reservoir component in waveform and electrocardiogram classification tasks.Their study proposes a basic circuit configuration for pattern classification by physical reservoir, emphasizing the potential benefits of memristive systems with intrinsic nonlinearity.The nonlinearity and input history dependence are explored by replacing memristors with normal resistors, revealing the superior performance of memristors in achieving high computational accuracy.The study also investigates the impact of using a single memristor, indicating that at least two memristors with different responsive characteristics are essential for optimal performance in pattern classification tasks.The memristive circuit reservoir, comprising two memristors and three additional circuit elements (a resistor, an inductor, and a capacitor), is identified as a minimal but

050803-7
© 2024 The Author(s).Published on behalf of The Japan Society of Applied Physics by IOP Publishing Ltd effective configuration (Fig. 8).The description of the memristive circuit reservoir depicted in Fig. 8 is depicted through the following equations:    can be expressed as follows: Here, y(n) denotes the output vector )) is the reservoir state vector.Future work involves experimental validation, assessment of energy efficiency, and further improvement of classification accuracy through the exploration of different memristor models.
Memristors show advantages, including nonlinearity and input history-dependent reactions, rendering them highly suitable for tackling challenges associated with linearly inseparable problems in time series data analysis. 92)Previous studies have underlined the promising capabilities of memristive reservoirs in temporal pattern recognition.This potential manifests itself in two primary categories: memristor networks (Fig. 9) and memristor arrays (Fig. 10).
Przyczyna et al. introduced a network of nanodevices comprising four memristors and a differential amplifier for the detection of epileptic seizures (Fig. 11).Feedback loop evolution enhances classification accuracy, and signal transformation alters complexity parameters, contributing to improved classification scores. 94)mong memristor-based RC systems, a variety of possible solutions, based on metal oxides, have been proposed through the literature.This is mostly because nonstochiometric oxides (from TiO 2 , WO 2 , HfO 2 ) have long been known to demonstrate memristive properties. 95,96)Additionally, they are characterized not only by a nonlinear response but, as their mechanism is based on charge carrier migration, they possess the volatility necessary for RC systems.
In some cases, cross-points (memristors) are made from a multilayer oxide configuration-such as Ti/TiO x / TaO y /Pt. 97)The RC system is denoted as a dynamic one -the signal is incorporated as a temporal sequence.In tasks such as waveform classification, the working configuration consists of several parallel memristor-based reservoirs, controlled by a mask process that tunes critical parameters.The system achieves good performance, with a low word error rate of 0.4% in spoken digit recognition and a normalized root mean square error of 0.046 in time- 050803-9 © 2024 The Author(s).Published on behalf of The Japan Society of Applied Physics by IOP Publishing Ltd

PROGRESS REVIEW
series prediction of the Hénon map for time-series prediction tasks, surpassing most previously reported hardware and software-based reservoir solutions.The schematic in Fig. 12 depicts the configuration of a dynamic memristorbased parallel RC system and also one of the results for the waveform classification task.

Nanotube/nanowire reservoirs
Realization of a brain-like connection type is straightforward in nanowire networks as they already consist of interconnected nonlinear processing nodes.Such a blueprint of design allows for effective projection of input signals to a higherdimensional feature space in case processing centers experience fading memory property.This in turn allows for the use of these physical reservoirs in an RC approach to machine learning.What is unique is the static part of the system, inseparably connected with its design and connectivity map between the nodes, although the connection weight updates possess a dynamic characteristic.Zhang et al. delve into the development of a memristorbased PRC, a framework inspired by the brain"s computational principles. 98)The key strategy involves constructing neural network-like random networks to enable efficient and energy-saving information transfer.The study explores the feasibility of in materia PRC systems through the demonstration of physical systems such as silicon-based photonic chips 99,100) and atomic switch networks (ASNs). 101,102)A physical reservoir incorporating ASNs is considered a highly promising framework for the implementation of hardwarebased RC.In such a setup, the emergent dynamics are generated by the entire network system rather than by individually tuning its elements.To achieve this, a typical approach involves coating a self-assembled metallic nanowire network with memristive materials.Consequently, each junction within the network functions as an atomic switch, similar to a biological synapse.Challenges arise in creating highly random, interpenetrating networks with nanoscale switching properties, necessitating unconventional processing methods for self-assembly and self-organization at nanoscales.The application of nanowire networks coated with organic and inorganic materials (such as Ag 2 S, polyoxometalate (POM), porphyrin-POM, and polyvinylpyrrolidone) in RC, considering factors like activation modes and 050803-10 © 2024 The Author(s).Published on behalf of [111][112] Another of the examples, this time by Tanaka lab, is also a system that constitutes a real RC platform. 113)It is a recurrent network formed by SWNCT-porphyrin POM (Por-POM) complexes.This architecture demonstrates properties such as nonlinearity and higher harmonic generation.The system can also process information in an "edge of chaos" manner.The RC system is benchmarked against a supervised object classification task using tactile sensory input datasets from the Toyota HSR.The sensor measured the change in gripper angle (θ) as a function of applied forces, providing tactile information to classify mixed hardness and softness, including a bus, block, dog, and hedgehog [cf Fig. 14(a)].Objects were sequentially presented to the device, and the resulting outputs of all electrode pads were collected for training and testing, utilizing a supervised regression model in Python with a hot target vector [cf Fig. 14(b)].
The dominance in the literature of nanowire networks derives directly from their constitution-it is relatively easy to just add linear classificatory in place of device output.Zhu et al. 106) were able to implement an RC system, consisting of a metallic nanowire network, where at the cross-point junctions electrochemical metallization created nanofilaments, asserting memristive switching abilities of the system.The physical form of the device was   multielectrode arrays (shown in Fig. 15), patterned with Cr/Ti (5 nm) and Pt (150 nm).Then the actual functional material was made from synthesized Ag 2 Se nanowires.This study showcases the effectiveness of online learning with nanowire networks, achieving a 93.4% accuracy in image classification tasks, highlighting the benefits of recursive least squares for faster convergence and reduced numerical errors.The computing system optimally utilizes the rich dynamical features of recurrent networks, changing the focus of the training strategy to a linear output layer only.
Ag 2 Se nanowires also constitute other types of physical systems, such as those presented by Kotooka et al., yet they are not always recognized by name as RC systems. 114)This happens even though the reported nonlinear and highdimensional properties are shown and the device executes classification tasks.
Another physical realization-a self-assembled metal (Ag) nanowire network with an MNIST type of test-was presented by Milano et al.The network, as with previous systems, features random connections among multiple nonlinear memristive elements. 102)The reservoir layer is constructed with a low-cost bottom-up approach using an interconnected memristive network.It has both nonlinear dynamics and fading memory properties and enables spatiotemporal processing of multiterminal inputs through functional synaptic connectivity.The readout layer is implemented by mapping synaptic weights associated with each output neuron using an array of TaO x resistive random-access memory (ReRAM) cross-point junction devices, where resistive switching relies on the formation/rupture of a filament.This structure is presented in Fig. 16.In conclusion, this study introduces a fully memristive RC architecture: selforganizing (bottom-up) nanowire (NW) networks combined with top-down ReRAM devices for general purpose intelligent systems.The exemplary computing capabilities of the physical reservoir of the NW network were demonstrated on the basis of MNIST digits classification.
Despite superior computing merits, the application of NWbased RC to real-life applications requires continuous efforts due to existing challenges such as material compatibility, the dynamic complexity of traditional CMOS technology, and achieving optimal performance. 18)

Reservoirs from nanoparticles
Other types of nanostructure, nanoparticles (NPs), sometimes still with the support of elongated connectors (NWs), can be implemented in physical RC systems.Some of the recent examples are of Ag-Ag 2 S core-shell NPs. 115)In this study, the examined system utilizes the dynamics between constituting NPs, generating in turn high dimensionality within the echo state property.These elevate the accuracy of target waveforms up to 99%.
The same group also implemented α-Fe 2 O 3 /titanium bismuth oxide compounds using sol-gel synthetic procedures. 116)The computational efficiency of this RC device was evaluated by introducing varying levels of noise injection.Characterization and identification of functional structures indicated the presence of α-Fe 2 O 3 , TiO 2 , as well as Bi 4 Ti 3 O 12 NPs.The accuracy of the predicted results was enhanced by additional white noise of a small voltage intensity, added to the input.The system had desired nonlinearity and allowed for waveform generation tasks/tests with 87% accuracy.Noise injection is similar to the phenomenon observed in the biological brain-fluctuations of the external world.
Universal concepts of device design for both of the above solutions are presented in Fig. 17

Enhancing dynamics: reservoirs with drive
All recently reported RC systems utilize the simplified approach-the only stimulus reaching the reservoir is the input data.Initial considerations, as well as numerous theoretical works, emphasize the importance of the drive-an additional signal that influences the reservoir along with the input data.This approach has been developed mainly for sensing applications: drive stimulates the reservoir using different physical channels, so direct interference is avoided; e.g.]117,118) To date, this approach has not been applied in any physical implementations of RC bar one: the FF-based computing systems reported in 2023 by Crepaldi et al. 90) In this particular case, however, the drive has been applied not along with the data to be processed, but before, in order to prepare the device for a particular task.This can be considered as a first step towards programmable reservoir computers, in which not only the readout layer is trained, but also the dynamics of the reservoir can be put into a desired state prior to computation.
A recent paper by Shibata et al. uses the term "physical masking," which in principle is the application of a drive to a physical reservoir. 119)The device under study is a redox/iondriven transistor with a LiCoO 2 channel, Li 3 PO 4 gate dielectric, and two independent drain electrodes [Fig.18(a)].
An input signal is supplied to the gate electrode, whereas the second drain can be used as a drive electrode (referenced by the authors as physical masking), as shown in Fig. 18(b).The absence of the drive input application of voltage pulses results in relaxation-type dynamics [Fig.18(b)], which is, however, rich enough to be engaged in computation, e.g. the prediction of waveforms following the second-order dynamic equations.Application of the saw tooth-like drive results in apparently uniform, featureless output [Fig.18(c)], the dynamics of which, however, are much richer and present higher variability, as reflected by the time course of the reservoir state evolution [Fig.18(d)].This simple example demonstrates the importance of the drive, which enriches the reservoir dynamics and contributes to the increased dimensionality of the phase space of reservoir states.This in turn directly translates to the separability performance and may also affect the generalization features of any RC system.
However, application of drive signals is not a trivial task, as it directly affects the reservoir phase space and may move the systems into an explosive mode [cf Fig. 2(d)] if either the sensitivity of the reservoir to the drive is too high or the PROGRESS REVIEW spectral radii of reservoir matrices are too high.So far, there are no good universal search algorithms for drive signals; however, Athanasiou and Konkoli derived periodic drive signals for memristive reservoirs that showed good performance in classification of electrocardiography (ECG) signals. 117)n light of Eq. ( 3) it can be seen that the input and the drive can interact both indirectly (like in photoelectrochemical sensors) or, assuming relations (10-11)  ( ) The input and the drive may be undistinguishable in the extreme case.The main obstacle in the application of drive-operated RC systems is the difficulty in finding the appropriate drive signal.However, depending on the task, some solutions can be found.For example, a search for a well-defined pattern can be performed by applying a drive, which is related to the patterns in question by some symmetry rules.In the simplest case, the drive that will be a negative of the searched signal will be suitable for the task.Then, any small difference between the target patterns and the input will be detected, and the reservoir will effectively measure the differences between the input and the target pattern.In this particular case, further simplification may be achieved; just observation of the reservoir dynamics, without a trained output layer, may be sufficient for the task.This may be useful in speech recognition, in particular for classification phonemes.This will, however, require either sequential operation of the same reservoir with different drives (time consuming) or operation of numerous reservoirs in parallel (or complex reservoirs with numerous inputs and drives, which automatically can be translated to parallel operation of simpler reservoirs). 69,120)nother alternative would be exploitation of reservoir evolution in time, as in single node echo state machines and hierarchical structures based there [Fig.][128][129][130][131][132][133] It is applicable to chemical sensing as well, even in very simplistic cases, as minute changes in the impedance of the layer at the electrode are translated into differences in signal evolution. 36,40)Most of these systems operate, however, without a drive signal.This hierarchy was also implemented in thin-layer memristor-based reservoirs for signal amplitude discrimination. 124,125)ome time ago, numerical simulations of the analysis of musical harmony with single node echo state machines was reported. 134,135)In these studies, two sine wave signals were applied to the reservoir feedback loop, and their evolution (d)  was followed.In the study of musical harmony, both signals were considered as inputs, but formally one of them could be considered as a drive.Therefore, here we present preliminary results on numerical simulations of a closely related circuit (memristive Wien bridge 136) with operational amplified and capacitive circuits).The simulator, implemented as a SPICE object in Multisim, is based on a memristor model by Vourkas and Sirakoulis (Fig. 20 and Table I). 137)he full circuit of the bridge synapse, with an internal capacitor and capacitive coupling to the operational amplifier, is shown in Fig. 21(a).An ideal amplifier with an internal input resistance of 10 MΩ and output resistance of 10 Ω has been used.The variable resistor was set to 2 kΩ for the highest nonlinearity of the response, evaluated on the basis of Fourier analysis of the output with sine input.No feedback resistor was used for the amplifier, but despite that the whole circuit performed both partial integration (charge storage at the central capacitor) and differentiation (capacitive coupling of the amplifier).The performance of the circuit was tested for a single-frequency sine input within 30-120 Hz.It was found that this type of input induces a series of fading self-oscillations, with subsequent echoes of a heavily distorted sine character.
The application of a two-sine function generator with proper output amplitude, operating at the same and at different frequencies, had no effect on the behavior of the circuit: fading echoes have been observed for almost all frequency combinations [Fig.21(b)].OriginPro's envelope function has been used to compare fading profiles [Fig.21(c)].It was found that the only input combination that can be differentiated from the others on the basis of fading profiles is the situation in which two input signals fulfill condition ( 12 i.e. they form an octave.This observation is related to the previous report, in which full characterization of musical harmony was performed.With this approach, frequency deviations of ca.0.5 to 1.5 Hz were detectable within the given frequency window.Therefore, in light of previous considerations of a proportional integral derivative controller, which also combines integrating and differentiating components and can be regarded as a primitive form of reservoirs, 138) we attempted to perform phase separation of sine signals as well as waveform discrimination, which was also demonstrated in an in materia device, but without delayed feedback. 139,140)Therefore, a pair of signals (100 Hz input and 50 Hz drive) were applied to the reservoir, and the input signal was subjected to phase shifts from 1 to 180 degrees [Figs.21(d), 21(e)].
The signal with a small phase shift (j < 3°) resulted in significant persistence of the signal [Fig.21(e)], while larger phase shifts resulted in rapid decay of the signal intensity.This observation indicated a very high sensitivity of a relatively simple reservoir circuit to even subtle changes in the input signal.It also demonstrates the power of a drive; application of a properly designed signal enables even readout-free operation of the reservoir and one-hot classification of inputs on the basis of output amplitude after a given evolution time.Finally, the same circuit has been subjected to different waveforms to check how octave-detection procedure behaves in the presence of higher harmonics (a square wave may be considered as a sum of sine harmonic components).Surprisingly, the persistence of an octave was significantly higher than other tone combinations; however, the effect was not as clear, as in the case of pure sine functions [Fig.21(f)].

Conclusions
In order to implement RC concepts in a form of physical devices, several types of internal processing nodes need to be considered-optoelectronic, spintronic, and memristive.The latter were implemented in thin layers, assemblies of nanowires, and NPs.Despite various compositions and different topologies, there is no clear relation between the physical form of the material (bulk or nanostructures of different dimensionalities).Due to a growing number of different examples, and no simple classification system, the final distinction of RC system types can be made based either on their structure or application.
Although typical solutions, such as software RNNs, have high complexity, RC physical systems avoid all of this, as their constitution relies in most cases on nanomaterials, leaving only a relatively simple output layer to be "programmable" by users.Among the main material requirements is for it to be characterized by a nonlinear response and complex internal dynamics.
For typical material-based (in materia) types of systems, there is a discrepancy regarding software simulations, as these cannot take into consideration all of the intrinsic phenomena.That is why, despite its growing popularity, the approach to model just one physical entity (either a single memristor or a single information-processing node) that as yet present the whole network only in silico should be treated as an approximation.
In order to describe these systems, one should follow the interface-input-reservoir-output distinction.In this way, the explanation of the principle of work for the system is made easier.The end user communicates with the system through an intrinsic interface, which allows the input signal to be incorporated further.Depending on the choice of solutions, the signal is then modified within the framework of the actual RC system.In order to utilize the full classification functionality, output signal is generated and the final signal classification is executed.Typically, there exists some kind of readout layer implementation; however, it is possible to find approaches, where the readout layer is omitted, relying only on specially designed postprocessing algorithms or procedures-see some examples from our group. 125,141)he difficulties related to postprocessing of the reservoir output (via application of readout perceptron, other neural networks, and complex mathematical treatment) are some designated cases and for specific computational tasks can be greatly simplified by the application of drive signals.Drive application has already been demonstrated in chemical sensing and analysis of acoustic signals.These RC systems operated without a proper readout layer, but were still capable of performing one-hot classification of inputs or yielded highquality analytical signals.
There is another difficulty in physical implementations of RC.The input must be appropriately converted in order to interact with the reservoir-no relevant data should be lost in this translation.Furthermore, as we demand universality of computing systems, then RC should operate in a taskindependent way.For physical systems, this may be very difficult to implement.The translation problem mentioned above is not trivial-some input data may be naturedly and intuitively translated, but it is not always true.The RC systems described so far are designed for one, well-defined computational task, so achieving universality in the context of hardware implementation is still an unsolved problem.
Intuitively, RC systems seem to be similar to oracle machines.An oracle machine (commonly abbreviated to omachine) is a variation of a Turing machine, which can solve any computable problem is a single step.In general, an oracle

PROGRESS REVIEW
can provide a solution to any decision-making or functional Such a problem does not have to be algorithmically decidable; the oracle can represent answers to any mathematically defined set of problems. 142,143)Complex reservoirs, in principle, can perform any classification task, but not in a single step (well, at least delayed feedback devices require finite time to establish a new internal dynamic setup, or require given time for evolution into a final step).If we associate this feature with the problem of the drive (an additional input, which can also be understood as an equivalent of software for the reservoir), we are approaching an interesting problem.A reservoir (with internal complexity suitable for a specific set of problems) may operate as a time-delayed oracle machine.The only problem is the communication language: appropriate drive (command) is not known (or rather the language in which it should be given is unknown).Furthermore, the output, understood as a modification of internal dynamics of the reservoir, may be considered as an answer given in an unknown language.To make the story even more confusing, one cannot assume a priori that these two languages are identical.Then, communication with complex reservoirs may be as complex as attempts to understand the famous Cthulhian incantation "Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn", 14) the Voynich's manuscript, 144,145) Codex Seraphinianus, 146) or fungal languages. 147)It has been demonstrated, however, that, for special classes of problems, the formal description of RC simplifies 138) and therefore the search for appropriate drives is not in vain. 117)herefore, in materia (or physical) RC constitutes a universal information-processing platform that, however, needs further development, not only from the material point of view (higher nonlinearity, robustness, and memory capacity are still to be developed) but also in the design of reservoirs and their operation protocols.

Fig. 2 .
Fig.2.Performance of a reservoir computer in terms of phase space transformation: too weak activity leads to the death of reservoir dynamics (a), efficient complete (b) and partial (c) separation of input data into selected categories, and an explosion of the phase space (d).

Fig. 3 .
Fig. 3. "An old pond, frog jumps in, splash!".Turbulence of water induced by a jumping frog (top).The full figure showing the frog in water (bottom).The upper panel illustrates the operational principle of RC: the input can be detected/classified/recognized according to the disturbances of the internal dynamic of the system under consideration.

Fig. 4 .
Fig. 4. Structure of a spin-wave-based RC device with a magnetic garnet film, stripe domain structure, and input/output electrodes.The input voltage excites spin waves in the film, propagating to the output electrodes, detected through the output voltages (a).A spin-wave-based RC system with input preprocessing, a reservoir, and a readout.Training follows in Sects.(1)-(8) and testing follows in Sects.(3)-(6) and Sects.(9)-(10).The reservoir includes an input exciter, a magnetic stripe domain structure, and expected spin-wave propagation.The blue curves represent time-series output waveforms at 72 detectors (b).Reproduced from Ref. 89 with permission.

Fig. 5 .
Fig. 5.The stimulus setup for the PRC tests includes parallelized outputs in the readout neural network layer, detailing the impact of each pixel value.It also covers information about the neural network layer and the conceptual liquid reservoir (a).The confusion map shows the real-time classification testing of digits 0 to 3 using the trained neural network (b).Reproduced from Ref. 90 with permission.

Fig. 6 .
Fig. 6.Illustrated signal-processing flow for physical reservoir computation.Using a triangular pulse as the input signal, output current values (x 1 , x 2 , …, x N ) at each time step were generated by redox reactions at the copper-2,5,8,11-tetraoxadodecane (Cu-G3)/electrode interface.Then, these values were input to N virtual nodes.The learning process, determining the weights (w 1 , w 2 , …, w N ), was performed through linear regression using Y train as the training data.Reproduced from Ref. 76 with permission on the CC-BY 4.0 licence.
the given context, where C represents the capacitance of the capacitor, v C (t) signifies the capacitor voltage, i(t) denotes the current at the voltage source, i m (t) represents the electric current at the mth memristor (where m = 1, 2), L is the inductance of the inductor, R is the resistance of the resistor, v m (t) [equal to v C (t)] stands for the voltage at the mth memristor (for m = 1, 2), and v in (t) is the input voltage at the voltage source.The correlation between i m (t) and v m (t) relies on the intrinsic characteristics of the mth memristor.During the readout process, the voltage v m (t) and current i m (t) are measured within the time interval á ñ T 0, for m = 1, 2. The evolution of memristance, denoted as

Fig. 7 .
Fig. 7. Evaluation processes for STM and NARMA2 tasks involve applying input voltage to IL-PRDs and conducting linear regression analysis with output current values.The binary data (0 and 1) is transformed into a triangular voltage pulse train and applied to the IL-PRD.The input dataset for linear regression is generated using a virtual node method with a node number k = 50.Reproduced from Ref. 75 with permission on the CC-BY 4.0 licence.

Fig. 8 .
Fig. 8. Schematic representation of the circuit of an RC system with two parallel memristors, a resistor, an inductor, and a capacitor.The reading part is a simple linear classifier.Reprinted from Ref. 91 according to the CC-BY-NC 4.0 licence.

Fig. 9 .Fig. 10 .
Fig. 9.The RC system utilizes a memristor network with preprocessing, input, reservoir, and readout components.It processes time-series data, measures current signals in the reservoir, and optimizes the output weight matrix W out through linear regression in training.Reproduced from Ref. 92 with permission.

Fig. 13 .
Fig. 13.Enhancement and noise generation in the POM-SWCNT network are illustrated in the experimental setup.(a) The network schematic depicts terminal electrodes (depicted as yellow cuboids), SWCNTs (represented by black tubes), and POM particles (depicted as purple spheres).(b) A photograph of the substrate featuring six terminal electrodes is included, with the entire substrate coated with the POM-SWCNT complex.(c) The sampled current density over time, indicating the current magnitude distributions, is presented in a plot where the bias voltage increases incrementally from 0 to 125 V across the electrodes in sample A. Reproduced from Ref. 112 with permission.

Fig. 14 .
Fig. 14.(a) Time-series inputs from various objects (hedgehog, dog, bus, and block) are individually introduced into the SWCNT (black line)/Por-POM (green circles) reservoir featuring recurrent connections (red arrows, left).The voltage readouts of the i output pads for each object are gathered, as illustrated on the right (result for one electrode pad).(b) One-hot vector encoding is used for binary classification.Each square box with the lines inside represents the target signal.The one-hot vector, as the object is truly predicted, is given a vector value 1 while the others are given a vector value 0. Reproduced from Ref. 113 with permission. .

Fig. 15 .
Fig. 15.An RC device design and the principle of operation.The nanowires used in the device are Ag 2 Se.A scannig electron microscope (SEM) image of the 16-electrode device features different electrodes: channel 0 (red), drain, channel 3 (green), readout electrodes, channel 1, 2, 12, 13, 15 (blue), and unused electrodes.On the right, the readout voltages (N × M × 784 dynamical characteristics) are fed into an external linear classifier, where the weight matrix (Wn) for M × 784 features per digit sample is updated after each sample, with the corresponding class as the target output (digit 5 displayed as an example of the classification result).Reproduced from Ref. 106 with permission.

Fig. 16 .Fig. 17 .
Fig. 16.Schematics of the device.The input is encoded as pulse streams, and, secondly, is passed to the NW network physical reservoir, and in the end is classified by the hardware (resistive switching based) readout neural network.(b) SEM image and schematic drawing of a reservoir layer.In place of NW junctions, the formation and rupture processes of metallic Ag filaments occur.(c) SEM image and schematic drawing of a TaO x resistive switching cell at the metal electrode cross-point in place of a junction formation, and rupture processes of the vacancy-based conductive filament occur.Reproduced from Ref. 102 with permission.

Fig. 18 .
Fig. 18.Schematic image of the LiCoO 2 -based redox ion gate reservoir (a).General scheme of the unmasking reservoir, with digital masking applied to the input data and the unmasking reservoir with physical masking (or a drive) (b).Gate voltages and drain currents of devices without (left) and with physical masking (right) (c).Time series corresponding to reservoir states without (left) and with (right) physical masking/driving (d).Reproduced from Ref. 119 with permission.

Fig. 19 .
Fig. 19.Concept of a hierarchical reservoir.States of the same physical system at different moments in time (and upon stimulation with the same or different inputs) are considered as different computational devices.The cost of device complexity is paid by the time required for computation.

Fig. 21 .
Fig. 21.Single-node echo state machine implemented on the Multisim platform.An appropriate gain was set to eliminate premature fading of the signal in higher epochs (a).Evolution of the input signal in a reservoir feedback loop based on bridge synapse (b) and normalized output envelopes recorded for various sets of input frequencies (c) for different combinations of input frequencies.Evolution of the input signal in a reservoir feedback loop for a 100 Hz sine input and 50 Hz sine drive (d) and normalized output envelopes recorded for various phase shifts of the input (e).Normalized output envelopes recorded for a combination of sine and square waveforms of different frequencies (f).