In materia implementation strategies of physical reservoir computing with memristive nanonetworks

Physical reservoir computing (RC) represents a computational framework that exploits information-processing capabilities of programmable matter, allowing the realization of energy-efficient neuromorphic hardware with fast learning and low training cost. Despite self-organized memristive networks have been demonstrated as physical reservoir able to extract relevant features from spatiotemporal input signals, multiterminal nanonetworks open the possibility for novel strategies of computing implementation. In this work, we report on implementation strategies of in materia RC with self-assembled memristive networks. Besides showing the spatiotemporal information processing capabilities of self-organized nanowire networks, we show through simulations that the emergent collective dynamics allows unconventional implementations of RC where the same electrodes can be used as both reservoir inputs and outputs. By comparing different implementation strategies on a digit recognition task, simulations show that the unconventional implementation allows a reduction of the hardware complexity without limiting computing capabilities, thus providing new insights for taking full advantage of in materia computing toward a rational design of neuromorphic systems.


Introduction
The development of new hardware platforms for computing relies on exploring and exploiting the relationship in between * Authors to whom any correspondence should be addressed.
Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. new device physics and information-processing capabilities [1]. Indeed, physical laws regulating a wide range of physical systems not conventionally used for computation have been proposed for energy-efficient hardware implementation of machine-learning calculations [2,3]. In this context, reservoir computing (RC) have been recently proposed as a versatile computing paradigm suitable for hardware implementation where complex dynamics of physical systems are exploited for information processing [4,5]. Derived from recurrent neural networks models such as echo state networks [6] and liquid state machines [7], RC represents a computational framework for efficient temporal/sequential data processing that endows fast learning and low training cost. These characteristics makes RC suitable for a wide range of real-world applications including signal processing, robotics and sensory devices in the framework of edge computing.
The basic concept of RC is related to the exploitation of intrinsic dynamics of a reservoir to project the input in a feature space, where learning is outsourced to a readout stage. This means that only the readout needs to be trained, while no training is required for the reservoir. For this purpose, a wide range of physical dynamical systems have been proposed for in materia implementation of the reservoir, differently from conventional reservoirs where dynamics are simulated on a computer [4,5]. In other words, physical RC explores the computational power of physical systems to process information. In this context, implementation of physical RC have been demonstrated in a wide range of mechanical devices [8], photonic chips [9,10], spintronic oscillators [11], ionic liquidbased systems [12], field programmable gate arrays [13] and devices based on 2D materials [14].
Among electronic-based systems, neuromorphic devices and non-von Neumann architectures based on memristive devices have shown their potentiality for in materia [15] reservoir computing. By coupling ionics with electronics, the internal state of memristive systems depends on the history of applied voltage and current, allowing processing of the input signal in a history-dependent and highly non-linear fashion [16]. For this reason, internal ionic dynamics has been exploited for the realization of memristor-based reservoirs able to process information in the temporal domain. Besides memristive reservoirs realized by organizing group of memristive cells in crossbar arrays with a top-down approach [17][18][19][20][21], RC have been demonstrated in self-organized [14] memristive nanonetworks by exploiting emergent dynamics of the system as a whole [3,[22][23][24][25][26][27][28][29][30][31]. In this case, where the emergent behavior arises from the memristive interaction of a multitude of interacting nano-objects [32][33][34][35][36][37][38], no fine tuning of network elements is required. While pioneering works reported physical implementation of RC by exploiting the complexity provided by physical substrates based on atomic switch networks [39] or carbon nanotubes [40,41], the RC paradigm has been then shown to be implementable in a wide range of self-organized networks including networks of coreshell nanoparticles [42], molecular neuromorphic networks [29], and organic electrochemical networks [43,44]. In addition, RC systems based on self-assembled nanowire (NW) networks have been demonstrated to be effective for a wide range of computing tasks including waveform generation [28,39], logic operations [26], pattern recognition [3], speech recognition [25,30], time series prediction [3], tactile sensing object classification [28]. For these purposes, different strategies for implementation of RC have been reported.
In this work, we report and compare different strategies for in materia RC with self-organizing memristive networks. Simulations show that the emergent dynamics of the system arising from the mutual interaction among a large amount of memristive elements allows unconventional implementation strategies of RC, including the possibility of using the same electrodes as both reservoir inputs and outputs. As shown by investigating the internal dynamics of the system through simulations during a pattern recognition task, it is shown that a rational implementation of RC allows a reduction of the hardware complexity without affecting computing capabilities.

Memristive nanonetworks as physical reservoirs
Self-organized memristive NW networks can be realized by randomly dispersing NWs in solution on an insulating substrate by means of drop casting. A representative image acquired through scanning electron microscopy of a memristive Ag NW network is reported in figure 1(a). These memristive NW networks have been shown to fulfill the requirements of a physical reservoir including high dimensionality, nonlinear dynamics and fading memory [3,24] (refer to [4] for a detailed description of requirements of a physical reservoir). These properties arise from the emergent memristive behavior of the network related to the mutual interaction of a large number of memristive NW junctions. Indeed, due to the network connectivity, the interaction in between memristive junctions is responsible for the emergence of spatially correlated structures of network activity that depend on the spatial location and temporal sequence of electrical stimulations [32]. In particular, the memristive behavior at the NW intersections is regulated by resistive switching effects associated to the electromigration of Ag + ions to form Ag conductive filament across the polyvinylpyrrolidone (PVP) insulating layer (thickness of ≈1-2 nm) that surrounds the NW inner cores, as schematically depicted in figure 1(b) [32]. The switching mechanism related to Ag dynamics is volatile [45][46][47], meaning that the conductive filament at the intersection in between NWs spontaneously dissolves after stimulation. The competition in between filament formation and spontaneous dissolution at NW junctions leads to an emergent potentiation of the network under voltage pulse stimulation with progressive increase of device conductance and subsequent network relaxation to the ground state. Indeed, while temporally correlated voltage pulses applied in between two areas of the network as reported lead to a progressive increase of the effective network conductance (and thus flowing current) (figure 1(c)), a progressive relaxation to the ground state can be observed by monitoring the effective conductance with a low read voltage after relaxation ( figure 1(d)). In other words, the memristive network state depends not only on the programming pulses by also on the history of pulses applied in the past. In this context, the interplay in between potentiation and spontaneous relaxation of different network areas through stimulation of spatiotemporal voltage pulse patterns in multiterminal configurations have been exploited for non-linear transformation of the input signals in the context of unconventional computing [3]. Notably, experimental results have shown that network dynamics can be regulated from the µs up to the hundreds of seconds timescale depending on the stimulation conditions [3,32].

Modeling reservoir spatiotemporal dynamics
Modeling of the emergent spatiotemporal dynamics of the reservoir was performed by mapping the nanonetwork in a grid-graph model [48], as schematized in figure 2(a). This model, that approximates the nanonetwork as a continuous and uniform medium (as expected for high density networks [49][50][51]), is based on the parcellation of the network domain that is then abstracted as a regular grid graph where interaction in between network areas (nodes) is provided by memristive connections (edges). A detail of a memristive edge interaction is schematized in figure 2(b), where the edge conductance depends on the history of electrical stimulation. Dynamics of each edge can be modeled by coupling an equation for electron transport with an equation regulating the memory state of memristive edge. Linear conduction is assumed for electron transport while a voltage-controlled potentiation-depression rate balance equation was exploited to model the memory state of the memristive edge [52].
The current flowing in edge ij is described by the relation: where ∆V ij is the voltage difference between nodes i and j acting as driving force, g ij is the normalized conductance, and G min,ij and G max,ij represents the minimum and maximum conductance values, respectively, of the memristive edge. The memory state dynamics are described through a balance rate equation: where κ P,ij (V ij ) and are the potentiation and depression rate coefficients that endow an exponential relationship with the applied voltage to model the diffusion processes of ionic species: where η P, η D are the transition rates and κ P0 , κ D0 are constants. Under these circumstances, equation (2) can be recursively solved as (by considering ∆t > 0): where g ij,t and g ij,t−1 represent the normalized conductances at times t and t − 1 of memristive edge ij, respectively. When a specific area of the network (node) is stimulated by means of an applied voltage, the conductance of edges of the network evolves over time giving rise to an emergent collective behavior of the network that depends both on the spatial location and temporal sequence of the input signal. Despite not including the effect of network inhomogeneities, the gridgraph modeling has been shown to well describe emergent dynamics of experimentally realized multiterminal memristive NW networks, including short-term plasticity, pair-pulse facilitation and heterosynaptic plasticity [3,48]. For all these reasons, the grid graph model can be exploited to explore different implementations of RC in memristive nanonetworks, where device geometry and positions of electrodes acting as neuron terminals are mapped onto the grid graph topology.

Implementations of RC in nanonetworks
Conventional two-terminal memristive devices have been exploited for the realization of RC systems by exploiting their internal short-term dynamics [17][18][19][20]. In this case, a memristive device can be considered as a neuron that possesses a recurrent connection with a synaptic weight less than 1, where the recurrent node continuously decay its state if a pulse input is not provided [18]. The reservoir state is represented by the device conductance. Since each memristive device is able to process a single input pulse stream, an array of discrete memristive cells is required for solving computing tasks that require the transduction of the input into multiple pulse streams. In this case, the reservoir state is represented by the collective resistance states of the whole set of memristors. A conceptual schematization of RC implementation in conventional memristive devices is reported in figure 3(a) (configuration a). Differently from conventional RC systems based on two-terminal memristive devices where addressing of each memristive cell is required, nanonetwork-based RC systems require accessing the emergent behavior of the whole system while it evolves under electrical stimulation. For this purpose, different implementation strategies can be explored as detailed in the following.
Computing capabilities assessed by means of simulations have been carried out by considering nanonetwork-based reservoirs as graphs where nodes are represented by single NW units, while edges represent their memristive interactions [53,54]. In this framework, specific NW nodes have been allocated as inputs and outputs of the reservoir. For example, Zhu et al [34] analyzed through simulations the information processing capabilities of NW networks by applying an input in form of a voltage signal in between a couple of selected source and drain NW nodes (reservoir input nodes), while reservoir outputs were recorded at other selected measurement NW nodes. Similarly, simulations by Fu et al [24] and Hochstetter et al [37] analyzed the implementation of nonlinear transformation tasks. While these simulations have strongly contributed to the understanding of computing capabilities of NW networks, it is experimentally unfeasible to access and record signals from each single NW node in very large networks. In this context, simulation by Daniels et al [31] reports that computing capabilities are preserved when considering realistic electrodes, showing performances close to upper bounds achievable by considering information from each wire. With a similar approach, pioneering works reported an experimental implementation of RC paradigms in multiterminal devices based on self-assembled systems where multiple electrodes allow to access signal from different areas of the networks [25,39]. Similarly to simulated multiterminal devices, also in these works the input in form of a voltage difference was applied in between selected source/drain nodes, while exploiting remaining electrodes for measuring output signals. Also, multi-electrode devices based on networks contacted by an array of input electrodes and an array of output electrodes were exploited for neuromorphic applications in the work by Diaz-Alvarez et al [55], where specific combinations of electrodes can be selected through electromagnetic relays that control the operative input/output channels. The common demonimator of all these implementations is the exploitation of separate and distinct sets of electrodes as network inputs and outputs. It is worth noticing that a similar implementation strategy was exploited also in case of a multichannel skyrmion-based reservoir [56].
Despite previous works reported several variations on the theme, broadly speaking these implementations can be conceptually schematized as in figure 3(b), where generic input(s) transduced in a sequence of electric signals are applied to different channels represented by input/output electrode couples that can be selected through appropriate relays (configuration b). Note that in the implementations reported in [25,39] input electrodes are represented by a single couple of source-drain electrodes, since the input is encoded in a single sequence of time-dependent electrical signals applied to one source electrode while grounding the selected drain electrode. In the configuration reported in figure 3(b), reservoir outputs to be passed to the readout can be represented either by (a) voltages measured at output electrodes (in this case, high impedance output channels to measure voltage at terminals B are ideally required, i.e. R → ∞) or by (b) currents flowing in between output electrodes and ground.
As an alternative, RC can be implemented by considering an unconventional configuration where electrodes can act simultaneously as inputs and outputs [3], as conceptualized in figure 3(c) (configuration c). Here, input signals are applied to terminals A of resistances R that are in series to each electrode pad, while outputs are represented by the voltage measured at terminals B of the same resistance. In this fashion, the same pad electrodes can be indifferently and even simultaneously exploited as input and outputs of the reservoir without any need of electromechanical relays.

Computing task implementation-digit recognition
Memristive RC systems require the transduction of the computing task inputs into one stream or, more in general, multiple streams of voltage pulses composed by n timeframes. As an example, in a pattern recognition task each input pattern can be transduced to a set of pulse streams each corresponding to a pattern row (spatial domain), while each pattern column is encoded in the temporal sequence of electrical pulses (temporal domain). The response of the two different implementations of RC in nanonetworks was tested by considering a simple task that consists in recognizing digits from input images reported in figure 4(a). Figure 4(b) reports an example of transduction of the 5 × 4 input pattern corresponding to digit '9' in a sequence of voltage pulses, where the input is encoded in both temporal and spatial domains. In particular, each pulse stream corresponding to a pattern row can be divided into timeframes, each corresponding to a pixel of a pattern column. In this case, the input pattern was transduced in five pulse streams, each composed of four timeframes. Each timeframe can be further divided in a 'stimulation' section and a 'read' section, as schematized in figure 4(c). During the stimulation section of each timeframe, the corresponding channel/pad of the reservoir is stimulated with a V p pulse if corresponding to a white pixel. The following read section allows to evaluate the evolution of the internal reservoir state over timeframes, where the final read of the reservoir state represents the nonlinear transformation of the input to be passed to the readout for classification (final reservoir state).
Stimulation and read configurations as well as the evolution of internal dynamics and output voltages of the NW networks reservoir in configuration b ( figure 3(b)) and c (figure 3(c)) during stimulation with digit '9' are reported as examples in figures 5 and 6, respectively (supplementary note 1). Here, the two implementation strategies are compared, using the gridgraph approach for modeling the reservoir internal dynamics. In all configurations, resistances of 82 Ω were exploited to realize the corresponding circuits, while model parameters were extracted from experimental data (details in supplementary information S1).
In configuration b, during the stimulation section of each timeframe the electrodes corresponding to channels that require a stimulation with a voltage pulse are connected, while all other electrodes are left floating, as schematized in figure 5(a). Instead, during the read section, a reading voltage is applied to a selected input electrode and output is recorded at terminals B of four selected output channels while keeping all the other terminals floating, as schematized in figure 5(b) (this reading scheme was adopted for a more direct comparison with the unconventional implementation of configuration c). Figure 5(c) reports the voltage distribution across the NW networks over timeframes during stimulation sections. As can be observed, the voltage drop occurs mainly in between areas of the network where input and output electrodes are located. Note that during timeframes where no pulses are expected the corresponding electrodes are not influencing the overall distribution of voltage since they are left floating. Since the conductive pathways tends to form perpendicularly to equipotential lines under the action of the electric field, the emergent dynamics is characterized by the formation and spontaneous relaxation of conductive pathways connecting corresponding input and output electrodes. These dynamics leads to reservoir output voltages (recorded at terminals B during the reading In configuration c, during the stimulation section a voltage pulse is applied to corresponding electrodes that require stimulation, while other electrodes were kept at 0 V, as schematized in figure 6(a). Instead, during the reading section, a reading voltage is applied to a selected pad, while output is recorded at terminals B of other four pads, as schematized in figure 6(b).
Note that in this latter configuration (configuration c), all terminals A (except from the electrode where a read voltage is applied) are kept at 0 V. Figure 6(c) reports the voltage distribution across the NW networks over timeframes during stimulation sections. As can be observed, in this case the voltage drop over the network relies on mutual differences of applied voltages among electrodes, driving the formation of conductive pathways in between network areas where a voltage difference is present, as reported in figure 6(d). As an example, during timeframe t1 of digit '9' pads 1, 2 and 3 are simulated with a voltage pulse while other electrodes are grounded. This results in a nearly equipotential area on the right side of the network. In this case, the voltage drop is mainly localized across couples of electrodes 1-5 and 3-5 and, less intensely, across couples of electrodes 3-4 and 1-4. These dynamics leads to reservoir output voltages (recorded at terminals B during the reading section of each timeframe) reported in figure 6(e), and to the corresponding histogram of the final output voltages (i.e. the final reservoir state) reported in figure 6(f).
As can be observed, the two different configurations result in peculiar evolution over timeframes of the output voltages. This peculiar evolution reflects to different histograms representing the final reservoir states. In other words, this means that the two configurations perform a different nonlinear transformation of the input pattern. This is due to the different evolution of the internal conductivity map in the two different stimulations, as revealed by modeling. Indeed, the same stimulation pattern of digit '9' was observed to result in strongly different patterns over the memristive network. These activation patterns are related to the interplay in between formation and spontaneous relaxation of conductive branches and pathways connecting electrodes. In particular, these branches forms depending on the local voltage gradient induced by stimulation, where the electric field spatial distribution for each timeframe depends on spiking electrodes.
The evolution over timeframes of reservoir output voltages while feeding the reservoir with the ten digits, the corresponding final reservoir conductivity maps and the final histograms for configuration b are reported in figure 7. Similarly, results for configuration c are reported in figure 8. As can be observed, in both configurations the final reservoir states (i.e. conduction maps and output voltage histograms) are significantly different, showing the ability of both reservoir implementations to separate the 10 digit inputs. Final reservoir states represented by the standardized reservoir output voltages at the last stimulation timeframe are then used as input to the readout for training and classification. The readout is composed by a 4 × 10 network, where output neurons labeled from 0 to 9 represent the predicted digit of the input pattern. For this purpose, a single layer feed-forward neural network was trained by minimization of the loss function through the Adam algorithm [57], similarly to a previous work [3]. After training (details in supplementary S2), both RC implementations can recognize all of the 10 digit inputs. In particular, similar learning curves were observed, suggesting that the two implementations endow comparable separability properties, at least for this task (details in supplementary information S2).

Discussion
The implementation of RC in self-organizing memristive nanonetworks realized with a bottom-up approach requires a change of paradigm respect to RC systems based on discrete memristive devices organized in crossbar, as shown by simulations. The main difference is that in self-organizing memristive nanonetworks the reservoir dynamics is regulated by the emergent behavior of the system as a whole and rely not only on the temporal processing capability but also on the spatial location of stimulation. In other words, memristive nanonetworks allows processing not only on the temporal domain but also in the spatial domain due to the intrinsic functional connectivity of the system. This allows the extraction of features by taking into account not only the temporal correlation but also the spatial relationship and correlation of input signals. As shown by results, the evolution of reservoir dynamics depends on the temporal sequence, spatial location and even on the interplay in between temporal sequence and spatial location of input signals. This opens the possibility of different implementations of RC, where not only the position of electrodes but also the way to assess the internal dynamics represent a crucial aspect.
The configuration where electrodes are partitioned in different sets of input and output electrodes (configuration b) can be considered a legacy of the crossbar array architecture, where stimulation is applied in between word and bit lines. In this framework, it is possible to observe that pattern stimulations result in the formation of conductive pathways connecting the input and the corresponding output electrodes. In the extreme case where input/output couples of electrodes are spatially located distant from each other (i.e. the crosstalk in between channel is negligible), the emergent behavior tends to the behavior of discrete memristive cells. Instead, the configuration where the same electrodes act both as inputs and outputs (configuration c) is based on a radically different concept, where stimulation is not applied in between couples of source/drain electrodes, but the emergent behavior relies on the peculiar electric field landscape generated by different voltages spatially applied to different electrodes. In other words, emergent behavior dynamics of the reservoir in configuration c rely on voltage differences spatially generated across the network.
Results showed that both implementations endow similar capabilities of extracting relevant features from the input. Indeed, both configurations provide separation capability to the reservoirs, a key aspect that allows the reservoir-based system to reduce the number of parameters to be trained compared to conventional neural networks. A conventional neural network for recognizing patterns will have a number of inputs that corresponds to the number of pixels. This means that, in our case, if the 5 × 4 = 20 input pixels are connected to the ten outputs to form a 20 × 10 neural network, training of 200 weights is required. Note that the number of weights can further increase if more hidden layers are considered. Instead, the digestion of the input signals in both the temporal and spatial domain through the nanonetwork allows a reduction of parameters to be trained. Indeed, both RC implementations digest the 5 × 4 images to output a 4 × 1 vector, so that following classification can be performed by a 4 × 10 neural network where only 40 parameters needs to be trained.
Despite comparable computing capabilities, it should be noted that by exploiting the same electrodes as both inputs and outputs as in configuration c it is possible to reduce the hardware complexity, thus limiting the wiring costs, and to increase the scalability, thanks to the reduction of the number of electrodes. Concerning the number of electrodes, for a generic input with N spatial inputs, configuration b requires 2 N electrodes while only N electrodes are required in configuration c. An advantage of using configuration c is also the simplified peripheral circuitry compared to configuration b, since no switches to selectively open/close channels are required. In perspective, it should be pointed out that even hybrid implementations of the two RC configurations can be explored for tailoring the hardware implementation on specific computing tasks.
In this context, it is worth mentioning that the choice of the best implementation strategy needs to take into account also reliability issues and inaccuracies related to both variability effects in the spatiotemporal evolution of reservoir dynamics and local inhomogeneities in the network topology. Despite these aspects have been neglected in our model, these issues are expected to differently affect the accuracy and robustness of different implementation strategies depending also on the specific computing task.
Finally, it is important to point out that the hardware complexity can be further simplified by increasing the temporal processing capability of the nanonetworks through tailoring the decay time of the system. Depending on the computing task, the decay time can be optimized by taking into account the trade off between (a) the capability of the network to remember the far history, since it determines the number of timeframes that can be processed without loosing information from initial timeframes, and (b) the pattern to pattern period target, since relaxation to the ground state is necessary for the network to digest a new input pattern. From an experimental point of view, tailoring of the decay time can be explored in perspective by fine tuning of operational conditions and/or an appropriate choice and engineering of involved materials.
Similar approaches of implementation of RC here discussed in nanowire networks can be extended to other complex nanostructured systems such as nanoparticle networks [58,59], discontinuous metallic films [60], nanotubes [61], polymeric substrates [43] and electrochemical systems [62]. While experimental hardware realization and characterization of multiterminal nanonetwork reservoirs with a large number of electrodes and a fine control of temporal dynamics still represent a challenge, it is important to point out that the here reported grid-graph model allows to explore different RC implementations and architectures in silico with low computational sources for a rational design of neuromorphic hardware.

Conclusions
In summary, we reported on the implementation of in materia RC in self-assembled nanonetworks that endow emergent memristive dynamics. Simulations of the emergent behavior of a memristive nanowire network show that the functional connectivity of the system opens the possibility for unconventional implementation strategies of RC. In particular, results show that a rational implementation strategy that exploit electrodes as both reservoir inputs and outputs allows the reduction of the hardware complexity without affecting the computational capability of the system, as proved in a pattern recognition task. These results provide new insights on physical implementation of RC towards a rational design of neuromorphic systems.

Data availability statement
The data that support the findings of this study are openly available at the following URL/DOI: https://doi.org/10.5281/ zenodo.7499746.