In materia implementation strategies of physical reservoir computing with memristive nanonetworks

Self-organized memristive NW networks can be realized by randomly dispersing NWs in solution on an insulating substrate by means of drop casting. A representative image acquired through scanning electron microscopy of a memristive Ag NW network is reported in figure 1(a). These memristive NW networks have been shown to fulfill the requirements of a physical reservoir including high dimensionality, non-linear dynamics and fading memory [3, 24] (refer to [4] for a detailed description of requirements of a physical reservoir). These properties arise from the emergent memristive behavior of the network related to the mutual interaction of a large number of memristive NW junctions. Indeed, due to the network connectivity, the interaction in between memristive junctions is responsible for the emergence of spatially correlated structures of network activity that depend on the spatial location and temporal sequence of electrical stimulations [32]. In particular, the memristive behavior at the NW intersections is regulated by resistive switching effects associated to the electromigration of Ag⁺ ions to form Ag conductive filament across the polyvinylpyrrolidone (PVP) insulating layer (thickness of ≈1–2 nm) that surrounds the NW inner cores, as schematically depicted in figure 1(b) [32]. The switching mechanism related to Ag dynamics is volatile [45–47], meaning that the conductive filament at the intersection in between NWs spontaneously dissolves after stimulation. The competition in between filament formation and spontaneous dissolution at NW junctions leads to an emergent potentiation of the network under voltage pulse stimulation with progressive increase of device conductance and subsequent network relaxation to the ground state. Indeed, while temporally correlated voltage pulses applied in between two areas of the network as reported lead to a progressive increase of the effective network conductance (and thus flowing current) (figure 1(c)), a progressive relaxation to the ground state can be observed by monitoring the effective conductance with a low read voltage after relaxation (figure 1(d)). In other words, the memristive network state depends not only on the programming pulses by also on the history of pulses applied in the past. In this context, the interplay in between potentiation and spontaneous relaxation of different network areas through stimulation of spatiotemporal voltage pulse patterns in multiterminal configurations have been exploited for non-linear transformation of the input signals in the context of unconventional computing [3]. Notably, experimental results have shown that network dynamics can be regulated from the µs up to the hundreds of seconds timescale depending on the stimulation conditions [3, 32].

Zoom In Zoom Out Reset image size

Modeling of the emergent spatiotemporal dynamics of the reservoir was performed by mapping the nanonetwork in a grid-graph model [48], as schematized in figure 2(a). This model, that approximates the nanonetwork as a continuous and uniform medium (as expected for high density networks [49–51]), is based on the parcellation of the network domain that is then abstracted as a regular grid graph where interaction in between network areas (nodes) is provided by memristive connections (edges). A detail of a memristive edge interaction is schematized in figure 2(b), where the edge conductance depends on the history of electrical stimulation. Dynamics of each edge can be modeled by coupling an equation for electron transport with an equation regulating the memory state of memristive edge. Linear conduction is assumed for electron transport while a voltage-controlled potentiation-depression rate balance equation was exploited to model the memory state of the memristive edge [52].

Zoom In Zoom Out Reset image size

The current flowing in edge ij is described by the relation:

$$\begin{equation}{I_{ij}} = \left[ {{G_{{\text{min}}}}\left( {1 - {g_{ij}}} \right) + {G_{{\text{max}}}} \cdot {g_{ij}}{ }} \right]{ }\Delta {V_{ij}}\end{equation} \tag{ 1 }$$

where $\,\Delta {V_{ij}}$ is the voltage difference between nodes i and j acting as driving force, ${g_{ij}}$ is the normalized conductance, and ${G_{{\text{min}},ij}}$ and ${G_{{\text{max}},ij}}$ represents the minimum and maximum conductance values, respectively, of the memristive edge. The memory state dynamics are described through a balance rate equation:

$$\begin{equation}\frac{{{\text{d}}{g_{ij}}}}{{{\text{d}}t}} = {\kappa _{{\text{P}},ij}}\left( {{V_{ij}}} \right) \cdot \left( {1 - {g_{ij}}} \right) - {\kappa _{{\text{D}},{ }ij}}\left( {{V_{ij}}} \right) \cdot {g_{ij}}\end{equation} \tag{ 2 }$$

where ${\kappa _{{\text{P}},ij}}\left( {{V_{ij}}} \right)$ and are the potentiation and depression rate coefficients that endow an exponential relationship with the applied voltage to model the diffusion processes of ionic species:

$$\begin{equation}{\kappa _{{\text{P}},ij}}\left( {{V_{ij}}} \right) = {\kappa _{{\text{P}}0}}{\text{exp}}\left( { + {\eta _{\text{P}}}{V_{ij}}} \right)\end{equation} \tag{ 3 }$$

$$\begin{equation}{\kappa _{{\text{D}},{ }ij}}\left( {{V_{ij}}} \right) = {\kappa _{{\text{D}}0}}{\text{exp}}\left( { - {\eta _{\text{D}}}{V_{ij}}} \right)\end{equation} \tag{ 4 }$$

where ${\eta _{{\text{P}},}}{\eta _{\text{D}}}$ are the transition rates and ${\kappa _{{\text{P}}0}},{\kappa _{{\text{D}}0}}$ are constants. Under these circumstances, equation (2) can be recursively solved as (by considering $\Delta t > 0$):

$$\begin{align} {g_{ij,t}} &= \frac{{{\kappa _{{\text{P}},ij}}}}{{{\kappa _{{\text{P}},ij}} + {\kappa _{{\text{D}},ij}}}}\left[ {1 - {{\text{e}}^{ - \left( {{\kappa _{{\text{P}},ij}} + {\kappa _{{\text{D}},ij}}} \right)\Delta t}}} \right] \nonumber\\ & \quad + {g_{ij,t - 1}}{{\text{e}}^{ - \left( {{\kappa _{{\text{P}},ij}} + {\kappa _{{\text{D}},ij}}} \right)\Delta t}} \end{align} \tag{ 5 }$$

where ${g_{ij,t}}$ and ${g_{ij,t - 1}}$ represent the normalized conductances at times $t$ and $t - 1$ of memristive edge $ij$, respectively.

When a specific area of the network (node) is stimulated by means of an applied voltage, the conductance of edges of the network evolves over time giving rise to an emergent collective behavior of the network that depends both on the spatial location and temporal sequence of the input signal. Despite not including the effect of network inhomogeneities, the grid-graph modeling has been shown to well describe emergent dynamics of experimentally realized multiterminal memristive NW networks, including short-term plasticity, pair-pulse facilitation and heterosynaptic plasticity [3, 48]. For all these reasons, the grid graph model can be exploited to explore different implementations of RC in memristive nanonetworks, where device geometry and positions of electrodes acting as neuron terminals are mapped onto the grid graph topology.

Conventional two-terminal memristive devices have been exploited for the realization of RC systems by exploiting their internal short-term dynamics [17–20]. In this case, a memristive device can be considered as a neuron that possesses a recurrent connection with a synaptic weight less than 1, where the recurrent node continuously decay its state if a pulse input is not provided [18]. The reservoir state is represented by the device conductance. Since each memristive device is able to process a single input pulse stream, an array of discrete memristive cells is required for solving computing tasks that require the transduction of the input into multiple pulse streams. In this case, the reservoir state is represented by the collective resistance states of the whole set of memristors. A conceptual schematization of RC implementation in conventional memristive devices is reported in figure 3(a) (configuration a). Differently from conventional RC systems based on two-terminal memristive devices where addressing of each memristive cell is required, nanonetwork-based RC systems require accessing the emergent behavior of the whole system while it evolves under electrical stimulation. For this purpose, different implementation strategies can be explored as detailed in the following.

Zoom In Zoom Out Reset image size

Computing capabilities assessed by means of simulations have been carried out by considering nanonetwork-based reservoirs as graphs where nodes are represented by single NW units, while edges represent their memristive interactions [53, 54]. In this framework, specific NW nodes have been allocated as inputs and outputs of the reservoir. For example, Zhu et al [34] analyzed through simulations the information processing capabilities of NW networks by applying an input in form of a voltage signal in between a couple of selected source and drain NW nodes (reservoir input nodes), while reservoir outputs were recorded at other selected measurement NW nodes. Similarly, simulations by Fu et al [24] and Hochstetter et al [37] analyzed the implementation of nonlinear transformation tasks. While these simulations have strongly contributed to the understanding of computing capabilities of NW networks, it is experimentally unfeasible to access and record signals from each single NW node in very large networks. In this context, simulation by Daniels et al [31] reports that computing capabilities are preserved when considering realistic electrodes, showing performances close to upper bounds achievable by considering information from each wire. With a similar approach, pioneering works reported an experimental implementation of RC paradigms in multiterminal devices based on self-assembled systems where multiple electrodes allow to access signal from different areas of the networks [25, 39]. Similarly to simulated multiterminal devices, also in these works the input in form of a voltage difference was applied in between selected source/drain nodes, while exploiting remaining electrodes for measuring output signals. Also, multi-electrode devices based on networks contacted by an array of input electrodes and an array of output electrodes were exploited for neuromorphic applications in the work by Diaz-Alvarez et al [55], where specific combinations of electrodes can be selected through electromagnetic relays that control the operative input/output channels. The common demonimator of all these implementations is the exploitation of separate and distinct sets of electrodes as network inputs and outputs. It is worth noticing that a similar implementation strategy was exploited also in case of a multichannel skyrmion-based reservoir [56].

Despite previous works reported several variations on the theme, broadly speaking these implementations can be conceptually schematized as in figure 3(b), where generic input(s) transduced in a sequence of electric signals are applied to different channels represented by input/output electrode couples that can be selected through appropriate relays (configuration b). Note that in the implementations reported in [25, 39] input electrodes are represented by a single couple of source-drain electrodes, since the input is encoded in a single sequence of time-dependent electrical signals applied to one source electrode while grounding the selected drain electrode. In the configuration reported in figure 3(b), reservoir outputs to be passed to the readout can be represented either by (a) voltages measured at output electrodes (in this case, high impedance output channels to measure voltage at terminals B are ideally required, i.e. R → ∞) or by (b) currents flowing in between output electrodes and ground.

As an alternative, RC can be implemented by considering an unconventional configuration where electrodes can act simultaneously as inputs and outputs [3], as conceptualized in figure 3(c) (configuration c). Here, input signals are applied to terminals A of resistances R that are in series to each electrode pad, while outputs are represented by the voltage measured at terminals B of the same resistance. In this fashion, the same pad electrodes can be indifferently and even simultaneously exploited as input and outputs of the reservoir without any need of electromechanical relays.

Memristive RC systems require the transduction of the computing task inputs into one stream or, more in general, multiple streams of voltage pulses composed by n timeframes. As an example, in a pattern recognition task each input pattern can be transduced to a set of pulse streams each corresponding to a pattern row (spatial domain), while each pattern column is encoded in the temporal sequence of electrical pulses (temporal domain). The response of the two different implementations of RC in nanonetworks was tested by considering a simple task that consists in recognizing digits from input images reported in figure 4(a). Figure 4(b) reports an example of transduction of the 5 × 4 input pattern corresponding to digit '9' in a sequence of voltage pulses, where the input is encoded in both temporal and spatial domains. In particular, each pulse stream corresponding to a pattern row can be divided into timeframes, each corresponding to a pixel of a pattern column. In this case, the input pattern was transduced in five pulse streams, each composed of four timeframes. Each timeframe can be further divided in a 'stimulation' section and a 'read' section, as schematized in figure 4(c). During the stimulation section of each timeframe, the corresponding channel/pad of the reservoir is stimulated with a V_p pulse if corresponding to a white pixel. The following read section allows to evaluate the evolution of the internal reservoir state over timeframes, where the final read of the reservoir state represents the nonlinear transformation of the input to be passed to the readout for classification (final reservoir state).

Zoom In Zoom Out Reset image size

Stimulation and read configurations as well as the evolution of internal dynamics and output voltages of the NW networks reservoir in configuration b (figure 3(b)) and c (figure 3(c)) during stimulation with digit '9' are reported as examples in figures 5 and 6, respectively (supplementary note 1). Here, the two implementation strategies are compared, using the grid-graph approach for modeling the reservoir internal dynamics. In all configurations, resistances of 82 Ω were exploited to realize the corresponding circuits, while model parameters were extracted from experimental data (details in supplementary information S1).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In configuration b, during the stimulation section of each timeframe the electrodes corresponding to channels that require a stimulation with a voltage pulse are connected, while all other electrodes are left floating, as schematized in figure 5(a). Instead, during the read section, a reading voltage is applied to a selected input electrode and output is recorded at terminals B of four selected output channels while keeping all the other terminals floating, as schematized in figure 5(b) (this reading scheme was adopted for a more direct comparison with the unconventional implementation of configuration c). Figure 5(c) reports the voltage distribution across the NW networks over timeframes during stimulation sections. As can be observed, the voltage drop occurs mainly in between areas of the network where input and output electrodes are located. Note that during timeframes where no pulses are expected the corresponding electrodes are not influencing the overall distribution of voltage since they are left floating. Since the conductive pathways tends to form perpendicularly to equipotential lines under the action of the electric field, the emergent dynamics is characterized by the formation and spontaneous relaxation of conductive pathways connecting corresponding input and output electrodes. These dynamics leads to reservoir output voltages (recorded at terminals B during the reading section of each timeframe) reported in figure 5(e), and to the corresponding histogram of the final output voltages (i.e. the final reservoir state) reported in figure 5(f).

In configuration c, during the stimulation section a voltage pulse is applied to corresponding electrodes that require stimulation, while other electrodes were kept at 0 V, as schematized in figure 6(a). Instead, during the reading section, a reading voltage is applied to a selected pad, while output is recorded at terminals B of other four pads, as schematized in figure 6(b). Note that in this latter configuration (configuration c), all terminals A (except from the electrode where a read voltage is applied) are kept at 0 V. Figure 6(c) reports the voltage distribution across the NW networks over timeframes during stimulation sections. As can be observed, in this case the voltage drop over the network relies on mutual differences of applied voltages among electrodes, driving the formation of conductive pathways in between network areas where a voltage difference is present, as reported in figure 6(d). As an example, during timeframe t1 of digit '9' pads 1, 2 and 3 are simulated with a voltage pulse while other electrodes are grounded. This results in a nearly equipotential area on the right side of the network. In this case, the voltage drop is mainly localized across couples of electrodes 1–5 and 3–5 and, less intensely, across couples of electrodes 3–4 and 1–4. These dynamics leads to reservoir output voltages (recorded at terminals B during the reading section of each timeframe) reported in figure 6(e), and to the corresponding histogram of the final output voltages (i.e. the final reservoir state) reported in figure 6(f).

As can be observed, the two different configurations result in peculiar evolution over timeframes of the output voltages. This peculiar evolution reflects to different histograms representing the final reservoir states. In other words, this means that the two configurations perform a different nonlinear transformation of the input pattern. This is due to the different evolution of the internal conductivity map in the two different stimulations, as revealed by modeling. Indeed, the same stimulation pattern of digit '9' was observed to result in strongly different patterns over the memristive network. These activation patterns are related to the interplay in between formation and spontaneous relaxation of conductive branches and pathways connecting electrodes. In particular, these branches forms depending on the local voltage gradient induced by stimulation, where the electric field spatial distribution for each timeframe depends on spiking electrodes.

The evolution over timeframes of reservoir output voltages while feeding the reservoir with the ten digits, the corresponding final reservoir conductivity maps and the final histograms for configuration b are reported in figure 7. Similarly, results for configuration c are reported in figure 8. As can be observed, in both configurations the final reservoir states (i.e. conduction maps and output voltage histograms) are significantly different, showing the ability of both reservoir implementations to separate the 10 digit inputs. Final reservoir states represented by the standardized reservoir output voltages at the last stimulation timeframe are then used as input to the readout for training and classification. The readout is composed by a 4 × 10 network, where output neurons labeled from 0 to 9 represent the predicted digit of the input pattern. For this purpose, a single layer feed-forward neural network was trained by minimization of the loss function through the Adam algorithm [57], similarly to a previous work [3]. After training (details in supplementary S2), both RC implementations can recognize all of the 10 digit inputs. In particular, similar learning curves were observed, suggesting that the two implementations endow comparable separability properties, at least for this task (details in supplementary information S2).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size