Performance of reservoir computing in a random network of single-walled carbon nanotubes complexed with polyoxometalate

Molecular neuromorphic devices are composed of a random and extremely dense network of single-walled carbon nanotubes (SWNTs) complexed with polyoxometalate (POM). Such devices are expected to have the rudimentary ability of reservoir computing (RC), which utilizes signal response dynamics and a certain degree of network complexity. In this study, we performed RC using multiple signals collected from a SWNT/POM random network. The signals showed a nonlinear response with wide diversity originating from the network complexity. The performance of RC was evaluated for various tasks such as waveform reconstruction, a nonlinear autoregressive model, and memory capacity. The obtained results indicated its high capability as a nonlinear dynamical system, capable of information processing incorporated into edge computing in future technologies.


Introduction
Reservoir computing (RC), a recently introduced framework derived from recurrent neural networks, is now attracting attention.RC requires a small computational burden of the training process because only the weight of the readout part is usually trained toward the target function [1][2][3].The main layer of RC was originally proposed as a network with a large number of nonlinear nodes coupled with each another.This network is called reservoir and is expected to be replaced by various physical systems [4].The exploitation of physical systems as information-processing devices for RC is particularly suited for edge computing devices in a decentralized manner [5].
Reservoirs that exploit physical dynamics or dynamics in signal response, they are called physical reservoir computing (PRCs) [3,4].Many types of PRCs have recently been proposed.Their physical resources are widely diverse, varying from photonics [6] and spintronics [7], nanomaterials, [8] quantum [9,10], and solid-state devices [11] to mechanics [12] and biological materials [13].Among the others, nanomaterials have the ability to form complex networks and are expected to realize highly integrated RC chips.
In a previous work, we reported the generation of a spike signal from a device composed of a random and extremely dense network of single-walled carbon nanotubes (SWNTs) complexed with polyoxometalate (POM) [14].It is known that the POM molecule displays a remarkable redox activity [15], which has been utilized as the source of fluctuations of electrical noise in molecular devices [16,17].In addition to utilizing the stochastic fluctuation of the redox state, we proposed an abstract model of the network by assuming a two-dimensional (2D) structure of molecular junctions for the spike signal.The model applies a special state transition rule depending on the degree of multireduction state of POM.According to this rule, electrons accumulated at POM will be transferred to another adjacent POM through a SWNT connection, thus triggering a chain reaction of electron flow.The results yielded by the model were consistent with the experimental results of the spike signal from the network device.
Furthermore, we demonstrated the learning ability of the model by introducing RC, which utilizes spiking dynamics and network complexity.In the simulation, a 2D POM/SWNT network model consisting of 5500 POM molecules connected via SWNTs was used as a reservoir, where 50-100 POM molecules were randomly selected as a readout; that is, the numerical state of the selected POM was used as the output data for RC.According to the definition of the model, the input signal is an applied voltage from the device terminal, and the output is the current.The results indicating a high performance for a benchmark task of RC made us expect the POM/SWNT network to have high capability as a reservoir.However, in a real experiment, detection of multiple currents at each single molecule from an aggregated small area is almost impossible.
In this study, we experimentally realized RC using multiple signal detection from 2D SWNT/POM random networks.Despite many limitations associated with sample preparation and current measurement conditions in realistic current measurements, the results showed the rudimentary functionality of RC.In the next sections, we evaluate the availability of nonlinear dynamics in SWNT/POM random networks as computing resources for a variety of RC tasks.The origins of the signal dynamics and nonlinearity that are related to molecular redox and/or electrochemical reactions are discussed.

Experimental methods
The random network of carbon nanotubes (CNTs) used in this study employed two different samples, as shown in figures 1(b) and (c): SWNTs dispersed on a paper filter and on a CNT composite paper.Our experimental goal was to evaluate the SWNT/POM network capability of RC, although the trials using CNT composite paper gave us important new aspects.
The sample preparation of SWNTs dispersed on a paper filter was as follows.Metallic SWNTs (99% purity, purchased from NanoIntegris) with an average diameter of 1.4 nm and an average length of 1 μm were dispersed in super dehydrated N,N-dimethylformamide (DMF) at a concentration of 500 ng ml −1 .A total of 10 μl of the SWNT dispersion solution was drop-casted onto a paper filter (ADVANTED filter paper with a thickness of 0.22 mm).In this work, phosphomolybdic acid (H 3 PMo 12 O 40 ; PMo 12 hereafter), a compound in the POM family, was employed.PMo 12 powder (hydrated, purchased from Sigma-Aldrich) was dissolved in DMF to form a PMo 12 solution with a concentration of 20 mg ml −1 .A total of 10 μl of a PMo 12 solution was drop-casted onto the paper filter at a gray SWNT circle with a radius of approximately 10 mm.The filter was annealed on a hotplate at 150 • C under atmospheric pressure and room temperature of 20 • C for 30 min.For the experiment described in section 3.4, a mixed solution of SWNT and PMo 12 with the same density as ultrasonic treatment for 30 min was dropped on a clean paper filter, and the filter paper was subsequently annealed in the same manner.
The CNT composite paper was prepared by mixing the pulp and disparted CNTs with catechin in pure water, followed by hot pressing and drying [18].The sheet resistance could be controlled by the mixing rate of the CNTs and pulp and the paper thickness.A total of 5 μl of a PMo 12 solution was drop-casted onto the CNT composite paper, and the paper was annealed in the same manner.
The proposed setup employed two different systems, as shown in figures 1(d) and (e): one used a manual prober system and another used an array of probe pins.The manual prober system is an ambient probe station (Apollowave), and new sharp BeCu probes were used.The contacting pressure was checked by probe bending using a CCD (Charge-Coupled Device) camera, as in the general procedure.The AC input bias was sourced from NF Wave Factory WF1945, and the output current was converted into voltage using a currentinput preamplifier (Keithley 424) with appropriate amplification.The time-domain signal of the output was measured using an oscilloscope.
The array of plunger probe pins shown in figure 1(e) is an 8 × 8 bunch of Rh-coated plunger probe pins with a pitch of 640 μm.The 64 pins in a 5 × 5 mm 2 area can simultaneously contact the sample with the same contact pressure.In the measurement, four pins in the array were connected to the input voltage, and the remaining 60 terminals were connected to a current/voltage (I/V) amplifier with an amplification of 5 × 10 7 , where the output voltage from the I/V converters was red by Arduino (Arduino Mega 2560).The time sequential input signal u(t) was generated by Arduino, and the signal u(t) was amplified and offset by a homemade OP-amp circuit to be an input voltage V(t).We prepared eight electric boards with switchable eight circuits before the I/V converter, and the switching was controlled by Arduino.Thus, the current signal from 60 terminals was measured in parallel and repeated automatically.In this work, the learning of RC, training of the weight of the readout, w out , was executed with a simple linear regression to make the system output Y (t) closer to the target signal T (t) by using the dataset of the current I (t) from multiple nodes of N = 60 (1) We used three tasks: periodic signal prediction, second-order nonlinear autoregressive moving average (NARMA2), and memory capacity (MC) tasks.The error was evaluated using the normalized mean square error (NMSE) to quantify the deviation between the target and predicted system output signals.The error expression is as follows:

Current measurement by the probing system
We first observed the voltage-current (I-V) response of the SWNT/POM network on the paper filter using a manual probing system.Figure 2(a) shows a sinusoidal signal with V p-p of 8 V and 1 Hz, and figure 2(b) shows an example of the measured current before fixing the sample preparation condition.Depending on the sample preparation conditions, the characteristics of the current varied: no current, a large linear current, and a very noisy current with spiking and sudden jumps.The current depicted in figure 2(b) shows typical features of a current that has sudden jumps with spiking and noise.These features were expected because the SWNT/POM network spiking device in our previous work showed similar I-V characteristics.The current shows a similar tendency in the sinusoidal repetition, but the features are not reproducible in long time measurements.Curiously, the measurement distance between the two probes varied widely, although the current values and characteristics did not show a clear distance dependence.
We found an adequate sample preparation condition to suppress the large noisy spiking and the sudden jump while maintaining current nonlinearity.Figure 2(c) shows the measured current at six different points on the SWNT/POM network sample.The current exhibits a nonlinear response against the input sinusoidal voltage.We can see that the respective characteristic shape of nonlinearity repeatedly appears.We gathered the current data, which showed adequate nonlinear data at 50 points.Figures 2(d)-(f) show the results of periodic signal prediction tasks by using these data as 50 nodes.The target signals are (d) cubic, (e) double wave, and (f) third-order harmonic wave of the input sinusoidal wave.The reconstructed system output Y (t) showed a very good agreement with the target signal T (t).The low error for the case of the third-order harmonic wave indicates that this system has high harmonic informational transform ability.
The RC time sequential data are generally treated in three domains: first washout, training, and evaluation.RC should be evaluated fairly based on results obtained in the evaluation period, the input and output data of which are not used to execute weight of readout, w out , learning.However, figures 2(d)-(f) show the system output Y (t) and the target signal T (t) in the training domain.In the case of figure 2, if the signal characteristics at each node are maintained, it is expected that the results in the evaluation period are not significantly different from those in the training period.The current data presented in figure 2 was acquired using an oscilloscope with repeated manual movement of the probe.It is a very troublesome work and not a rational method for the detection of multiple nodes in a reservoir.

Automatic current measurement for RC
Next, we used the experimental setup shown in figure 1(e).An array of probe pins measured the current from 60 points on the sample.The distance between the voltage source and the drain pin varied in the range of 640-1800 μm.Each pin had a single lead, connected to the electrical switching boards with an I/V converter.The data acquisition time was very short, as compared with that of the previous experiment.However, the sample that was adjusted for the previous measurement condition did not yield sufficient current in this experimental setup.Here, we measured CNT composite papers with a composite random networking structure with pulp because they were easily treated and had a sufficiently small resistance for current measurement.We chose a CNT composite paper made of pulp with a CNT ratio of 18:1 and a sheet resistance of 1.6 kΩcm −2 .

Evaluation of nonlinear signal transformation
Figure 3 shows the results of the periodic signal prediction tasks performed using 60 current data for (a) CNT composite paper, (b) CNT composite paper with a POM solution after annealing, and (c) CNT composite paper with a POM solution without annealing.The system outputs Y (t) were generated in evaluation time steps from 1000 to 2000, where the data in steps from 100 to 1000 were used for training.We were surprised that the CNT composite paper without added POM showed a signal reconstruction ability for some tasks, although the current from the CNT composite paper seemed almost linear.We considered that some periodic noise superimposed on the data might assist with the reconstruction tasks.
The CNT composite papers with added POM showed better results for every task than those with CNT composite paper, as expected.However, we found that the data obtained from the CNT composite paper with added POM without annealing was the best.Especially for the tasks for quadruple and sawtooth waves, which are very difficult tasks, the reproduction quality of the CNT composite paper with added POM under the wet condition is extremely high.As the added POM was dissolved in a DMF solution, the DMF solution was supposed to play an important role in the improvement of the results.We checked the results for only DMF solution and pure water, both with and without CNT composite paper, as the control experiment.On the basis of the results, we conclude that the current flowing though these solutions under this measurement condition has large nonlinearity, which enhances the capability of performing the periodic signal reconstruction task.The results for these solutions were lower but almost the same as those of the CNT composite paper with added POM without annealing.The signal response from the solutions is also very interesting in terms of the material variety; therefore, we performed the study of RC using solution materials, and the results will be reported in another research paper [19].

Evaluation of signal dynamics
In addition to the periodic wave reconstruction task, the complexity of the nonlinear response from the materials exploited as a reservoir and the relevance of the signal in time series should be evaluated.The information transferers within nodes with a small time delay yield the dynamics of the signal conveyance in the reservoir.It is known that any dynamical system has the potential to be a reservoir for information processing [20].This is the reason why physical dynamics are expected to be exploited as reservoirs.The information of the past signal is kept for a while in the reservoir, resulting in short-term memory.These complexities in time series make RC capable of solving complex time-series prediction tasks.
Here, we evaluate the performance of dynamics in signal response from the CNT composite papers with added POM under the same experimental conditions as in the previous section, using a NARMA task and a MC task.The NARMA task [21] is a widely used benchmark for RC, when evaluating the ability to duplicate a higher-order dynamical model, in which the target signal is composed of past self-signals and adjusted input.We used the NARMA2 task as a benchmark.NARMA2 is a second-order nonlinear model, where the target signal T (t) is defined as A well-known feature of RC is its fading (short-term) memory, meaning that the current reservoir state contains information from the recent past inputs, but it is unrelated to older ones.The MC is introduced by Jeager [22] to reflect the correlation between the current reservoir states, Y (t), and past inputs.MC in RC networks is defined to be represented by the squared correlation coefficient of the target testing signal, u (t − k), and system output, Y (t).MC k is defined as below for the respective delay k, where the correlation ranged from 0 to 1 MC is defined as the sum of the values of MC k for all time delays k.
Based on this definition, we obtained MCs for higher-order target signals [20], where the target signal, T q , was constructed from the kth delayed original input signal (u(t − k)) according to the qth order of Legendre polynomials (see the appendix) Under the same nonlinear order, MC q k was used to calculate the degree of correlation between the optimally trained output signal, Y (t), and the target signal, T q .MC q are summed over all MC q k as same as equation ( 5), for the respective nonlinear order of q.When q is 1, equation ( 6) is same with equation ( 4) that is first order MC k .This procedure details are described in our previous report [23,24].In this paper, MC q s are shown as MCs against degree of higher nonlinearity.
The input signal u(t) was a random sequence of 4000 steps.For the NARMA2 and MC tasks, the u(t) was mapped in the ranges of [0, 0.5] and [−1, 1], respectively, which are the standard specified ranges for these tasks.The first 500 steps were used as washout, 501-3000 steps were used for training, and the last 1000 steps were used for evaluation.
The input voltage V(t) formed a random square wave within the range of ±3 V with a time step of 10 ms, as shown in figure 4(a).Figure 4(b) shows an example of the current flow at one node, which shows transient dynamics.The features of the transient within the signal time step have some diversity, even in the current observed from the same sample.Signal diversity also appeared in the acquired time sequential data.Figure 4(c) shows ten data points of I(t), which have a slight mismatch in signal sequence.
Figures 4(d)-(f) show the results of the NARMA2 task for the following samples: (d) CNT composite paper, (e) CNT composite paper with POM after annealing, and (f) CNT composite paper with a POM solution without annealing.It is clear that the signal from the CNT composite paper does not contribute to the reconstruction of the NARMA2 target.The current measured on the CNT composite paper seems linear, and signal diversity such as transients and mismatches in the shape of the sequential data were hardly observed.The error of the CNT composite papers with POM decreases, which indicates that the output current not only has nonlinearity but also keeps past information.The dried sample performed worse than that under wet conditions in a similar manner to that observed for the periodic signal prediction tasks.
Figures 4(g)-(l) show the results of the first-order MC k and MCs for degree of nonlinearity.The MC k decreases as the delay k increases, which indicates that the information of the distant past becomes more difficult to include in the output data.The pristine CNT composite paper has no past information, as shown in figure 4(g), in addition to the pieces of CNT composite paper with POM maintained until two steps, as shown in figures 4(h) and (i).CNT composite paper with POM under wet conditions exhibits short-term memory (although very small) from steps three to eight, as shown in figure 4(i).These results suggest that the CNT composite papers with added POM have short-term memory.However, their capacity is not sufficient to have the ability to solve novice classes of nonlinear dynamical tasks such as NARMA2.Moreover, it is clear that the origin of the higher reservoir performance depends heavily on the electrochemical reaction of molecules within solutions.In our previous work, we proposed a 2D SWNT random network with molecular junctions having a reservoir ability, originating from the molecular redox functionality.In the model, it was defined that the current flows through SWNTs and not in the solution.The CNT composite paper with added POM showed insufficient reservoir ability after drying.It is considered that the added molecules were not inserted in the junctions in the SWNT network in the CNT composite paper.Thus, we attempted to fabricate a sample that showed reservoir ability without adding a solution.The procedure is described in the next section.

Evaluation of the dried SWNT/POM network
As array of probe pin systems was designed to measure the current within a 10 μA range, and a certain uniformity of the data was also required because increasing the number of inadequate data degraded the reservoir ability.In this experiment, no data selection was performed to improve the results.To create an SWNT/POM network without an electrochemical current, the CNT/POM network should have an appropriate degree of POM insertion at junctions without excessively decreasing the net conductance of the sample because POM molecules are inherently insulating materials.We prepared a dried SWNT/POM network by mixing SWNT and POM solutions with sonication and further annealing, as mentioned in the experimental section.
Figures 5(a)-(c) show the results of the NARMA2, first-order MC k , and MCs of the dried SWNT/POM network, where the experimental setup is the same as in the previous section.Although the error of the NARMA2 task does not fall below that of the CNT composite paper with the added POM solution, a significant increase in MC was attained.The MC k of the delay from 3 to 7 considerably increased, as compared with that of the CNT composite papers, and a short memory of the past 8-10 steps still exist.We consider that the MC increase indicates that the signal transform pass in the SWNT/POM network includes multimolecular junctions.Multiple molecular redox reactions should provide signal retention, which becomes memory functionality.We expected that the MC increase would also provide better results for the NARMA2 task; however, they are not consistent.The first-order MC of the dried SWNT/POM network is significantly higher; however, MCs at higher nonlinearities are small.It was confirmed that MCs in higher order nonlinearities are required for high-performance NARMA tasks [25].
Figure 5(d) shows the output current data of the SWNT/POM network plotted against the input voltage in comparison with that of the CNT composite paper (figure 5(e)).It can be observed that the output-to-input relation in the SWNT/POM network is scattered in a certain band width, whereas that of the CNT composite paper shows an almost unique relationship as a normal I-V characteristic.The scattering of the output against the input originates from the dynamics of the signal response.

Effect of input signal frequency and node number
Unlike the CNT papers shown in figures 3 and 4, the results of the dried SWNT/POM networks varied widely, despite using the same sample preparation process as the CNT experiment.The SWNT/POM networks generally exhibited higher first-order MCs than the CNT papers.The sample shown in figures 5(a)-(d) exhibits the highest MC value (4.6), though the NARMA2 error is poor.
We found that a higher frequency input signal improved the results of the NARMA2 task. Figure 6 shows two of dried SWNT/POM networks, labeled (I) and (II), using a node number, V p-p , and input frequency of 60, 6 V, and 500 Hz, respectively.The NMSE of the NARMA2 were 0.0934 and 0.023 for (I) and (II), respectively, and the MC for (I) and (II) were 3.61 and 3.41, respectively.
Figure 6 show first-order MC k and MCs for (a) networks (I) and (b) networks (II) and the node number dependence of (c) MC and (d) the NMSE of NARMA2 for the networks.The node number dependence values were calculated by RC learning for different numbers of nodes by randomly selecting from the data of 60 nodes.Figure 6(c) shows that, in both networks, the MC initially increases with an increase in the number of nodes until 12 nodes are in the network.After surpassing 16 nodes, the MC of network (I) is relatively constant but decreases slightly as the number of nodes increase.The MC of network (II) continues to increase with more nodes, but at a progressively slower rate as the number of nodes increase.
Figure 6(d) shows that, in both networks, the NMSE error of the NARMA2 decreased significantly until 12 nodes are in each network.The NMSE of network (I) keeps small decreasing until 56 nodes, where it drastically decreases from 56 to 60 nodes.The NMSE of NARMA2 in the network (II) decreases steady as the number of nodes increases until 40.From 40 to 44, and from 48 to 52, the error of network (II) decreases largely.From 52 to 60, both the MC and NARMA2 error seems almost saturated.Since both networks show similar tendencies as they accumulate 12 nodes, we infer that this number of nodes significantly affects the performance of our RC system.However, expanding the network beyond 12 nodes can improve the RC performance further.Substantial decreases in NARMA2 error, such as those seen between 57 and 60 nodes in network (I) and between 40 and 44 nodes in the network (II), are considered more likely represent accidental results.These changes suggest that the samples are lacking the uniformity.
The results shown in figure 6 tell us that the RC performance will not be improved by increasing the node number alone without fundamental improvement of the process required to prepare the SWNT/POM network.However, a subset of data, such as those seen between 57 and 60 nodes in network (I) and between 40 and 44 nodes in the network (II), shows very high computational power.They suggest that a sample that consists exclusively of higher computational power components will have higher performance than could be achieved in this study.A more rigorous exploration ought to be conducted to maximize the systems computing ability, and to harness the coupling of the functionality of complex network structure with molecular linking.

Conclusion
We performed RC using multiple signals collected from a SWNT random network complexed with POM.The signals showed a nonlinear response with wide diversity originating from the network complexity.The acquired data demonstrated good performance of the periodic wave reconstruction task.A multiway data acquisition system with a probe pin array was used to measure the current from multiple nodes that were physically separated points in the SWNT/POM network.The CNT composite papers showed reservoir functionality after POM addition; however, we found that the electrochemical current through the solution significantly affected the nonlinear signal response and the signal transfer dynamics.We prepared a dried SWNT/POM network that showed a comparable ability for the NARMA2 task of the CNT composite paper with POM under the wet condition.The dried SWNT/POM network exhibited a higher MC than the others.The lower error of the NARMA2 task was obtained in a sampling parameter with a higher input signal frequency.
In this study, the reservoir functionality of the SWNT/POM network that was predicted by a model simulation in a previous study was experimentally demonstrated, although the reservoir's ability was lower than expected.We found that the molecular electrochemical reaction and multiple molecular redox processes in these complex networks yield nonlinear and dynamic responses that are exploited in RC; these responses have pros and cons for their respective RC tasks.These results are expected to contribute to the design and fabrication of SWNT/molecular networks with high reservoir functionalities for future development.A physical reservoir consisting of nanomaterials may become a computing system with low cost, low power consumption, and highly integrated hardware devices for increasingly important edge computing.

Figure 1 .
Figure 1.(a) Conceptual illustration of PRC, where the CNT/molecule network is a reservoir, and the colored small circles are input and output terminals.(b) Sample preparation process of a SWNT network spread on a paper filter.(c) Sample preparation process of CNT composite paper.(d) Measurement setup of a manual prober.(e) Measurement setup of plunger probe pins.

Figure 2 .
Figure 2. (a) Sinusoidal signal with frequency f = 1 Hz, that is, input voltage V(t) to the SWNT/POM network through the manual probing system.(b) Example of the measured current.(c) Superimposed six measured currents showing reproducible nonlinearity.Results of the periodic signal prediction task for the target of (d) cubic, sin 3 f; (e) double wave, sin 2f; and (f) third-order harmonic wave, sin 3f.The system output Y(t) and the target T(t) are plotted in red and black, respectively.Numerical values shown at the corner of the left bottom are NMSEs.

Figure 3 .
Figure 3. Results of the periodic signal reconstruction task for samples, (a) CNT composite paper, (b) CNT composite paper with added POM after drying, (c) CNT composite paper with added POM (wet condition).The input signal V (t) is sin f, where the V p-p is 6 and f is 1 Hz.The signal prediction tasks from the top are cubic, sin 3 f; double wave, sin 2f; third-order harmonic wave, sin 3f; quadruple wave, sin 4f; square wave; and sawtooth wave.Numerical values shown at the corner of the left bottom are NMSEs.

Figure 4 .
Figure 4. (a) Input voltage V(t) that is a random square wave with frequency f = 100 Hz.(b) Example of the current measured on CNT composite paper with added POM.(c) Superimposed ten measured currents.Results of the NARMA2 task for (d) CNT composite paper, (e) CNT composite paper with added POM after drying, (f) and CNT composite paper with added POM (wet condition).The system output Y(t) and the target T(t) are plotted in red and black, respectively.Numerical values shown at the left upper side are NMSEs.Results for tasks of first-order MC k (g)-(i) and MC (j)-(l), where the evaluated data are the same as the data used in the NARMA task of (d)-(f).

Figure 5 .
Figure 5. Results of the SWNTs/POM network after drying.(a) NARMA2 task, where the system output Y(t) and the target T(t) are plotted in red and black, respectively.Numerical values shown at the left upper side are NMSEs.(b) MC k and (c) MC against higher nonlinearity.(d) Output current data plotted against input voltage.(e) Output current data of CNT composite paper plotted against input voltage for comparison with (d).

Figure 6 .
Figure 6.Results of two SWNTs/POM networks after drying, where the input voltage V(t) is a random square wave with frequency f = 500 Hz.MC k and MC against higher nonlinearity for (a) network (I) and for (b) network (II).(c) MCs and (d) NMSE errors evaluated with different number of nodes for networks (I) and (II).The vertical axis of (d) is in log scale.