Performance improvement of in-materio reservoir computing by noise injection

Computation performance of in-materio reservoir device was evaluated by varying intensity of noise injection. Materials for the reservoir device was synthesized using a α-Fe2O3/titanium bismuth oxide composite by using the sol–gel method. The prepared samples were characterized by conducting X-ray diffractmetry, transmission electron microscopy, and energy dispersive X-ray spectroscopy to confirm presence of α-Fe2O3, TiO2, and Bi4Ti3O12 nanoparticles. The I–V and V–t curves show nonlinearity, and phase differences between input and output signals, and the fast Fourier transform of the V–t curve showed high harmonics at the input sine wave with 11 Hz of frequency. In the waveform prediction task, the prediction accuracy was improved only when a small intensity of white noise voltage was superimposed to the input information signal.


Introduction
Research on artificial intelligence (AI) began to attract attention at the Dartmouth Conference in 1956, 1) ushering in the first AI wave. During this wave, generative grammar, which was the basis for later natural language analysis, 2) retrieval, and knowledge representation, and a single-layer perceptron model, 3) an early concept of neural networks, were conceived. The second AI wave saw research on neural network backpropagation methods 4,5) and expert systems, 6) leading to the current third wave. Despite the technological developments in the first and second AI waves, the advancement ceased due to changes in the economic situation, low computing performance, 7) and lack of established methods for collecting and processing large amounts of data. In the 21st century, the progress made in neural network research, 8) application of Bayesian statistics to machine learning, [9][10][11] and the establishment of big data utilization technologies 12) had led to the third AI wave. This wave continues today, 13) and although the development of high-performance supercomputers has made it possible to address the complex issues with big data and AI, 14) the high power consumption remains a bottleneck in AI research. 13) There is a relationship between computational performance and power consumption in computer calculations, and reducing the power consumption is important for improving the cost performance. 14) In the future, as the amount and complexity of data required for learning increase in machine learning, the required power is expected to increase, which becomes a problem in maintaining sustainable energy.
Artificial neural network (ANN) is a mathematical model that mimics the neural network of a living organism by simplifying its structure and mechanism. Examples include feedforward 15,16) and recurrent neural networks, 17) which differ in terms of the connection mechanism between the layers and nodes. Among ANNs, reservoir computing (RC), which is expected to be faster and consume less power, is drawing attention. RC uses a recurrent neural network comprising three layers: an input layer, a hidden (reservoir) layer, and an output layer. 18) The unique characteristics of this network are that the weights between the input and reservoir layers and between the nodes within the reservoir layer are fixed, and only the weights between the reservoir layer and output layers are updated. 19) With the weights fixed, the amount of calculation can be significantly reduced, thereby reducing the power consumption. An important feature of RC is that a physical system can replace a reservoir layer with nonlinear characteristics, called a physical reservoir. Various types of physical systems have been used as reservoirs, including soft arms, 20) spintronic oscillators, 21) and optoelectronic neuromorphic devices. 22) In recent years, nanomaterials have attracted considerable attention as inmaterio reservoir layers. For example, atomic switch nanowires, 23,24) polymer, 19) and carbon nanotube-polyoxometalate networks 25) have been used to solve specific tasks in the RC framework. In specific tasks, it has also been reported that applying a certain amount of noise on the software simulation suppressed overlearning and improved generalization performance, thereby increasing prediction accuracy. 26) In this study, we focused on an α-Fe 2 O 3 /titanium bismuth oxide (Ti-Bi-O) composite material 27) as the in-materio reservoir. α-Fe 2 O 3 has low band gap (∼2.0 eV) and suitable valence band for water oxidation. 28) However, α-Fe 2 O 3 itself has low electrical conductivity. Its heterostructure with TiO 2 has been found to overcome the limitations because of effective transport of charge carriers. In addition, previous studies reported that TiO 2 with Bi atom doping leads to more effective charge separation and electrical conductivity. 27) Deionized water was used to increase the conductivity of the material and ion conductance is focused on as a nonlinear dynamical system. 29) We focused on electrochemical ionic reactions in α-Fe 2 O 3 /Ti-Bi-O composite material, which have applications in fields such as solar cells. Effective charge separation and low recombination 27) are expected to be applied to the reservoir layer as nonlinear dynamical systems of ion conductance. 27,30) Here, we evaluated the structural and electrical properties of the devices. Waveform generation tasks were performed to evaluate the RC performance controlled by noise injection.

Experimental procedure
α-Fe 2 O 3 and Ti-Bi-O were synthesized using the sol-gel method as described in a previous report. 27) A α-Fe 2 O 3 precursor solution was prepared with 0.2 M iron acetylacetonate (Sigma Aldrich, <=100%) added to a mixture of isopropanol (1.8 ml), ethylene glycol (0.1 ml), and hydrochloric acid (0.1 ml), and completely dissolved with stirring. A Ti-Bi-O precursor solution was prepared with 0.2 mol of titanium butoxide (Sigma Aldrich, ⩾90 ⩽ 100), added to a mixture of ethanol (20 ml) and sifiminodiethanol (3 ml), and dissolved completely with stirring. Subsequently, 0.01 mol of bismuth nitrate pentahydrate (Sigma Aldrich, ⩾98.0%) and titanium butoxide were added to the mixture and stirred for 10 min. After preparing these precursor solutions, the precursors were annealed at 500°C for 2 h, and powdered samples were obtained. A SiO 2 /Si wafer was used as the substrate for fabricating the composite devices. Au was deposited on a SiO 2 /Si substrate using optical lithography to create spider-like patterns of 16 electrodes. To fabricate the α-Fe 2 O 3 /Ti-Bi-O composite device, α-Fe 2 O 3 and Ti-Bi-O powders were dispersed in isopropanol and ethanol and dropcasted onto a substrate on which Au electrodes were formed. After the drop-cast substrate was heated and dried at a temperature of 80°C, Ti-Bi-O dispersion was drop-casted in the same manner, heated, and dried to obtain the final product: the α-Fe 2 O 3 /Ti-Bi-O composite device (Fig. 1). The fabricated materials were characterized using X-ray diffraction (XRD, SmartLab, Rigaku), transmission electron microscopy (TEM) (JEM-F200), and EDS mapping (JEM-F200). The XRD measurement conditions were as follows: scan rate of 0.01°/step, applied voltage of 45 kV, angular range of 10°-90°, and scan speed of 10°min −1 .
The I-V and V-t characteristics were measured to confirm the electrical properties of the α-Fe 2 O 3 /Ti-Bi-O composite device. In this experiment, all the measurements were performed with a drop of deionized water covering the material of the composite device. This is because the composite requires charge separation and transfer during water splitting to exhibit a conductive performance. The I-V characteristics were measured using a semiconductor analyzer (Keysight Agilent 4156 A) at voltages ranging from −5 to 5 V and a measurement interval of 0.25 V/step. The V-t characteristics were measured using a data acquisition system (National Instruments Model 9234) with software coded in LabVIEW with a sinusoidal input (11 Hz, 6 V pp , and a sampling rate of 1000 s −1 ). One of the 16 electrodes was used as the input, and the other 15 electrodes were used as the outputs. The Lissajous plots of the input and output signals were produced to confirm phase shifts. A fast Fourier transform (FFT) analysis was performed on the V-t measurement data to confirm the higher harmonics [integer multiples of the input sinusoidal wave frequency (11 Hz)].
A waveform generation task was performed using the same data acquisition system as that used for the V-t measurement, to evaluate the RC performance, as shown in Fig. 4(a). 6 V pp amplitude and 11 Hz sinusoidal waves were inputted to one electrode of the device, and the 15 outputs were used for learning by linear regression 31) to approach the shape of the target waves. The sum-of-products operation of the output signals can be expressed as å W x , 31) where W, x, and i are the weights between the reservoir and output layer, reservoir state that represents the output signal, and the number of output electrodes, respectively. The output signals were optimized by updating all W ; where W , out Y , target X, a, and I represent the W i matrix, target matrix, x i matrix, regularization factor, and identify matrix, respectively. A waveform generation task was then performed using the optimized weight values, and the accuracy was calculated. A total of 10 000 data points were used for the waveform generation task: 8000 for training and 2000 for testing.  Fig. 2(a). The Scherrer formula was used to approximate the crystallite size (CS) of the materials, as follows: 32,33) where CS, K, λ, β, and θ denote the crystal size, Scherrer constant (0.891), X-ray wavelength, FWHM, and Bragg angle, respectively. The CS was calculated to be 16.4 nm. The electron diffraction image shows that the lattice plane appears to be ring-shaped in Fig. S1(a). The XRD results show the peak

SG1042-2
near the (104) plane of the hematite phase is measured, and its FWHM suggesting that the material is single-crystalline. The FWHM is 0.14, close to the previous single crystal study. 34) EDS mapping confirmed the presence of Fe and O atoms in the same region, suggesting the presence of iron oxide, as shown in Figs. 2(c)-2(e). The bright-field (BF) image from the TEM image confirmed particle agglomeration [ Fig. S1(b)]. A histogram was created from the BF image to calculate the average particle size, as shown in Fig. S1(c). The average particle size was 158.4 nm. The particle size was larger than the CS. The XRD and TEM results suggest the successful fabrication of α-Fe 2 O 3 pure nanoparticles.
In the case of Ti-Bi-O, the XRD results show TiO 2 and Bi 4 Ti 3 O 12 peaks in Fig. 2(b). The CS values were calculated to be 5.65 nm and 20.3 nm, respectively. The electron diffraction image suggested that the crystal ring patterns were polycrystalline, consistent with the XRD peak analysis results shown in Fig. S1(d). EDS mapping showed the presence of Ti and O atoms in the same region, suggesting the presence of titanium oxide. The Bi region was included in the Ti and O regions, suggesting the presence of Ti-Bi-O compounds, as shown in Figs. 2(f)-2(i). The BF image from the TEM image confirmed particle agglomeration [ Fig. S1 (e)]. The histogram shows that the average particle size is 13.75 nm in Fig. S1(f). The histogram results show different particle sizes of TiO 2 and Bi 4 Ti 3 O 12 . In the TEM image, TiO 2 and Bi 4 Ti 3 O 12 crystals are observed to be mixed. Therefore, the average particle size was 13 nm. The above XRD and TEM results demonstrate the successful synthesis of TiO 2 and Bi 4 Ti 3 O 12 .

Electrical characterization of α-Fe 2 O 3 /Ti-Bi-O composite devices
The electrical properties of the fabricated composite devices were evaluated based on the I-V and V-t characteristics. The I-V curve in Fig. 3(a) shows that the composite device exhibits nonlinear electrical characteristics. Figures S2(a), S2 (b) show the I-V curves for the α-Fe 2 O 3 and Ti-Bi-O devices. The results show that using each material as a composite improves carrier transport and increases the current, consistent with previous results. 27) Figure S2(c) shows the electrical properties of the composite device without distilled water. The result suggests that ionic conduction is dominant in the composite devices because the current is dramatically reduced in the absence of water. Electrical properties of Ti-Bi-O nanoparticles depend on nanoparticle density. The smaller the grain size, the larger the relative crystal density, which leads to an increased resistance due to an increase in grain boundaries. 35) The Lissajous plot and FFT analysis were performed on the output data obtained from the V-t measurements, as shown in Fig. 3(b). A Lissajous plot can be created by combining two oscillations. A Lissajous plot 36) is expressed as follows: where A and B, aθ and bθ, and d are the amplitude, angular frequency, and phase shift, respectively. Here, the Lissajous curve was created using two signals: one is input signal to the material device and other is output signal obtained from the material device. The Lissajous curve in Fig. 3(c) shows a phase shift with an angle of 5.07°between the input and output signals because the curve shape is elliptical. The FFT analysis can be used to confirm the frequency characteristics of the V-t curves. The FFT analysis results showed higher harmonic generation from integer multiples of the 11 Hz input frequency, as shown in Fig. 3(d). The I-V and V-t curves confirm that the composite device has the following three properties: nonlinear current-voltage characteristics, phase shift, and higher harmonics. These properties are necessary for a reservoir material to exhibit high performance. 19,24,25) From these results, we can conclude that the composite device has the potential to be used as a high-performance reservoir material.

Performance evaluation of the device as a reservoir material
The waveform generation task, which is a typical benchmark, was performed to verify the RC performance of the composite device. Figures 4(b)-4(e) show the learning results with a triangle, sin2ωt, square, and sawtooth waves as the target, respectively. The prediction accuracy of the waveform generation was calculated using the following equation: where x , ix , and y i are the output data, the mean of the outputs, and the target data, respectively. The prediction accuracy exceeded 87% indicating that the composite device had a high RC performance. Figure 4(f) shows the relationship between the prediction accuracy and voltage white noise in the waveform generation task. With the change in the ratio of the voltage white noise in the input, the prediction accuracy of square and sawtooth were improved when a small amount (<3%) of voltage white noise was added to the input signal. The result indicates that direct noise injection into the input signals contributed to improving the waveform prediction accuracy. The software simulation results showed that the normalized root-mean-square error (NRMSE) of the prediction reduced significantly when a small amount of noise (0.25%) was injected into the input signal. Our experimental results showed a trend similar to the results of a software experiment 26) Since prediction accuracy in this work represents the generalization ability, we demonstrated that adding a small noise in RC system prevents overlearning and improves generalization ability as well as software simulation. The phenomenon was demonstrated for the first time in an experimental system in which noise injection prevented overlearning and improved the waveform prediction accuracy.

Conclusions
We successfully synthesized α-Fe 2 O 3 and Ti-Bi-O powders using the sol-gel method. Material identification was performed by XRD, TEM, and EDS mapping. The I-V and V-t characteristics showed that the composite devices exhibited electrical nonlinearity, phase shift, and higher harmonics. From these results, the composite material devices satisfied the properties required to serve as high-performance reservoir materials. A waveform generation task was performed to evaluate the RC performance of our device; the results showed a high prediction accuracy of over 87%. Although machine learning also empirically showed an improvement in accuracy when noise was applied to the signal, 26) the results of this study experimentally demonstrated that the RC performance could be improved by injecting a small amount of noise. This is analogous to the phenomenon observed in the biological brain, which is subjected to potential fluctuations due to stimuli from the external world. The successful incorporation of noise injection into the existing information process can serve as a basis for developing new information processing systems.

Acknowledgments
This work was technologically supported by Yamaguchi University and Kitakyushu Semiconductor Center under the "Advanced Research Infrastructure for Materials and