Inverted input method for computing performance enhancement of the ion-gating reservoir

Physical reservoir computing (PRC) is useful for edge computing, although the challenge is to improve computational performance. In this study, we developed an inverted input method, the inverted input is additionally applied to a physical reservoir together with the original input, to improve the performance of the ion-gating reservoir. The error in the second-order nonlinear equation task was 7.3 × 10−5, the lowest error in reported PRC to date. Improvement of high dimensionality by the method was confirmed to be the origin of the performance enhancement. This inverted input method is versatile enough to enhance the performance of any other PRC.

the inverting input, we quantitatively evaluated high dimensionality and STM, which are requirements for PRC. 1,25) schematic diagram of the IGR is shown in Fig. 1(a).This IGR is based on an electric double-layer (EDL) transistor 22,23,26,27) fabricated with a Li-Si-Zr-O (LSZO) lithium ion-conducting amorphous electrolyte thin film, which is deposited by pulsed laser deposition, and a hydrogen-terminated diamond homoepitaxial (100) substrate, which is deposited by microwave-plasma chemical vapor deposition.][28][29][30] As shown in Fig. 1(a), the drain current (I d ) is largely modulated by the V g application.Please refer to the Supplementary Material for details of the device's fabrication and electrical measurements.
A general scheme of PRC is shown in Fig. 1(b).In the input layer, time series data u(k) are inputted, where k is the time step.Then, in the reservoir layer, input time series data is transformed nonlinearly to high dimensional space as reservoir states X i (k) at given node i (i = 1, 2, …, N). 1,31) For the output layer, the readout weight w i connecting X i (k) and output y i (k) is trained by linear regression to obtain the desired output.The reservoir output y(k) is described as a linear combination of X i (k) and the w i , as follows, where N and b are the number of the reservoir state and bias, respectively.
As shown in Fig. 1(c), we prepared the input u(k) as a random value (0.0 ≦ u(k) ≦ 0.5) and inverted input u′(k) (u′ (k) = 0.5-u(k)).By combining both input u(k) and u′(k), the inverted input method enhances high dimensionality in PRC.The principle is explained below.The function of the reservoir in reservoir computing is to map the input to a high-dimensional feature space. 1,31)The reservoir's mapping function f i for the input u i and reservoir state X i is shown in Fig. 1(a).In a simulation reservoir such as an echo state network, nonlinear functions such as tanh are employed identically for all nodes (i.e., f i = f j≠i ). 25,31)On the other hand, in PRC, a variety of nonlinear functions that originate from nonlinear dynamics that are inherent in the physical system are employed as mapping functions for each node (i.e., for each measurement terminal or sampling time/virtual node). 1, Thes mapping functions change dynamically according to the driving conditions (e.g., input intensity and time scale) of the physical system (i.e., f i ≠ f j≠i ).In this IGR, the I d -V g and gate current (I g )-V g characteristic were utilized as mapping functions for the nonlinear transformation to high dimensional space.The relationship between the reservoir state and the mapping function is defined by the following equation.
As shown in Fig. 1(c), the relation between u(k) and u′(k) is linear.Yet, the nonlinearity of the f i causes them to be different outputs from each other (i.e., two outputs are not in a linear relationship), allowing them to map to a higher dimensional feature space than is possible with a single u(k).This is the mechanism of higher performance with inverted inputs, which is a particularly effective scheme for physical systems with diverse and dynamic mapping functions, compared to simulation reservoirs with uniform mapping functions.In addition, the effective number of nodes in the reservoir increases with the improvement in high dimensionality; therefore, STM can also be improved. 1,25)][18][19][20][21][22] The target output y t (k) for the task is determined from the 024501-2 © 2024 The Author(s).Published on behalf of The Japan Society of Applied Physics by IOP Publishing Ltd equation below.
For the input data, input u(k) and inverted input u′(k) were prepared.As referenced in Fig. S2, u(k) and u′(k) were converted to a voltage pulse stream with a pulse period T (10-80 ms) and duty rate D (40%-80%).The constant drain voltage V d was set to 0.1 V. Voltage pulse stream was applied to the common gate and the output current was measured from 9 terminals of drains (i.e., 9 drain current) which have different channel lengths and 1 terminal of the gate (i.e., 1 gate current), in a total of 10 physical nodes.Under conditions without u‵(k), 20 virtual nodes were extracted with even spacing from each current response from 10 terminals.Therefore, 200 virtual nodes (10 physical nodes × 20 virtual nodes) were utilized in total for the task.Under condition with u′(k), 10 virtual nodes were extracted with even spacing from each current response from 10 terminals.Thus, 200 virtual nodes (2 ways of input × 10 physical nodes × 10 virtual nodes) were utilized similarly in total for the task.By utilizing both physical and virtual nodes, unique diverse reservoir states can be extracted, which enables the higher dimensionality necessary for high performance.
To evaluate the performance of the task, we calculated the normalized mean squared error (NMSE) of the y t (k) from Eq. ( 3) and the prediction output y(k) trained by Eq. ( 1), as follows.
where L(=500) is a data length.Please refer to the Supplementary Material for details of T and D dependence on the performance.By combining u(k) and u′(k), it was possible to predict more precisely which achieved the NMSE of 7.3 × 10 −5 , whereas it was 1.4 × 10 −4 under conditions without u′(k).[18][19][20][21][22] In order to clarify the origin of such performance enhancement of the IGR due to the introduction of the inverted input method, we quantitatively analyzed the high dimensionality and STM of the system under the two conditions (i.e., input without u′(k) and with u′(k)).As discussed above, introducing an inverted input provides diverse reservoir states due to the nonlinearity and asymmetry of the mapping function.Figures 3(a) and 3(b) show all reservoir states used for the task under conditions without u′(k) and with u′(k) respectively.The system diversity is clearly enhanced by adding the reservoir state X′(k) for the inverted input u′(k) as shown in Fig. 3(b) to the reservoir state X(k) for the original input u(k) as shown in Fig. 3(a).In particular, X(k) and X′(k) are not inversely symmetric due to the nonlinearities of the mapping functions, despite u(k) and u′(k) being inversely symmetric.Therefore, the inverted input surely improved the high dimensionality of this system.The high dimensionality in PRC is achieved by the presence of multiple nodes that behave independently, which can be characterized by a correlation coefficient r between each node X i and X j≠i defined by following equation 20 where X i ̅ is the average value of X i. Please refer to Supplementary Material for the details of the correlation coefficient analysis.Figure 3(c) shows the correlation coefficients between each node under the conditions with u(k) and u′(k) combined.X 1 ∼X 100 are nodes under the condition with u(k) and X′ 1 ∼X′ 100 are nodes under the condition with u′(k).In addition, X 90 ∼X 100 and X′ 90 ∼X′ 100 were extracted from the I g response, and all other reservoir states were extracted from the I d response, resulting in lower correlation coefficients between reservoir states extracted from I g and I d .Overall, the correlation coefficients between X 1 ∼X 100 and X′ 1 ∼X′ 100 were low compared to pairs of both X 1 ∼X 100 and both X 1 ∼X 100 , which suggests that X(k) from u(k) and X′(k) from u′(k) are not linear relationships due to the nonlinear transformation by mapping function of the IGR.Therefore, the inverted input method effectively boosts PRC's high-dimensionality, which maps the input data to high-dimensional feature space based on the nonlinear dynamics inherent in the physical system. 20,21)High dimensionality is also dependent on the diversity of each node and the number of nodes. 1) To evaluate the overall high dimensionality of the system, the sum of 1-|r| is rather suitable for quantification. 20,21)Figure 3(d) shows the relationship between NMSE of the second-order nonlinear equation task under all T and D conditions and the sum of 1-|r| under both conditions without u′(k) (red dot) and with u′(k) (blue dot).The sums of 1-|r| tended to be larger with u′(k) than without u′(k), meaning that the overall high dimensionality is enhanced with u′(k).More importantly, there appears to be a clear trend where NMSE is low when the sum of 1-|r| is high.Therefore, it is evidenced that utilizing the inverted input method improves the high dimensionality, leading to high performance.
Moreover, improvement of the high dimensionality can further increase STM, which can be evaluated with a delay task in which the reservoir reconstructs historical time series data. 1) The input of this task is the same random input as used in the second-order nonlinear equation task, and the u(k-τ) before the delay time τ was reconstructed by a linear combination of X(k) and w obtained from the current response.The difference between the target waveform u(k-τ) and the reconstructed waveform y(k) was evaluated by the following equation of determination coefficient r 2 where Cov() and Var() are covariance and variance respectively.Figure 3(e) shows the forgetting curve, the τ dependence of r 2 .The ability to reconstruct past data, represented by r 2 , decreased with increasing τ.Memory capacities (MCs) were calculated by integration of the forgetting curves described in the following equation MCs were calculated as 4.34 without u′(k) and 4.71 with u′(k).The theoretical limit of the MC is the same as the reservoir size (the number of nodes), but generally, in physical systems, the MC is much lower than the number of nodes. 25)That is because, in a physical system, there are many nodes that behave similarly, which causes a lower effective reservoir size than a number of nodes. 1) Thus, it is essential to improve high dimensionality, which corresponds to an effective reservoir size to increase the MC.To evaluate overall STM, we analyzed the relationship between the MC and the sum of 1-|r|, as shown in Fig. 3(f).Under the condition with u′(k), both the sum of 1-|r| and the MC tend to be large.Conversely, under the condition without u′(k), the sum of 1-|r| and the MC are small.Therefore, it is suggested that the MC increased due to the improvement in high dimensionality despite using the same number of nodes.Introducing the inverted input significantly enhances the performance of this IGR due to the improvement in high dimensionality and STM.
In conclusion, the inverted input method was applied to EDL-based IGR for PRC performance improvement.The NMSE in a second-order nonlinear equation task was  024501-4 © 2024 The Author(s).Published on behalf of The Japan Society of Applied Physics by IOP Publishing Ltd significantly reduced from 1.4 × 10 −4 to 7.3 × 10 −5 , confirming the effectiveness of the additional inverted input and achieving the best performance of any physical reservoir reported to date.High dimensionality was evaluated by calculating the correlation coefficient r, r between each node notably decreased by applying the additional inverted input.Higher dimensionality due to additional inverted input was the main origin of performance improvement of the reservoir by analyzing the relationship between NMSE and the sum of 1-|r|.Also, the MC increased from 4.34 to 4.71 by applying the inverted input method.The MC increase is accompanied by a decrease in r.This inverted input method is versatile enough to be applied to any physical reservoir and may improve performance.Compared to masking, this method is easy to use and improves computational performance at a low computational cost without hyperparameters, because it can generate diverse reservoir states with a simple process and avoid increasing the input data length.

Fig. 1 .
Fig. 1.(a) Schematic illustration of electric double layer effect-based IGR.(b) General scheme of PRC.(c) Basic role of inverted input in PRC.
Figures 2(a) and 2(b) show y t (k) and y(k) of the second order nonlinear equation task without u′(k) and with u′(k) when the NMSE was at a minimum under the best conditions of T and D (i.e., T = 70 ms and D = 70%), which suggest the long T and large D enhance at least one of the three properties required for PRC, leading to precise prediction performance in the task.

Fig. 2 . 3 ©
Fig. 2. The target and prediction waveforms under conditions without u′(k) (a) and with u′(k) (b) in pulse period of 70 ms and duty rate of 70%.

Fig. 3 .
Fig. 3. (a) All reservoir states X under conditions without u′(k) (a) and with u′(k) (b), (c) The heatmap of r for 200 nodes consists of u(k) and u′(k) at T = 70 ms and D = 70%.X 1 ∼ X 100 corresponds to condition without u′(k).X′ 1 ∼ X′ 100 corresponds to condition with u′(k) (d) The relation between NMSE of second order nonlinear equation task and SUM of 1-|r| (e) The forgetting curve under both conditions without u′(k) and with u′(k) at T = 70 ms and D = 70% (f) The relation between MC and SUM of 1-|r|.