A sensor system integrating sensing and intelligence based on MEMS reservoir computing

Reservoir computing (RC) is a bio-inspired neural network structure which is easy to be implemented in hardware. It has been constructed in a great many fields such as memristor, electrochemical reaction, among which MEMS is the closest to integrate sensing and computing. We propose a novel sensor system of MEMS RC based on stiffness modulation, that natural signal directly affects the system stiffness as the input. Under this paradigm, information can be processed locally without data collection and pre-processing. We inherited the nonlinearity tuning principle and optimized the post-processing algorithm by creating a digital mask operator. In this way, the system can deal with classification tasks as well as forecasting tasks. We integrated MEMS, IC and FPGA with a small volume and low power consumption, so complicated setup for data discretization and transduction in traditional MEMS RC is eliminated. The system can process word classification and chaos forecasting with high accuracy, which preliminarily proves the integrated RC architecture.


Introduction
With the booming of the Internet of Things (IoT) in our information-driven society, massive raw data generated by thousands sensor nodes are consuming more and more transmission capacity, which calls for system ability to process information efficiently with low power consumption.Therefore, localization of signal processing is highly desired that sensing and intelligence need to be integrated, achieving edge computing.Reservoir computing (RC) is inspired by recurrent neural network (RNN), which is best-in-class to deal with time-series data and well-suited to be implemented physically because the reservoir can be realized by nonlinear devices.Most physical RC staid in the verification stage which proved the feasibility of the devices acting as a reservoir for computing, but lost sight of sensing.In addition, some of those RC systems often take a huge volume and complicated setup, such as optical RC [1].
MEMS devices are primarily designed as sensors, so MEMS RC is closer to the intention of edge computing.Previous researches only verified its feasibility as well [2], but failed to provide an integrated sensor system.Information was injected as a dataset and then modulated to the drive voltage by amplitude modulation.The sensing characteristic of MEMS was laid aside but only computing characteristic was considered.Moreover, the three layers: input layer, reservoir layer, and output layer, were always set up separately at hardware level, resulting in a big volume and a discrete system.Some improvements have been proposed, such as using bias time multiplexing [3], and using hybrid nonlinearity (HNL) [4], but they did not obtain a system fully suitable for edge computing dealing with various scenes.
This paper provides an integrated MEMS RC paradigm.The system makes use of stiffness modulation, that the input is sensed and injected as natural signal to disturb the system stiffness, then the reservoir states is collected by IC and sent to FPGA for computing.We eliminated the data discretization so that ADC and DAC between the first two layers, which greatly reduce the system complexity.The novel architecture is applied to a MEMS resonant accelerometer.In this way, information from natural signals, such as acceleration or temperature, can be directly processed by the integrated system.We also optimized the algorithm in the third layer, in order to overcome the weakness of our original in forecasting tasks [4], and retained the good performance in classification tasks.It is hoped that this research will contribute to a new generation of intelligent MEMS sensors and its sensor system.

MEMS RC with stiffness modulation
In our MEMS RC, the reservoir states are obtained by nonlinear resonator which acts as the reservoir layer, where a comprehensive duffing function is given by: where meff, x, Q, ω0, k1 and k3 are effective lumped mass, displacement of silicon beam, quality factor, natural resonant frequency, linear mechanical stiffness, and nonlinear mechanical stiffness, respectively, and C0, d0, Vdc, Vac and Ω are initial capacitance, initial gap of the parallel plate electrode, bias voltage, drive voltage and drive frequency, respectively.In amplitude modulated RC, input signal is modulated to Vac with delay and mask procedure which increases system complexity as well as error rate.We proposed a stiffness modulated RC which injects the input into k1 by natural signal.The stiffness disturbance reflects information and directly affects the system response x.In this way, the drive force is fixed and no transduction exists, so the system is simpler but can simultaneously sense signals and compute tasks.This new kind of modulation avoids energy conversion from electricity to force which is often need in amplitude modulated RC because a dataset has to be collected in advance and then modulated into electrostatic force.
When a nonlinear resonator is operating, HNL occurs which contains two kinds of nonlinearities: duffing nonlinearity (DNL) and transient nonlinearity (TNL).The former brings a strong nonlinear mapping, while the latter reflects fading characteristic for the sake of MC. Figure 1 shows the two kinds of nonlinear response.In compare, stiffness modulation mode has a more unique nonlinearity that bidirectional data can be map to high-dimensional space more effectively.In this work, we controlled input acceleration signal to keep stiffness within a bound around bifurcation point A in Figure 1(B), and optimized with RC algorithm by nonlinearity tuning [5].These curves are obtained by measuring the MEMS device described in the next section.

Physical RC architecture
We applied this new architecture to a differential MEMS resonant accelerometer [6], and provided an integrated sensor system including MEMS RC, IC and FPGA.As shown in Figure 2  In order to get forecasting ability which needs long-term time dependencies, we did not omit the delay-loop and mask operation but implemented them digitally, getting a better performance because of less noise.As shown in Figure 3, the input layer and reservoir layer are combined in the green analog domain.Our reservoir states take advantage of HNL which brings a self-masking effect.By nonlinearity tuning with the driving voltage and the time interval θ, there is a corresponding coupling between adjacent data points, thus it provides a rich response.This nonlinear transformation with only HNL is suitable for classification tasks, but is unable to deal with complex forecasting tasks.So, in the output layer, we proposed a digital mask operator colligated mask, delay and nonlinear node (the blue NL block), which is equivalent to the corresponding part of conventional delay-based RC algorithm, but totally implemented in FPGA after the response of resonator is sampled by ADC.Other detailed operation principle are similar to our previous work [7].

TI-46 task
We first tested our integrated sensor system via a speech word classification task chosen from the TI-46 dataset [8].The dataset consists of ten spoken digits (0-9) pronounced ten times by five different female speakers.We took 450 words for training and 50 words for testing and applied a ten-fold cross-validation.We trained ten classifiers for different digits, with a target value of 1 when the word corresponds to the sought digit, and 0 otherwise.A winner-takes-all approach was applied to determine the predicted digit, that the output value of each classifier was average in time and we took the largest one.The interested data points were down-sampled at 1/θ.We set Vdc=20V, Vac=2V, fd=349.2kHzfor an optimum operation point as discussed in Figure 1(B), and N=100, α=0.4,j=1, s=3, k=1, and time dependent parameters θ=0.2Td to strengthen data coupling for classification tasks.Td=2Q/ω0 is the decay time of the resonator, where ω0 is its natural frequency.Figure 4 shows the result as a confusion matrix.Our integrated sensor paradigm obtains a 99.8% accuracy which is better than 95.6% in electronic system [9], and is the same as our previous work with a disjointed system [2].

NARMA-10 task
In order to verify our new architecture as well as the optimization method, we tested the well-known NARMA-10 forecasting task [10].As there is no natural signal which can completely reflected data from an artificial chaotic system, an electrostatic force varying with input data was applied to the accelerometer to simulated a varying acceleration series.We generated 1000 points for training and 300 points for testing.We set Vdc=80V for a bigger nonlinearity, and Vac=2V, fd=352kHz.As ten adjacent data points are corelated, we chose N=50, α= ., j=10, s=10, k=1, and θ=4.5Td.We need a θ bigger than several times of Td to reduce data coupling in forecasting tasks.We retained 20% of the total feature points to avoid overfitting.Finally, we swept the τ hown n F 5, getting a smallest normalized mean square error (NMSE) of only 0.0218.Compared with previous works in Table 1, our new architecture shows superiority.
(A), there are two double-ended tune fork resonators both with one end connected to a proof mass by a pair of micro-levers.The top resonator performs the sensing module, while the bottom resonator performs the computing module.The optical microscope image is displayed in Figure2(B).The computing resonator has a length of 500μm, width of 7μm and thickness of 40μm, and the natural resonant frequency is simulated around 350kHz.It is driven by a bias Vdc and a Vac with a frequency fd.The setup of stiffness modulated RC is shown in Figure2(C).Analog signal is directly input into MEMS without pre-processing, and the response is collected by interface circuit.The displacement of resonator is detected as current and then amplified to voltage by a transimpedance amplifier (TIA).A Lowpass filter (LPF) demodulates information which is sampled by an ADC.The system is integrated with FPGA for algorithmic processing of the output.

Figure 2 .
Figure 2. Schematic of hardware.(A) Differential MEMS resonant accelerometer.(B) Optical microscope image.(C) RC setup.In order to get forecasting ability which needs long-term time dependencies, we did not omit the delay-loop and mask operation but implemented them digitally, getting a better performance because of less noise.As shown in Figure3, the input layer and reservoir layer are combined in the green analog domain.Our reservoir states take advantage of HNL which brings a self-masking effect.By nonlinearity tuning with the driving voltage and the time interval θ, there is a corresponding coupling between adjacent data points, thus it provides a rich response.This nonlinear transformation with only HNL is suitable for classification tasks, but is unable to deal with complex forecasting tasks.So, in the output layer, we proposed a digital mask operator colligated mask, delay and nonlinear node (the blue NL block), which is equivalent to the corresponding part of conventional delay-based RC algorithm, but totally implemented in FPGA after the response of resonator is sampled by ADC.Other detailed operation principle are similar to our previous work[7].