Review—Deep Learning Methods for Sensor Based Predictive Maintenance and Future Perspectives for Electrochemical Sensors

IoT can be used to continuously acquire sensor data and interact with the environment.²² This has been demonstrated by Refs. 23–25 and 26. Our research group has demonstrated the use of electrochemical sensors for continuous monitoring of bio-signals in Refs. 23 and 24 respectively. Reference 23 developed and tested a miniaturized wearable fuel cell sensor that can detect isoflurane gas vapors in the therapeutic ranges. The IoT device is interfaced with a Bluetooth radio module and incorporated into a life support system for casualty evacuation with autonomous UAV emergency medical operations. Reference 24 developed a platform for monitoring wound healing and management. The miniaturized flexible platform was based on a wearable enzymatic uric acid biosensor interfaced with a wireless potentiostat. These examples demonstrate the use of electrochemical sensors for continuous monitoring.

Reference 25 also developed IoT sensor sheets to help everyday home and commercial gardeners reduce excessive fertilizer application. The nitrate sensing platform functions by using wireless potentiometry to monitor leached nitrate from south Florida garden soils in real-time. Reference 26 also utilized an IoT sensor network approach to smart farming. The real-time sensor network, which consisted of soil moisture sensors, solar radiation sensor, soil temperature, leaf wetness sensors, and a weather station, offered insight on the nexus between the food, energy, and water resources and crop yield for several essential crops.

In addition, oxygen sensors, which are electrochemical in nature,^27,28 have been used in the automotive industry for a long time and are also being used in IoT data acquisition.

The vision of Industry 4.0 involves collecting data from sensors, machines, and tools connected through IoT. The processing of the data can happen either on the device or centrally.

There is also an uptick in the application of artificial intelligence (AI) with sensor data. For example, The researchers in Ref. 29 have achieved water quality monitoring using AI (conclusion and findings). The data from the embedded electrochemical and optic-based sensors can be used for anomaly detection, defect localization, prognostics, forecasting, diagnosis, optimization, and control in industrial applications.¹ Table II provides examples of sensors and their potential use in the industry.

Supervised learning is the most frequently used type of ML. In this type of learning, training samples are accompanied by target variables, which are also called labels. The data in this case can be represented as shown in Eq. 1,³⁹ which represents a set of ordered pairs. x_i represents a vector of independent variables and y_i represents the value of the dependent variable for the ith sample. N represents the number of data samples.

$$\begin{eqnarray}&&D={\{({x}_{i},{y}_{i})\}}_{i=1}^{N}\end{eqnarray} \tag{ 1 }$$

x_i can be numbers, audio or images based on the sensor being used. ML attempts to learn representations of the data that is useful in making predictions. The learning process involves finding the optimum values of the parameters that constitute the representation, called model, in order to minimize the loss function. Equation 2 shows an example loss function, which is Mean Squared Error (MSE). The choice of the function depends on the specific problem being solved.

$$\begin{eqnarray}&&{MSE}=\displaystyle \frac{1}{n}\sum _{i=1}^{n}{({\hat{y}}_{i}-{y}_{i})}^{2}\end{eqnarray} \tag{ 2 }$$

The loss function is an estimate of the overall error in the relationship learned by the model. The error for each training sample is the difference between the actual and predicted values.

ML approaches applied when the target data or annotated training labels are not available is called unsupervised learning. The data in this case can be represented as shown in Eq. 3.³⁹ The representation is similar to Eq. 1, where x_i represents a vector of independent variables for the ith sample, but values of y_i are not available.

$$\begin{eqnarray}&&D={\{{x}_{i}\}}_{i=1}^{N}\end{eqnarray} \tag{ 3 }$$

Clustering, dimensionality reduction and denoising are examples of unsupervised tasks. Since there is no target data, which provides reference values for learning, unsupervised learning is very challenging. It is an active research area.

The processing units in each layer of an Artificial Neural Network (ANN), referred to as nodes, are connected to all the nodes in the next layer and are known as a fully connected layers.^41,42 ANNs are commonly used for function approximation and pattern recognition purposes using supervised learning.⁴³ They are therefore well suited for forecasting and prognosis. ANNs consists of an input layer, an output layer and one or more hidden layers as shown in Fig. 3.

$$\begin{eqnarray}&&{h}_{i}^{m}=\sum _{j=1}^{{N}_{h}^{m-1}}{W}_{i,j}^{m}.{y}_{j}^{m-1}+{b}_{j}^{m},\end{eqnarray} \tag{ 6 }$$

$$\begin{eqnarray}&&{y}_{i}^{m}=A({h}_{i}^{m}).\end{eqnarray} \tag{ 7 }$$

Zoom In Zoom Out Reset image size

Equations 6 and 7 are used to calculate the output of the mth layer of the ANN shown in Fig. 3.⁴⁰ The number of nodes in the mth layer are N_h^m. The weight and bias of the mth layer are given by W_i,j^m and b_j^m. In Eq. 7, the function A represents the activation function.

A network layer that uses the convolution operation is referred to as a convolutional layer. Figure 4 shows a representative image of a CNN applied to two dimensional sensor data. CNNs can also be applied to one dimensional sensor data. There are two parts in the CNN architecture. The first part consists of convolutional and pooling layers, that helps to extract features from the input data and the second part (fully connected layers) learns a representation of the training data to predict the target variables.^11,44

Zoom In Zoom Out Reset image size

The discrete convolution operation is used to learn local patterns in data, either 1D or 2D, using a learnable filter. If the input and filter are denoted by I and K respectively, then the output of the convolution operation C is given by Eq. 8. The network in Fig. 4 uses two convolutional layers. The output of each convolutional layer is referred to as the feature map.^45–47

$$\begin{eqnarray}&&C(i,j)=\sum _{m}\sum _{n}I(m,n)K(i-m,j-n)\end{eqnarray} \tag{ 8 }$$

Pooling operation is a way of down-sampling that decreases the number of features and reduces overfitting. Before being fed into the fully connected layers, the feature map is flattened into a 1D array.

The Recurrent Neural Networks (RNN) is a generalization of neural networks to learn sequences.⁴⁸ RNNs can easily map sequences into a single output or sequential output.^49,50 They are networks with loops that allow information to persist. Figure 5 shows the loop architecture and visualizes the unrolled RNN. X_t represents the input and h_t represents the output at the current state. RNN contains several instances of the same nodes, where the output of one is fed as the input to the successive node. RNNs have the ability to handle sequences of various lengths. PM data consists of time-series and can have input samples of various lengths.

$$\begin{eqnarray}&&{h}_{t}=f({h}_{t-1},{x}_{t};\theta )\end{eqnarray} \tag{ 9 }$$

$$\begin{eqnarray}&&{h}_{t}={g}_{t}({x}_{t},{x}_{t-1}...,{x}_{2},{x}_{1}))\end{eqnarray} \tag{ 10 }$$

Zoom In Zoom Out Reset image size

Equation 9 represents a dynamic system,⁴⁰ which is a system with memory. h_(t) is the output at time t, and depends on the input x_(t) and the previous output at t − 1, which is h_(t−1). The output also depends on θ which represents the learned weights. Equation 10, which is equivalent to Eq. 9, shows how the output at time t depends on all the previous inputs. Infact, Fig. 5 is a visualization of these equations.

Long Short-Term Memory (LSTM) networks are a special type of RNN with a capability of long-term dependencies which would be essential for our dataset. LSTM will help us look back for long periods before prediction. LSTMs will also help us extract and identify features from the sensor data that are essential for prediction. For a thorough discussion on LSTM please refer to the seminal paper that introduced it.⁵¹

An autoencoder is a unique neural network with a goal of learning a representation that closely replicates the inputs as the output. The autoencoder has two parts, the encoder and the decoder as shown in Fig. 6. The input and output layers have the same number of nodes and all the layers are fully connected, but the network introduces a bottleneck in order to force the network to learn only the essential features. The bottleneck is introduced by choosing the number of nodes in the connecting layer, between encoder part and the decoder part, to be less than the number in the input layer. The training process is similar to other neural networks, where the weights and bias of the network are learned while minimizing a loss function.⁵²

Zoom In Zoom Out Reset image size

The encoder learns a representation of the input data and the decoder reconstructs the input from the learned representation. This process of learning and reconstruction is leveraged for various purposes. Denoising and dimensionality reduction are applications that are relevant to PM. Denoising can help in improving the quality of data and hence the quality of the forecasts. Dimensionality reduction can help improve computational efficiency when the data is high dimensional.^53–55

Forecasting and prognosis are usually implemented using supervised learning. For time-series data, the sensor values themselves constitute the target values. A sliding window is used to form sequences and the sensor value immediately next to the window is the target value for each position of the window.⁵⁶

Anomaly detection can be supervised or unsupervised. When annotated historical data is available with both normal and anomalous samples, supervised learning can be applied. In several cases, there is no annotated data available, and unsupervised learning is the only option.

There are several variations of CNN, RNN, and autoencoders that have not been discussed in this paper. For more details refer to Refs. 57 and 58. In addition, there are training techniques such as transfer learning, that are also out of scope for this paper. For more details refer to Refs. 44, 59, 60.

ANNs have been used for PM for several years. For example, Refs. 61, 62 and 63 used ANN for fault diagnosis in motor rolling bearing fault diagnosis and Ref. 63 has demonstrated the use of neural networks for machine condition monitoring. Several researchers have demonstrated the effectiveness of using neural networks for system monitoring and prognosis.

The work referenced in this subsection demonstrates that the specific applications for neural networks in PM were identified and demonstrated in the prior decades. The work done since 2012 has been focused on applying the variants of DL neural networks such as CNN, RNN, etc., which improved the accuracy of the predictions based on the data. A case study presented later in this Paper, shows the improvements in accuracy and precision by using DL methods.

Sensor data from industrial and automotive equipment can be 2D or 1D. The right variant of CNN is chosen as per the available data. References 64–66 used 2D CNNs for fault detection and Ref. 67 used it for fault diagnosis. Reference 45 has used 1D CNN for fault diagnosis and Ref. 68 has used 1D CNN for fault detection.

CNN also has applications in RUL and structural damage detection. Reference 69 used CNN for RUL. Reference 46 has used 1D CNN for structural damage detection using data from vibrational sensors. CNNs are highly suited for images and may be used for the analysis of image sensor data analysis.

RNN based architectures are well suited for sequence learning and hence used extensively in time series forecasting. Prognostics and RUL are direct applications of predicting future values of time-series sensor data. Reference 70 used LSTM for fault diagnosis and Ref. 7 1 for RUL estimation. Reference 71 used LSTM for forecasting and prognostics. Fault diagnosis is achieved by predicting future sensor values and comparing them with actual values. If the deviation is very high, it can be an indicator of an underlying problem or a direct fault. Table IV presents the brief summary of DL methods applied to PM using sensor data.

Reference 55 has demonstrated the effectiveness of denoising autoencoders in 2008. Post-2012, with more focus on DL based methods, several other researchers applied autoencoders for fault detection, fault classification, and denoising. References 77–79 used autoencoders to pre-train the neural network and then fine-tune it using the backpropagation algorithm. The underlying approach is to first use unsupervised learning to learn a representation of the data and then fine-tune it using supervised learning. References 33, 73, 80 also used autoencoders for fault diagnosis. References 74, 75 used a similar approach and applied autoencoders for induction motor faults classification.

Autoencoders are highly effective in anomaly detection. References 76 and 36 have successfully used autoencoders for anomaly detection. This approach includes learning a representation of the data and reconstructing the data. An exceptionally high error in reconstruction is an indicator of an outlier or an anomaly. Reference 56 used a semi-supervised approach for failure prediction. Among all the DL architectures, the autoencoders have been used for more PM tasks compared to other variants. LSTM nodes can also be used to build autoencoders. Such autoencoders are very well suited for sequential data. Reference 38 used an LSTM based autoencoder for sensor data forecasting.

The sensors group at Florida International University in collaboration with the sensors and systems group at University of Dayton Research Institute studied PM using ML and tested various approaches on a publicly available dataset.

The dataset used in the project was made available public by Ref. 81. The dataset consists of data from 21 sensors on a turbofan aircraft engine. The data was extracted under simulated engine degradation conditions. The researchers simulated four different sets under various conditions to create several fault modes. They also captured data from several sensors to study the fault evolution. This dataset is part of a NASA's prognostics data repository.

The various sensors used include, temperature sensors, pressure sensors and sensors to measure fan speed, fuel-air ratio, and coolant bleed. Please refer to Table III in Ref. 82 for more detailed descriptions of the sensors.

The focus of the project is to predict a machine's failures well in advance and take appropriate action. In the template provided on the website, three modeling solutions were studied: regression-based model to predict the RUL and Time to Failure (TTF), binary classification model in order to classify if an asset will fail within a particular time frame and a multi-class classification model to predict if an asset will fail in different time frames.

We utilize the same set of training, validation and testing data as provided in Ref. 81 summarized in Table V. The data consists of multi-variate time series with the cycle as the time unit. Each cycle comprises 21 sensor readings which essentially serves as features (or independent variables) for our ML algorithm. Each time-series is assumed as being generated from a different engine of the same type. The last cycle in each time-series is considered as the failure point for the respective engine during the training process. While testing, we utilize the truth data provided along with the testing data in order to estimate our performance. The focus is on solving this problem as a classification task where the labels are determined as described in the database⁸¹ with TTF values at a threshold of 30. The distribution of dataset is as shown in Table V.

Various methods for determining the probability of aircraft engine failing within a cycle were researched and studied. All the data points are converted into a time-series (with a window length of 50). Each time-series is generated from a different engine of the same type. Traditional ML methods are developed solely based on features which do not consider past values for predictions thus making it harder.

Sensor values of 50 cycles prior to the failure of each engine (engines are distinguished by id) are extracted. Note that the sequences are not padded, and the sequences that do not meet window length of 50 are not considered for the study. This type of pre-processing is implemented for both training and testing dataset.^{38,70,71,83–85}

A total of five algorithms are implemented and tested. The traditional ML algorithms used are, Logistic Regression (LR), Support Vector Machines (SVMs) and an Ensemble Model. A fully connected neural network (ANN) is also implemented. And finally, an LSTM network as described in the previous Section is applied to the dataset. The results are summarized in Table VI. For more details about SVM, ensemble models and LR, refer to Refs. 3, 86 and 87 respectively.

The overall accuracy is the ratio of the number of predictions that are right to the total number of predictions. LSTM achieved an accuracy of 99.3%. Also, the Receiver Operating Characteristic (ROC) curve is a graph of the True Positive (TP) rate vs the False Positive (FP) rate. The Area Under the ROC Curve (AUC) is a quantitative measure of the performance of the model. AUC varies from 0 to 1, with 1 being the ideal scenario. The AUC achieved with LSTM is 0.998. From Table VI, it can be noticed that all the ML algorithms that is tested show good performance as per the AUC.⁸⁸

The LSTM model is found to be more effective compared to the other models for this dataset. Even though there is minimal difference in AUC values for all the models, there is a striking difference in terms of the precision score. LSTM is able to detect 268 out of the 307 faults, thereby achieving a high precision score of 87.3%. This demonstrates the superiority of LSTM, when learning from sequential data. Table VI shows the precision scores of all the algorithms.