Switched Fault Diagnosis Approach for Industrial Processes based on Hidden Markov Model

Traditional fault diagnosis methods based on hidden Markov model (HMM) use a unified method for feature extraction, such as principal component analysis (PCA), kernel principal component analysis (KPCA) and independent component analysis (ICA). However, every method has its own limitations. For example, PCA cannot extract nonlinear relationships among process variables. So it is inappropriate to extract all features of variables by only one method, especially when data characteristics are very complex. This article proposes a switched feature extraction procedure using PCA and KPCA based on nonlinearity measure. By the proposed method, we are able to choose the most suitable feature extraction method, which could improve the accuracy of fault diagnosis. A simulation from the Tennessee Eastman (TE) process demonstrates that the proposed approach is superior to the traditional one based on HMM and could achieve more accurate classification of various process faults.


Introduction
With the growth of complexity of industrial processes, effective monitoring and diagnosis play a significant role in ensuring operational safety of chemical equipment, maintaining product quality, optimizing product profit, and improving environmental sustainability [1].
In process industry, more and more data have been collected with a wide range utilization of sensors. However, it will lead to low efficiency if original data are stored and processed directly. Besides, they do not need all the features of objects in some particular applications. This is because many features cannot reflect the nature of the objects. Sometimes redundant features will even become obstacles to the subsequent processing. Based on the views above, feature extraction is needed to gain essential characteristics of data. It will reduce the interference information like noise and highlight the useful information. Feature extraction is the key step in fault diagnosis. Because of this, the accuracy of the fault diagnosis will be improved by enhancing the effectiveness of feature extraction.
There are a lot of traditional feature extraction methods, such as PCA、KPCA and ICA [2]. PCA can deal with linear relationships among variables effectively, but it cannot extract the nonlinear relationships, which are very common in the process industry [3]. KPCA was proposed to deal with it. It could extract nonlinear relationships among variables. Because of this, it is widely utilized in image recognition, feature extraction, fault detection and other fields [4]. However, there is no effective way to select the kernel function and parameters of KPCA at present. If KPCA was used to deal with process variables with linear relationships, not only the choice of kernel function and its parameters is cumbersome and time-consuming, but also it will greatly decrease accuracy of fault diagnosis with the inappropriate parameters. So it is inappropriate to use KPCA for feature extraction blindly. So far, when KPCA is utilized, nonlinear relationships among variables are a prior condition in most researches. That is to say, it is necessary to analyse the nonlinear relationships among process variables. This article proposes a switched feature extraction method to deal with the proposed problem. As we all know, a better feature extraction would lead to a better classification. So the proposed method has a great research value. A nonlinearity measure proposed by Kruger to analyse the nonlinearity among variables is used to choose the appropriate method for feature extraction [5]. The extracted features will be used as the observation sequences of HMM, which is used as a classifier.
HMM is a valuable pattern recognition tool for sequential data. Its application for industrial processes monitoring is gradually increasing. It is an intelligent diagnosis technology with some unique characteristics. It is a doubly stochastic process which establishes a statistical model of time series by Markov chains and random functions [6]. So HMM has a strong ability to set up model of the dynamic process and classify sequential patterns. Due to the characteristics above, HMM has a good application prospect in the field of fault diagnosis and detection in process industry. So we choose it to clarify the efficiency of the proposed method.
In this article, the proposed method properly takes the characteristics of process data into consideration. By analysing data characteristics, we could overcome the insufficient of the traditional fault diagnosis methods which sometimes extract features of process variables by inappropriate method. By this way, we could improve the accuracy of the fault diagnosis.
The remainder of this article is organized as follows. In section II, PCA, KPCA and HMM are briefly described. In section III, switched fault diagnosis approach for industrial processes based on HMM is developed in detail. In section IV, TE process is used to illustrate the effectiveness of the proposed method. Section V gives the conclusions.

Principal component analysis
In essence, KPCA has realized the nonlinear transformation between data space and feature space. i x and j x are sample points in data space and the mapping function is  .The fundamental of KPCA is inner product transformation.
need to be processed in the feature space F, the process is as follows By eigenvalue decomposition to K as shown We can obtain orthogonal eigenvectors 12 , , , n    and corresponding eigenvalues 12 n       . The dimension reduction can be obtained by mapping , which can be expressed as In the KPCA method, we just need to calculate matrix K to deal with the problem of eigenvalues by ignoring the specific nonlinear mapping relation . In this article, we choose radial basis kernel, that is with rm   , where r is a constant to be selected and m is the dimension of the input space.

Hidden Markov model
Generally speaking, HMM is a valuable pattern recognition tool for sequential data. Its application for pro-cess monitoring is gradually increasing. A continuous HMM (CHMM) is used since the TE data are continuous. It usually can be described by the following parameters:  N: the number of hid den states, the states are expressed as

Switched fault diagnosis approach based on HMM
Industrial processes are usually sophisticated, so the process data have various characteristics, such as non-Gaussianity, dynamic correlations, nonlinear relations and uncertainties in the process measurement, which makes feature extraction significant [3]. Only one feature extraction method would be used in traditional fault diagnosis approach based on HMM, such as PCA [7]. But data have their own characteristics in different operation conditions, which makes it unable to perform well by using a unified feature extraction method. Here, we just consider linear and nonlinear relationships among process variables. This article proposes a switched feature extraction method based on PCA and KPCA. Nonlinearity measure is used to analyse the nonlinear relationships among process variables and choose the appropriate method for feature extraction. The method will be introduced below.

Nonlinear measure
The original data can be divided into several regions based on a priori knowledge of the process, and the accuracy boundaries are determined according to the confidence intervals of the correlation matrix of one region. By using the principle of cross-validation [8], we can deal with the problem of choosing region, which helps us to determine the accuracy boundaries. The relationships among variables are decided by comparing the residuals in each region with their corresponding accuracy boundaries. Specific steps are shown as follows.
Suppose a data matrix nm ZR   having m variables and n samples is divided into l regions, each of which includes / n n l  observations. The correlation matrix for the original data is defined as: The element in matrix is expressed as: Step Description Equation

1
Determine solution of i c for t distribution Define confidence limit where the subscripts U and L represent the upper and lower limit. The more details of calculation are presented in a paper authored by Kruger [5]. The formula above can be simplified as follows The accuracy boundaries can be determined according to the confidence interval of correlation matrix. In terms of frobenius norm, the sum of the standard deviation  is equal to the sum of the discarded eigenvalues of a PCA model.
Because the elements in correlation matrix zz S decide the eigenvalues 1  If all the residual variances fall inside the accuracy boundaries, the process is considered to be linear and then we choose PCA for feature extraction, vice versa.

Fault diagnosis approach
In this article, HMM is integrated with the switched feature extraction method for fault diagnosis. The method mentioned above contains three procedures: nonlinearity measure, feature extraction and fault classification.
Firstly, measure the nonlinearity among original process variables for every mode. The original data are divided into several regions and the accuracy boundaries are determined. If all the residual variances lie inside the accuracy boundaries for each region, PCA is much more appropriate than KPCA for feature extraction and vice versa.
Secondly, the data from every mode are processed respectively by the feature extraction method above selectively. For example, if PCA was selected, the PCs extracted are utilized as observation sequences of HMM.
Thirdly, set up a model library of HMMs by Baum-Welch algorithm, including one HMM for normal operating condition and other HMMs for abnormal conditions. When an unknown fault needs to be classified, it is processed by the suitable method for feature extraction.   ( 1, , ) iN  occurred. A flow diagram of the proposed approach is shown in Fig.1.

Application of examples
TE process was created to provide a realistic industrial process to test the performance of various monitoring and diagnosis approaches [9]. This process consists of five major units: reactor, condenser, separator, compressor and stripper. It has 41 measured variables (22 continuous process variables and 19 composition variables) and 12 manipulated variables. It includes 21 preset faults which are denoted  (7). There are 33 variables that have been selected: 22 process measurements and 11 manipulated variables. The agitation speed is not involved because it is not manipulated in simulation process. In addition, 19 composition measurements are excluded [10]. More details are shown in Table 3.     Table 4. The accurate boundaries and residuals of each region are presented. The bold figures represent the residuals fall outside of the accurate boundaries. Fig.2 is a graphical representation of the case where the accuracy boundaries were obtained from the first region. According to it, the discarded eigenvalues of PCA model for all data sets fall outside the accuracy boundary. It shows the strong nonlinear relationships among variables. So it is necessary to use KPCA for feature extraction. The other modes are also processed in this way. Because of length limitations of paper, the results of the other faults are not presented here.    Baum-Welch method. The results are shown in Fig.3. It is clear that three training processes are quite different from each other, which indicates the modes of operation present distinct characteristics.
There are also three modes in the testing set,

Conclusion
Switched fault diagnosis for industrial processes based on hidden Markov model is proposed. Generally speaking, the swiched fault diagnosis approach can enhance the diagnostic capacity of TE process because the method properly takes the characteristics of process data into consideration. By analysing characteristic of data, we overcome the insufficient of the traditional methods which sometimes extracts data features by inappropriate method. By implementing the simulation on TE, we are able to get the results which exhibit that the proposed diagnosis approach possesses a superior performance when it is compared to the traditional diagnosis based on HMM. This study shows the superiority of the switched fault diagnosis approach for industrial processes based on HMM. However, there are some issues to be further addressed. The effectiveness of the proposed method is demonstrated by using the simulated process data. Future work can implement the proposed method with the real world industrial data. In this article we just consider linearity and nonlinearity by using PCA and KPCA. In the future, we can take more characteristic of data into considerration, such as Gaussianity and non-Gaussianity by enhancing the accuracy of fault diagnosis for different modes.