Quality spectra fluctuation modeling for manufacturing process based on deep transfer learning

It is difficult to characterize and monitor the quality fluctuation caused by multi-correlation parameters in manufacturing process. Motivated by the powerful ability of digital images to characterize process states, this paper presents a quality spectra fluctuation modeling method based on deep transfer learning. Firstly, through the multi-parameter correlation of spectra pixels, the quality spectra is constructed to characterize quality fluctuation. Then, a deep residual network transfer learning model is used to identify the types of quality fluctuation. Finally, the effectiveness analysis of proposed model is demonstrated by the Tennessee Eastman process.


Introduction
In actual manufacturing process, a large number of multi-correlation parameters are produced. They not only reflect the status of the manufacturing process, but also are very important to the quality of products. Quality engineering activities should also be data driven [1]. Therefore, digging out the quality information contained in the multi-correlation parameters of the manufacturing process is an effective way to grasp the quality status of the manufacturing process.
There have been many remarkable research-works on quality fluctuation modeling to ensure the stability of quality fluctuations in the manufacturing process. As an effective tool of quality monitor, statistical process control chart has been widely used in quality control. Aslam et al proposed a twostage process attribute control chart design [2]. In the Literature [3], a new double sampling X control chart for monitoring an abrupt change in the process location is provided. Cao et al proposed a method of statistics design of exponential distributed median control chart based on the high-quality process [4]. In order to improve the monitoring efficiency for abnormal patterns in dynamic process, Liu et al proposed a novel quality abnormal pattern recognition model based on multi-features hybrid with Multi-class Support Vector Machine [5]. In recent decades, data-driven modeling and intelligent monitoring methods are valued by scholars and gradually used in the field of quality control [6][7]. Zhang et al presented a modified canonical variate analysis based on dynamic kernel decomposition approach is proposed for dynamic nonlinear process quality monitoring [8]. Liu et al considered a quality improvement model for the development of complex products based on data mining and mechanism analysis, which could provide quality decision-making for enterprises [9]. Literature [10] provided a new technology named color-spectrum assessment method by combining scientific data visualization and digital image processing together. Du [12].
Although the above research has provided an important reference for quality control research, there still exist several sides of issue which should be worthy of further explored. This paper presents a quality spectra fluctuation modeling method based on deep transfer learning. The key contributions of this work can be summarized in two sides. Firstly, the quality spectra fluctuation model proposed in this paper can clearly characterize quality fluctuations. Secondly, the deep transfer learning framework for monitoring the manufacturing process is developed.
The remainder of this paper is organized as follows. The construction of a quality spectra fluctuation model is described in Section 2. Section 3 proposes the manufacturing process quality spectra identification monitoring framework. In Section 4, one case is used to verify the model. Conclusions and future research direction are provided in Section 5.

Quality fluctuation characterization of multi-correlation parameters
Assuming that there are n multi-correlation parameters reflecting the quality fluctuation of the manufacturing process, m monitoring values are obtained at a certain interval , and a total of m×ndimensional quality data is obtained. Construct an m×n -dimensional matrix   ij m n Q x   called the quality fluctuation matrix. Where ij x is the value of the multi-correlation quality fluctuation parameter j at the i-th time, the i-th row vector represents the distribution of the multi-correlation parameter based on the spatial sequence, and the j-th column vector represents the distribution of the multi-correlation parameter j based on the time series.
In order to avoid large modeling errors caused by differences in data dimensions, normalization is used to standardize the elements in Q . Normalization ensures the equivalence of multiple related parameters, and the generated standardized quality fluctuation matrix is called as  Q .The standardized formula is as follows: where ij j min x and ij j max x are the smallest and largest number in the data sequence.

Quality spectra fluctuation model construction
Using the different gray values of different pixels in the gray space, the quality fluctuation matrix  Q is transformed into a quality spectra * Q . The specific step is described as follows. Firstly, assuming that the total number of colors in the gray space is 255, the gray matrix is generated by multiplying each element in the normalized matrix by 255. This step realizes the mapping of the quality fluctuation matrix to the gray space, which is shown in Formula (2): . Where x min corresponds to a gray value of 0, which is black; max x corresponds to a gray value of 1, which is white. The gray in the middle between black and white is used to indicate the quality fluctuation. Finally, according to the monitoring time sequence, each element of the column vector j in the gray matrix  g Q is mapped according to the gray gradient interval criterion. Because of the difference in the value of the element, the corresponding gray value in the gray gradient interval is also different.
, and the matrix is converted into a gray value.
When the gray values are obtained, the quality spectra can be drawn on the computer. Gray-scale images can directly present the quality information contained in the manufacturing process, so the image can be defined as the quality spectra. When the manufacturing process is in a stable state, the multi-correlation parameters fluctuate normally, and the quality spectra is shown in Figure 1(a). The spectrum is clear, the fluctuation is small, and it remains relatively stable. It can be called normal fluctuations in the quality spectra. When the multi-correlation parameters fluctuate abnormally, as shown in Figure 1(b). The color difference is obvious and the fluctuation is obvious, reflecting the abnormal quality fluctuation in the manufacturing process. Behind the changes in these spectra is actually the fluctuation of the multi-correlation quality parameters.

Deep transfer learning model construction for quality spectra recognition
Deep Residual Network (Resnet) [13], the residual module is adopted to form a deeper network model. The structure of each residual module can be represented by the following formula.
where i x and i y are the input and output vectors of the i-th residual module, i l ω the weight matrix of different layers in the i-th residual module, E W the unit diagonal matrix of the matching dimension. Resnet34 has a total of 34 layers. The main components of the Resnet34 network are: 1 convolutional layer (Conv1_x), 4 residual modules (Conv2_x, Conv3_x, Conv4_x, Conv5_x), 2 pooling layers (Max Pool, Avera Pool), activation function, and fully connected output layer. The structure diagram of Resnet34 is shown in Figure 2. The input of the network is the generated multicorrelation parameter quality spectra, and the output is the category of the quality spectra. The training depth convolution neural network often needs a lot of label data and training time, and requires high-performance computer equipment. When the image data set is small, it is easy to cause problems such as poor network convergence and over-fitting. However, the use of transfer learning can effectively solve these problems [14]. Improve the feature extraction ability of network when facing small sample data. Redesign the classification layer structure of Resnet34. Then use the transfer pre-training weight parameters to realize the transfer learning of the network. Assuming that the network has m layers, the structure design and weight training of the latter m−n layers are carried out for the quality spectra data set. The source domain model and quality spectra identification process can be expressed by the following formula.
   , , , where M is the deep learning model, X and Y are the pictures and picture labels, Ŷ is the model output, ω is the weight parameter, the subscripts I and Q are the source domain ImageNet and target domain quality spectra. The weight parameters of the pre-training model for deep transfer learning are as follows.
It indicates that when the loss function is the smallest in the training process, I ω is selected as the weight parameter to be transferred. Where L is the cross-entropy loss function, and arg is the optimized selector.
In the transfer learning process, the first n-layer parameters where ( ) is all the weight parameters of the quality spectra recognition model.

Quality spectra fluctuation monitoring framework
In order to solve the difficulty of obtaining data from quality fluctuations in the manufacturing process. Therefore, the data enhancement function is used to expand the sample set. Avoid insufficient model training due to the small number of generated spectra. The manufacturing process quality spectra identification monitoring framework of deep transfer learning includes two modules: offline modeling and online monitoring. The quality spectra fluctuation monitoring framework is shown in Figure 3.  Figure 3. Quality spectra fluctuation monitoring framework.

Case study
In order to verify the feasibility of proposed method, the Tennessee-Eastman process (TE) [15] simulation data set is used to construct a quality spectra set. The TE data set includes 22 sets of training data, each with 52×480 dimensions.22 sets of test data, each with 52×960 dimensions, the first 160 dimensions are normal data, and the fault is introduced from the 161st dimension.

Construction and recognition of multi correlation parameter quality spectra
Twelve multi-correlation parameters are selected as monitoring variables. Which represent: Ingredient H, D feed volume, E feed volume, A feed volume, Total feed, Compressor recirculation valve, Drain valve, Diverter irrigation flow, Liquid product flow rate of stripper, Stripper water flow valve, Reactor cooling water flow, Cooling water flow rate of stripper. Suppose a total of 198 sets of 12×160dimensional data are acquired in an 8h sampling period, and the normal data and fault data are selected to construct the quality spectra. 30 normal spectra and 168 abnormal spectra are generated, a total of 198 spectra. Firstly, the generated normal spectra are enhanced by 3 times, a total of 120 spectra. And the quality spectra is divided into 248 sheets in the training set and 40 sheets in the test set. Then, the training set is used for the training of the quality spectra recognition model. Finally, the recognition accuracy obtained by the test set verification is used as the standard for evaluating the pattern recognition model. After repeated iterative experiments, the batch size is set to 16, the learning rate is 0.0001, and the round is 200. the training process of the manufacturing process quality spectra recognition model for deep transfer learning is shown in Figure 4. The recognition accuracy rate gradually increases from the first epoch, and the recognition accuracy rate reaches about 0.9 after iteration.

Comparison and analysis of results
In order to verify the effectiveness of the quality spectra recognition model based on deep transfer learning, Alexnet and Resnet34 were selected for comparison. Control the hyperparameters of different networks during training to be the same as before. The model accuracy rate (Acc) and confusion matrix are used to measure the recognition performance of the model. The accuracy rate represents the ratio of the number of samples correctly classified by the model to the total number of samples.

TP TN TP TN FN FP
where TP predicts actual positive samples as positive samples, FN predicts actual positive samples as negative samples, FP predicts actual negative samples as positive samples, and TN predicts actual negative samples as negative samples. It can be seen from Table 1 that the accuracy of Alexnet is 67.5%, and the accuracy of Resnet34 is 92.5%, indicating that the Resnet34 network has a strong ability to extract quality spectra features. The recognition accuracy of the Resnet34 model of deep transfer learning reached 97.5%. Furthermore, the comparison of the confusion matrix for different recognition methods is shown in Figure 5. It can be seen that the Alexnet misjudged a large number of normal quality spectra as abnormal quality spectra. In the actual manufacturing process, if such an event occurs, it will cause a lot of economic losses. However, the less misjudgment of Resnet34 recognition indicates that the Resnet34 model is better than Alexnet for feature extraction of the quality spectra of multi-correlation parameters. The Resnet34 network model of deep transfer learning has a lower misjudgment positive rate. Using the quality spectra model can make full use of the quality information generated during the manufacturing process. In addition, the quality spectra recognition model of deep transfer learning can realize intelligent monitoring. These verify that the proposed model has a high recognition accuracy for the quality spectra of multi-correlation parameters, and can meet the requirements of intelligent monitoring of quality fluctuation in the actual process.

Conclusions
This paper proposes a quality spectra model with multi-correlation parameters to characterize the quality fluctuation. What's more, the information contained in the quality spectra is digged out and a deep transfer learning model is constructed to achieve qualitative monitoring of quality fluctuations. The case results show that the proposed method can effectively use multi-correlation parameter data to build a quality spectra. Future work will focus on early warning for abnormal quality fluctuation.