A Model Based on CNN-LSTM for the Remaining Life Prediction of Equipment in Aircraft Assembly Pulsation Production Line

The aircraft assembly pulsation production line is an advanced and efficient assembly method widely used in aircraft manufacturing. However, equipment malfunctions would occur and can disrupt production takt, affecting the production efficiency. Therefore, accurately predicting the remaining useful life (RUL) of equipment is very crucial. To meet the requirements of both prediction accuracy and efficiency for RUL model used in aircraft pulsation production line, this work proposed a model combining convolutional neural network (CNN) and long short-term memory (LSTM) for RUL prediction. Optimization was performed for the number of neurons in the proposed model with genetic algorithm (GA) to regulate the prediction accuracy and efficiency. Two public datasets representing typical equipment in the pulsation production line were used to validate the proposed model. The results show that the proposed model outperforms the traditional model with substantial improvements in the fitness function of 19.8%, and 30.2% for the two testing datasets. These findings demonstrate the effectiveness of the proposed model in enhancing the accuracy and efficiency of RUL prediction.


Introduction
The aircraft pulsation production line is an efficient assembly method characterized by clear division of labor, smooth processes, and improved quality and efficiency [1].However, equipment malfunctions can disrupt production takt and affect the production efficiency.Therefore, equipment malfunctions need to be avoided with the best effort.Regular equipment maintenance is necessary, but it cannot take full use of equipment potential and can result in excessive maintenance cost.To ensure cost-effective operation and management of aircraft pulsation production lines, prognostics and health management (PHM) have gained increasing attentions.By continuously monitoring and analyzing the running status parameters of equipment, it is possible to detect equipment degradation states, enabling the estimation of the remaining useful life (RUL) and maximizes the operational availability and security of equipment [2,3].
Model-based and data-driven approaches are two fundamental methods in PHM.The increasing availability of low-cost sensors and big data has led to the growing popularity of data-driven approaches over model-based ones.Notably, advancements in neural networks, such as convolutional neural networks (CNN) [4,5], denoising autoencoders (DAE) [6], long short-term memory (LSTM) [4], gated recurrent units (GRU) [7], and adjacent difference neural networks (ADNN) [8], have been extensively applied to predict the RUL of diverse industrial systems and components.To enhance prediction accuracy, researchers frequently combine multiple algorithms to develop RUL prediction models.For instance, Tong et al. [9] realizes a complex gated recurrent unit (GRU) network that contains monotonicity-Las Vegas wrapper based feature selection and accelerated GRU based RUL prediction.Peng et al. [10] proposed a two-channel LSTM architecture with a momentum smoothing module to improve the accuracy of predicted RUL curves.Furthermore, Peng et al. [11] developed a three-stage prediction model by integrating CNN, LSTM, and support vector classification (SVC), utilizing multisensor vibration signals.Hu et al. [12] combined LSTM with generative adversarial networks (GANs) to predict temperature trends.Additionally, Han et al. [13] introduced a window-dependent long shortterm memory network (TWA-WDLSTM) that incorporates temporal window attention to enhance temporal dependence.
Based on the above literatures, researchers often prioritize improving the accuracy of neural network models over efficiency, which can be problematic for real-time applications or settings with large amounts of data.Taking the aircraft pulsation production line as an example, there are many systems, equipment, and operating state parameters involved, the amount of data is huge.Overly complicated models usually require a large amount of computing resources and long processing time, making them unsuitable for situations that require high computational efficiency [14].Besides, in neural network models, the number of neurons plays a crucial role in the accuracy and efficiency of predictions [15].However, most current research works determine the number of neurons empirically, lacking a systematic method to determine the optimal number of neurons.
Considering the powerful capabilities of CNN in handling large data for regression and classification, and the advantages of LSTM in RUL prediction due to its ability to capture time series correlation, this work proposes a combined CNN-LSTM model for predicting the RUL of equipment in aircraft pulsating production line.To achieve a balance between prediction accuracy and efficiency, the number of neurons in the RUL prediction model is optimized using the genetic algorithm, enhancing the method's generality.To evaluate the model's generality and the effectiveness, two different datasets that align with the operation data characteristics of equipment in aircraft pulsation production line were utilized for validation.

Convolutional Neural Network (CNN)
Convolutional neural networks (CNNs) are widely used in various fields, such as feature recognition, image processing, and signal processing, due to their excellent feature extraction ability [16].The components of CNNs are the convolutional layer and pooling layer.
In the convolutional layer, convolutional kernel extracts feature from the original data and outputs them to corresponding positions in the matrix.As the kernel slides over the data, it captures fundamental features, and with more convolutional layers, the network recognizes more complex features.CNNs require fewer trainable parameters than fully connected neural networks for large input data, reducing computational costs and the risk of overfitting [17].
The pooling layer in CNNs, using maximum or average pooling [18], reduces the input data size and neural network parameters, alleviating overfitting and lowering computational requirements.Additionally, pooling layers can correct for translation, distortion, or rotation of features, minimizing the impact of errors on feature extraction.

Long-Short-Term Memory Network (LSTM)
LSTM, an improved version of RNN, enhances its predictive ability through the inclusion of long-term memory and forgetting functions, making it suitable for data prediction and natural language processing [19,20].Its forward propagation process involves updating the cell state   and hidden state ℎ  based on previous moment's cell state  t−1 , hidden state ℎ t−1 , and input data  t−1 , enabling the model to achieve accurate long-term and short-term memory, thus facilitating accurate predictions.The calculation process in LSTM neurons is shown in figure 1.

Figure 1. The operation process of neurons in LSTM.
Its mathematical expression is as follows [21]: ( ) ( ) ( ) ( ) ( ) where   is the input of the current network; While ℎ −1 is the hidden layer state at the previous time;   is the forgetting gate state at the current moment; It is the input gate state at the current time; ̃ is the current input node status;   is memory unit state;   is the output gate state; ℎ  is the hidden layer state at the current time;  is the corresponding weight value;  is the corresponding partial weight value; ⨀ represents point-wise multiplication; ℎ and  are a nonlinear function.

Genetic Algorithm (GA)
The genetic algorithm, proposed and improved by John Holland et al. [22] in the 1970s, has found wide applications across various fields [23].Inspired by biological evolution, it simulates problem-solving as an evolutionary process.Initially, numerous individuals are initialized within the feasible range, and their fitness functions are evaluated.Through a process of natural selection, individuals with higher fitness are retained, while those with lower fitness are eliminated.Next, parameters are exchanged to simulate gene recombination, and some parameters undergo random changes to mimic gene mutation [24,25].Over several generations, individuals with higher fitness gradually evolve, leading to the refinement of the optimization process.The genetic algorithm demonstrates desirable properties, such as resistance to local optima, the absence of an initial point selection requirement, and strong convergence properties.The weighted calculation method is employed in this work to establish the fitness function, which considers both the accuracy and efficiency of the model.The fitness function proposed in this paper is as following: where (), () and   () represent the fitness function, loss function and the total number of parameters of the model with input  .Using MSE as the loss function of the model.Efficiency is represented by the total number of parameters, where a larger number of parameters indicates lower efficiency.α and β are the weights of accuracy and efficiency, and they can be chosen according to the specific equipment and requirements.
To address the disparity in magnitude between the loss function and the number of parameters, a normalization process is applied.Moreover, during the calculation, the minimum value in the sample is replaced with 0 to prevent issues arising from the denominator approaching zero.Additionally, the number of parameters is squared, reducing the disparity in the rate of change among parameters.

IMS Bearing Data Set
The IMS bearing dataset is released by the Center for Intelligent Maintenance Systems (IMS) at the University of Cincinnati [26].The platform includes four bearings, and each bearing contains two sensors.The data set is sampled every ten minutes, and 20480 sets of sensor data can be obtained for each sampling.The dataset is divided into three parts according to the type of bearing damage.The first part is used for the work, and the damage results in defects in the inner ring of bearing 3 or defects in the rollers of bearing 4.
Due to the large amount of data in this dataset, the requirements for model efficiency are high.In this case, α and β set to 1 and 5.After preliminary testing and analysis, the optimal value for  1 in this case should be between 200 and 800,  2 between 5 and 15, and  3 between 25 and 45.Therefore,  1 is selected as 200, 400, 600, 800,  2 is set to 5, 10, 15, and  3 is set to 25, 35, 45.A total of 36 CNN-LSTM models are built using these values and optimized for  1 ,  2 , and  3 using the method described in Section 3. The optimal values of  1 ,  2 , and  3 are 200, 15 and 25 respectively.
Furthermore,  1 ,  2 , and  3 are empirically chosen as 15, 80, and 60, respectively.The proposed model and the traditional model are then used to predict the complete degradation process in the first part of the IMS bearing dataset, as shown in figure 3  Compared with the traditional model, the number of parameters of the proposed model is reduced by 38.1%, while the loss function is only increased by 8.9%, so its fitness function is increased by 19.8%.

Li-ion Battery Data Set
This data set was released by the Ames Research Center of the NASA [27], including a set of four lithium-ion batteries, which mainly records 12 variables such as current, voltage, temperature, and duration during charging and discharging of lithium-ion batteries.It can simulate the complete life cycle of lithium-ion batteries, so it is widely used in the training and testing of RUL prediction model of lithium-ion batteries.
This work used data from lithium-ion batteries 25, 26, 27, and 28, and divided the training and testing sets.Since SOH is often used as an important indicator for life prediction of Li-ion batteries [28], the SOH after each charge/discharge cycle of the battery is used as the RUL of the battery in this work.The CNN-LSTM model inputs current and voltage data during the discharge process of lithium-ion batteries to predict the SOH after discharge, in order to verify the predictive ability of the model.In addition, due to the large number of equipment using lithium-ion batteries in the production line, this model can also be used for RUL prediction of these equipment, such as AGVs.
Considering the risk of explosion when lithium-ion batteries run out of service, the accuracy of model predictions is highly required, α and β were set to 8 and 0.1 in this case.After preliminary testing and analysis, the optimal value for  1 in this case should be between 5 and 20,  2 between 60 and 100, and  3 between 30 and 60.Therefore,  1 is set to 5, 10, 15, and 20,  2 is set to 60, 80, and 100,  3 is set to 30, 45, and 60.These values were used to build a total of 36 CNN-LSTM models, and the optimal values for  1 ,  2 , and  3 were obtained using the methods described in Section 3, which were 31, 40, and 44, respectively.
To evaluate the effectiveness of the proposed model,  1 ,  2 , and  3 are empirically chosen as 15, 80 and 60, respectively, in the feasible domain.the process of battery degradation is obtained using the proposed model and the traditional model, as shown in figure 5

Conclusion
This work focused on predicting the remaining lifetime of different devices using a CNN-LSTM model.Optimization was performed for the number of neurons in the proposed model with a genetic algorithm (GA).The weight between accuracy and efficiency could be adjusted using a developed fitness function, enabling customization for individual devices.The model's effectiveness was demonstrated through testing on two diverse datasets, indicating its applicability to various production line equipment for RUL prediction.Results from the IMS bearing dataset showed an 8.9% accuracy improvement and a 38.1%

3 .
The Framework of Proposed Model Based on CNN-LSTM This work proposes an improved model that combines CNN and LSTM for RUL prediction as shown in figure 2. The CNN part plays a crucial role in extracting relevant features from the equipment operation data, thereby reducing the complexity of subsequent prediction tasks.The LSTM part effectively utilizes the feature sequence of the data to accurately estimate the RUL of equipment.Meanwhile, the genetic algorithm optimizes the number of neurons in the CNN-LSTM model to improve its efficiency and accuracy.

Figure 2 .
Figure 2. The structure of the RUL prediction model.To begin, some specific values of = [ 1  2  3 ] within the feasible domain D was selected, which is denoted as   ( = 1, ), where  1 and  2 stand for the convolutional kernel size of the first layer and output layer of CNN, and  3 represent the units in the first layer of the LSTM.Different CNN-LSTM models are then built with different   .Each CNN-LSTM model is trained using the training set data.Once the training process is finished, the value of loss function (  ) and the value of total number of parameters   (  ) can be determined, which are then used to construct the continuous function for () and   () by piecewise interpolation.() and   () serve as the basis for establishing the fitness function, which is served as objective function for optimizing the values of  1 ,  2 , and  3 .The optimized values of  1 ,  2 , and  3 would be used to build CNN-LSTM model to predict RUL.In this work, the mean square error (MSE), which is commonly used in machine learning, is used as the loss function.The weighted calculation method is employed in this work to establish the fitness function, which considers both the accuracy and efficiency of the model.The fitness function proposed in this paper is as following: (a) and (b).The blue line represents the predicted RUL and the red line represents the actual RUL.Compared with the traditional model, the RUL prediction results of the proposed model are more in line with the actual.

Figure 3 .
Figure 3. RUL prediction of bearing, (a) traditional model, (b) proposed model.The fitness function, loss function and total number of parameters of the proposed model are compared with the traditional model, as shown in figure 4. The proposed model achieves a loss function value of 0.20 out of a total of 27960 model parameters, resulting in a fitness function value of 12.84.

Figure 4 .
Figure 4. Comparison of results before and after optimization, (a) loss function, (b) number of parameters, (c) fitness function.

Figure 5 .
Figure 5. SOH prediction of Li-ion battery, (a) traditional model, (b) proposed model.In addition, the fitness function, loss function and total number of parameters of the proposed model and the conventional model are compared, as shown in figure6.The proposed model converges with a total number of parameters of 34445, a loss function of 0.068, and a calculated fitness function of 73.2411.Compared to the traditional model, the proposed model only grows the total number of parameters by 6.7%.However, it succeeds in reducing the loss function by 52.2%, which leads to an increase in the fitness function by 30.2%.

Figure 6 .
Figure 6.Comparison of results before and after optimization, (a) loss function, (b) number of parameters, (c) fitness function.