An ensemble approach for enhancing generalization and extendibility of deep learning facilitated by transfer learning: principle and application in curing monitoring

Machine learning (ML) and deep learning (DL) have exhibited significant advantages compared to conventional data analysis methods. However, the limitations of poor generalization and extendibility impede the broader application of these methods beyond specific learning tasks. To address this challenge, this study proposes a transfer learning-based ensemble approach called SMART. This approach incorporates synthetic minority oversampling technique, average reinforced interpolation, series data imaging, and fine-tuning. To validate the effectiveness of SMART, we conduct experiments on curing monitoring of polymeric composites and construct a hybrid dataset with highly heterogeneous features. We compare the performance of SMART with exemplary ML algorithms using conventional evaluation indicators, including Accuracy, Precision, Recall, and F1-score. The experimental results demonstrate that the SMART approach exhibits superior generalization capacity and extendibility, achieving indicator scores above 0.9900 in new scenarios. These findings suggest that the proposed SMART approach has the potential to break through the limitations of conventional ML and DL models, enabling wider applications in the industrial sectors.


Introduction
Machine learning (ML) and deep learning (DL) are broadly adopted techniques in artificial intelligence applications, allowing systems to learn and improve from experience without explicit programming [1].Common ML methods include gradient-boosting decision trees (GBDTs), random forests (RFs), k-nearest neighbors (KNNs), and support vector machines (SVMs).DL approaches, such as convolutional neural networks (CNNs), multi-layer perceptron, and longshort term memory, have also gained popularity.
In recent years, ML and DL approaches have been widely used in various industry sectors, such as structural failure diagnosis [2][3][4][5][6], damage identification and detection [7][8][9][10], remaining strength evaluation [2,[11][12][13][14], and remaining useful life estimation [15][16][17][18][19].However, conventional ML and DL methods have limitations when applied to new cases, as they can only tackle isolated tasks and customized requirements [20], whose lack of adaptability to real-time changes hinders their broader application.To overcome the limitations of the conventional ML and DL methods, researchers have developed the transfer learning (TL) approach [21][22][23][24], which can improve the overall performance and extendibility of models while reducing the cost of developing new models.Regarding the TL method, researchers have made substantial efforts to address various challenging issues over the past several years.Enciu et al [25] investigated the application of TL and entropic indicators in space vehicle survival.Feng et al [26] proposed a highly accurate damage detection method using TL-driven deep neural networks for the long-term operation of hydro-junction infrastructure to ensure structural safety.Lin et al [27] developed a feature extractor that generates damage-sensitive and domain-invariant features, bridging the gap between engineering applications.Wang et al [28] utilized TL to determine damage types caused by the repeated bearing of mechanical operations and natural factors, enhancing the service life of a pressure container.Dunphy et al [29] applied TL and generative adversarial networks for multiclass damage detection within infrastructures.Chamangard et al [30] reported successful TL application in accurately diagnosing invisible damage within complex structures.
However, the present TL-relevant studies encounter problems like domain differences, negative transfer, and limited target domain data, necessitating strategies for effective knowledge transfer and adaptation.Based on the existing TL-boosted methods and studies, we developed a universal ensemble approach called SMART (SM-AR-T), whose major novelty is to enhance the generalization and extendibility to a higher level by using multiple feature representations compared with the conventional ML and DL approaches.The proposed SMART approach combines the synthetic minority oversampling technique (SMOTE), average reinforced interpolation (ARI), series imaging, and fine-tuning (FT).In general, the proposed SMART approach has the advantages of enhancing the classification performance in the new scenarios, mitigating data scarcity issues, and improving robustness by addressing class imbalance, generating synthetic samples, and leveraging knowledge transfer between domains.
To evaluate the effectiveness of the proposed SMART approach, experiments are designed and conducted in our laboratory.Raw data are collected in the above-mentioned experiments via piezoelectric transducers (PZTs) and optical fibers, respectively.The experimental data are then enhanced and transformed into images, which are fed into the customized model based on CNNs alongside the TL strategy.Finally, the performance of the rebuilt model is evaluated using the introduced indicators of Accuracy, Precision, Recall, and F1score, which are then compared with those of exemplary ML methods.
The paper is organized as follows: section 2 presents the fundamentals relevant to the proposed SMART approach.Section 3 introduces the configuration of experiments for the proof-of-concept application, followed by section 4, which conveys the experimental results and discussions.Lastly, section 5 concludes the paper and provides avenues for future work.

Principle and methodology
This study aims to introduce and evaluate the SMART approach developed to enhance the generalization capabilities and ultimate performance of intelligent algorithms in new application scenarios beyond specific learning tasks.The SMART approach includes an FT-based TL strategy for reconstructing the architecture of CNNs, a SMOTE-based manipulation for eliminating imbalances, and an ARI-based method for increasing the sample scale.The schematic of the SMART approach is illustrated in figure 1.

Model rebuilding with FT-based TL strategy
The rebuilt CNNs model used in the SMART algorithm is established by the FT strategy.The FT-based rebuilt architecture includes three convolutional and max-pooling blocks, together with dropout and dense layers, as depicted in figure 2.
It can be seen from figure 2 that the layers in Block #1 and #2 are frozen, whose knowledge is learned from the hybrid dataset in the source domain.By contrast, Block #3 is trainable and can be retrained by the small dataset in the target domain  Massive methods are available for oversampling a dataset up to date.Among these algorithms, the most prevalent is SMOTE algorithm [31], which offers advantages such as data augmentation, prevention of overfitting, enhanced classification performance, preservation of information, wide applicability, and simplicity in addressing class imbalance issues.Further, after being processed with SMOTE, the sample quantity can be increased using a simple but effective method named ARI in this study.The ARI is developed to calculate the mean values between two adjacent data points as their interpolation.The sample scale can then be expanded by repeating the ARI.
Significantly, the proposed SMART approach exhibits a noteworthy capacity to proficiently undertake class prediction tasks, especially within contexts encompassing multiclass classification scenarios.When dealing with multi-class problems in the DL community, one of the commonly used cost functions is cross-entropy loss.The cross-entropy loss is also employed to measure the disparity between the predicted probability distribution and the actual labels in this study, making it a widely used loss function by obtaining the minimized values for the classification tasks.The cost function used in this study can be given by equation (1): where N is the number of samples, indicating the total number of samples in the training set.C is the number of classes representing the number of categories in the classification problem.y ij is the jth class label, and if the sample belongs to class j, then y ij = 1, otherwise it is 0. p ij is the predicted probability by the defined model that the ith sample belongs to class j.

Series data-to-image transformation
To meet the input requirements of CNNs in two dimensions, the numeric data collected with piezoelectric wafers and optical fiber must be transformed into the images of recurrence plot (RP) [32], Markov transition field (MTF) [33], and Gramian angular summation field (GASF) [33] after being processed by the SMOTE and ARI.
RP is an image representing the distance between trajectories extracted from the original series data, which is a primary method for analyzing the periodicity, chaos, and nonstationarity of series data.Supposing that a group of series data can be defined as a sequence of vectors or scalars changing with time, as shown in equation ( 2): where The RP can be generated if the first states in the phase space trajectory of the series sequence are calculated in terms of equation (3): ( Then, the graphs of RP can be generated using the following equation ( 4): where RP denotes the formed recurrence images, and D is the distance function that can be calculated using Euclidean distance [34].
Concerning the MTF method, if the above series data can be divided into Q quantile bins, it generates a matrix M with the dimensions of Q × Q.The quantile bins containing the data at steps i and j are q i and q j .The element w ij in the matrix M is the transition probability of q i → q j .The MTF can be defined as the following matrix shown in equation (5).With this equation, the series data can be encoded as images, With respect to the GASF method, this method adopts the polar coordinate to map the numeric data into a polar plane.Before generating the GASF, the given series data are usually rescaled into an interval [−1, 1] by equation ( 6 The rescaled data can be further expressed in polar coordinates using equation (7): Then the GASF can be obtained by considering the following equation ( 8): By employing the above functions, the initial series data can be effectively transformed into the RP, MTF, and GASF images in this study.Converting numeric data into images provides the advantages of visual representation, contextual information, feature extraction, data compression, and interactive visualization, which outperforms the numeric data for the proposed SMART approach.

General description of the problem
The curing process plays a critical role in composite manufacturing and significantly impacts the final product quality.Therefore, investigations into curing monitoring have been of great significance over the past few decades.The electromechanical impedance (EMI) and optical fiber Bragg grating (FBG) measurements have shown advantages in high sensitivity, ease of data collection, and effectiveness in hotspot sensing.Given their benefits, these measurement techniques are also options to be considered for curing monitoring in this study.
Initially, raw data is independently collected using EMI and FBG on two different composite structures.Corresponding datasets are then established, processed, and fed into the SMART approach, as illustrated in figure 2.

Datasets construction for the SMART approach
3.2.1.Source domain dataset construction.The initial dataset of the source domain collects EMI data from all specimens.The experiments used three co-cured structures incorporating carbon fiber reinforced polymers (CFRP) and aluminum plates with varying specifications.The 3D explosive view of these structures is presented in figure 3. The waterfall graphs of the original EMI data are depicted in figure 4.
Figure 3 shows the schematic of specimens utilized in the experiments: a square PZT, a CFRP laminate, and an aluminum plate.The discrepancies among specimens are attributed to differences in heating time, ply sequence, and thickness, whereas the aluminum plates employed in the experiments have identical specifications.The specifications of the specimens utilized in the experiments are outlined in table 1.
Figure 4 displays 3D waterfall graphs with three coordinate axes representing the frequency range, heating time, and resistance magnitude, respectively.Notably, distinct peaks within the selected frequency scope can be utilized to characterize the curing progress during prolonged heating.
To investigate the varying regularity of the co-cured specimens, the root mean square deviation (RMSD) indicator is introduced and used for characterizing the curing condition based on the raw sensor data via EMI and FBG methods with equation ( 9): where Re (Z i ) denotes the real part of EMI signatures at the frequency of f i , and Re (Z 0 i ) represents the counterpart in the selected state as reference.
The RMSD values are diagramed in figure 5, where the inflection points (IPs) can be observed from experimental curves marked with colored vertical and horizontal lines.The overall trend of curves rises with the heating time and then approaches a constant.Moreover, the RMSD curves provide a   basis for categorizing the original data into designated groups.
Here, the original data are classified into five groups in terms of the RMSD values from 0 to 1 with an interval of 0.2.SMOTE and ARI are employed to process the classified dataset to increase the sample scale.Table 2 lists the scales of the dataset in the source domain.The augmented samples are separately transformed into RP, MTF, and GASF images.Table 3 presents the final architecture of the classified dataset in the source domain.This hybrid dataset can be utilized for model training.In this case, the 'hybrid' can be viewed as   the dataset incorporating blended and heterogeneous features from the varied structures.

Target domain dataset construction.
The data in the target domain are collected by implanting an optical fiber.
The specification of the specimen can be described as that the CFRP is made of T300 unidirectional prepreg, with dimensions of 400 mm in length, 400 mm in width, and 2 mm in thickness.The CFRP laminate is constructed of ten plies and cured at 120 • C. The waterfall graphs of the sensor data are shown in figure 6, and the distribution of the embedded optical fiber with a 3D explosive view is illustrated in figure 7.
Similarly, the initial dataset in the target domain is also processed by the SMOTE and ARI methods, whose sample quantity is enlarged from 63 to 497.The augmented dataset is split into a training set with 397 samples, and the remaining 100 samples are used for testing.Moreover, after applying the SMOTE and ARI methods to augment the initial dataset in the target domain, to further enhance the feature representations of the dataset in the target domain, these augmented samples in the form of numeric data underwent a conversion process to RP, MTF, and GASF images.
By incorporating the above-converted image types, the dataset possesses hybrid feature representations that capture different aspects of the target domain.The availability of this hybrid dataset opens up new possibilities for model retraining and testing in new scenarios.The target dataset construction mentioned in this section ensures a more comprehensive and robust data analysis, enabling the model to effectively adapt to the complexities and variations inherent in the practical applications of the industrial sectors, different from the previous scenarios, which is the mentioned source domain in this study.
Furthermore, to ensure the reliability and representativeness of the measured raw data obtained from the piezoelectric and optical sensors, we validate its quality via meticulously experimental design.The curated datasets specifically include data highly relevant to the problem addressed in this study, encompassing a diverse range of representative samples essential for practical model training.This process involves carefully selecting and extracting the most pertinent features from the raw data and transforming them into a suitable format that facilitates meaningful analysis and interpretation.

Training base model with the dataset of the source domain
To achieve the ultimate performance of the base model using the dataset from the source domain, the base model is optimized with the grid search and cross validation algorithm.For the customized base model, the main hyperparameter items and their corresponding values adopted for model training are enumerated in table 4.    Figure 8 indicates that both accuracy curves initially show an upward trend, improving accuracy over training epochs.However, after reaching a turning point at ten epochs, both curves stabilize, suggesting that further iterations may not significantly enhance accuracy.Similarly, the two loss curves consistently decrease during training, indicating that the model effectively reduces errors and converges toward optimal performance.After the turning point at ten epochs, both loss curves level off, indicating that additional epochs might have a limited impact on further reducing the loss.Besides, in figure 8, both accuracy curves exceed an impressive threshold of 0.9900, implying that the model achieves high accuracy in both the training and validation stages.The magnitude of loss curves is below 0.02, suggesting that the model effectively minimizes errors and maintains high precision.
To provide a more straightforward representation of the classification accuracy of the hybrid dataset, a confusion matrix (CM) has been plotted in figure 9.The presented CM provides an insightful depiction of the model performance.The horizontal axis is dedicated to the true labels by employing a graphical representation, while the vertical axis corresponds to the predicted labels.The term DoC on the vertical and horizontal axes means the degree of curing.In figure 9(a), the numbers displayed indicate the sample quantity that has been successfully quantified.Meanwhile, the numbers in figure 9(b) represent the ratio of accurately quantified samples out of the total samples in each class.
To be more specific, across the main diagonal of the CM in figure 9(b), which signifies instances where the predicted labels align with the true labels, accuracy metrics are discerned for each class.Specifically, the accuracy rates for C1, C2, C3, C4, and C5 are 0.9970 (99.70%), 1.000 (100.00%),1.000 (100.00%),0.9940 (99.40%), and 0.9870 (98.70%), respectively.This collection of accuracy values offers valuable insights into the model performance across the distinct classes.Notably, the model exhibits exceptional accuracy for classes C2 and C3, achieving perfect predictions (100% accuracy) in these cases.While maintaining a notably high accuracy level, the model performance is marginally lower for classes C1, C4, and C5.In summary, the presented CM effectively visualizes the model classification prowess, highlighting its varying degrees of accuracy across different classes.
Overall, figures 8 and 9 display the results of the base model on the dataset in the source domain by depicting the training  history curves and CM diagrams, which are all visual illustrations of the base model.The observations indicate that the base model reaches a satisfactory level of accuracy while effectively minimizing loss.These findings highlight the effectiveness and efficiency of the base model, serving as a foundation for the solution to the tasks in new application scenarios, which is regarded as the target domain in this study.

Testing SMART approach using the dataset of target domain
In order to evaluate the generalizability of the reconstructed model within a novel context, we have utilized a target dataset comprising ten distinct classes.The layers within Blocks #1 and #2 have been deliberately frozen to impede weight updates predicated on the source domain data.In contrast, the layers encompassing Block #3 have been subjected to fine-tuning procedures utilizing the target domain dataset.The trajectory of the training process for the reconstructed model is visually illustrated in figure 10.
Drawing upon the accuracy and loss curves delineated in figure 10, it is evident that the reconstructed model has undergone efficacious training by strategically utilizing recycled weight values originating from the source domain dataset.Moreover, the training time consumption is only 16.92 s, reaching the pinnacle of its performance.This prompt convergence underscores the considerable curtailment in training time costs realized by the proposed SMART approach.
For the purpose of furnishing a more comprehensive exposition of the outcomes pertaining to class prediction, this section integrates graphical representations of the CM, shown in figure 11. Figure 11(a) distinctly underscores the misclassification of only three samples by the rebuilt model.By rescaling the classification outcomes presented within figure 11(b) onto the interval denoted as [0, 1], calculating the proportion of precisely classified and misclassified samples within each distinct class becomes attainable.The presented graphical representation of the CM serves to depict the evaluative aspects of a classification framework encompassing a total of ten discretely delineated categories, denoted as C1 through C10.The ordinate axis corresponds to prognosticated designations, while the abscissa axis delineates veritable designations.The diagonal elements within the matrix furnish a manifestation of precision rates germane to each category, thereby mirroring the proportion of occurrences wherein the prognosis corresponds harmoniously with the actual categorization.
More precisely, both C1 and C2 evince an immaculate precision of 100.00%, a sentiment similarly echoed by C4, C5, C8, C9, and C10 cohorts.By contrast, the accuracy pertaining to C3 exhibits modest mitigation to 90.00%, a phenomenon mirrored within the precincts of C6 and C7.The conspicuously elevated levels of precision manifesting along the diagonal asseverate the commendable performance of the proposed SMART approach across a substantial gamut of scenarios.Nevertheless, in the absence of data proffering access to values situated beyond the principal diagonal, the feasibility of engaging in a thorough exploration of the discrete misclassifications transpiring amidst categories, as well as the potential motifs inherent therein, remains constrained.Further investigation into these non-diagonal entries could potentially provide valuable insights relevant to the effectiveness of the approach in accurately identifying specific categories.

Comparison between the SMART and conventional ML methods
To demonstrate the effectiveness of the SMART approach in a wider context, this study introduces four widely-recognized performance-scoring metrics in the ML and DL communities: Accuracy, Precision, Recall, and F1-score.Additionally, the proposed SMART approach is compared with the performance of representative ML algorithms, such as KNN, SVM, RF, and GBDT, using the entirely same target domain dataset, as detailed in table 5.
Table 5 provides a comprehensive analytical examination of diverse ML methods, encompassing SMART, KNN, SVM, RF, and GBDT, with regard to distinct performance indicators.The metric of accuracy, denoting the proportion of accurate predictions within the entirety of predictions, accentuates that SMART attains the highest precision at 0.9900, with SVM closely trailing at 0.9500.The precision metric, illustrating the proportion of precisely anticipated positive cases relative to the predicted positives, underscores the remarkable performance of the SMART approach at 0.9910.Shifting attention to recall, a measure assessing the capacity to accurately predict positive instances concerning the entirety of actual positive instances, both SMART and SVM distinguish themselves with recall values of 0.9900 and 0.9500, respectively.Finally, the F1-score, a comprehensive metric that combines precision and recall, highlights SMART's exceptional performance with a score of 0.9900.Table 5 enables a comprehensive evaluation of the diverse strengths and limitations demonstrated by these ML methods across various important performance metrics.According to table 5, a bar chart has been generated to provide a visually intuitive representation of the superiority demonstrated by the proposed SMART technique over conventional ML methods, as depicted in figure 12, wherein each bar's height correlates with the respective indicator scores.The visual depiction in figure 12 lucidly conveys that, despite minor fluctuations, the SMART approach consistently maintains the most elevated bar height across all indicators.As elucidated earlier, this graphical presentation underscores that the proposed SMART approach surpasses the performance of J Zhu et al the remaining prominent ML algorithms on the designated dataset of the target domain with a reduced training expenditure.

Conclusion
The constrained capacity for generalization and extendibility within ML and DL methods represents a substantial impediment to their wider deployment.In response to these issues, we propose an ensemble approach termed SMART, encompassing functionalities that address data-imbalance rectification, quantity augmentation, series-to-image transformation, and an FT-based TL strategy.
This study develops a customized model trained on a hybrid dataset with diverse representations from the source domain.The hybrid dataset originates from three co-cured structures and is processed by SMOTE and ARI.These hybrid data are then transformed into RP, MTF, and GASF images, serving as inputs to the model.The primary goal of the SMART approach is to achieve precise categorization using a small dataset derived from embedded optical fibers.The effectiveness of the proposed SMART approach is compared to traditional ML methods using metrics such as Accuracy, Precision, Recall, and F1-score.The results demonstrate that the SMART approach efficiently identifies and categorizes target data in novel scenarios by leveraging weight values from a base model pre-trained on distinct datasets from previous application scenarios.These outcomes highlight the effectiveness of the SMART approach in enhancing generalization and extendibility.Besides, the proposed SMART ensemble approach has the potential to be applied in scenarios involving variations in structural conditions, utilizing different sensors, as long as the collected raw data consists of numerical values.
Although the SMART approach has been demonstrated effective by verification experiments of the curing monitoring of composite structures, several drawbacks should also be considered.The limitations of the proposed SMART approach include the possible introduction of noise and overfitting in SMOTE-generated synthetic samples, inaccuracies in data averaging interpolation for missing value imputation, sensitivity to domain dissimilarity affecting the TL effectiveness, increased computational complexity with large datasets, and the need for domain expertise in parameter selection and transfer strategies.Therefore, further research should focus on refining data generation techniques, domain adaptation methods, scalability improvements, automated parameter tuning, and standardized evaluation protocols to mitigate these limitations.

Figure 1 .
Figure 1.Schematic of the SMART approach.

Figure 2 .
Figure 2. Schematic of the FT architecture to reconstruct a model.

Figure 5 .
Figure 5. RMSD regularity of specimens with heating time increasing.

Figure 6 .
Figure 6.3D waterfall plots of FBG data in the target domain.

Figure 7 .
Figure 7. Schematic of the specimen with an implanted optical fiber.

Figure 8 .
Figure 8. Training history of the base model on a hybrid dataset in the source domain.

Figure 8
Figure 8 depicts a graph that provides a comprehensive analysis of the model performance throughout the training process using the dataset of the source domain.The graph consists of two vertical axes, with the left axis labeled 'Accuracy' and the right axis labeled 'Loss'.The horizontal axis represents the number of training epochs.The graph presents four distinct curves, which offer insights into the training accuracy, validation accuracy, training loss, and validation loss.Figure8indicates that both accuracy curves initially show an upward trend, improving accuracy over training epochs.However, after reaching a turning point at ten epochs, both curves stabilize, suggesting that further iterations may not significantly enhance accuracy.Similarly, the two loss curves consistently decrease during training, indicating that the model effectively reduces errors and converges toward optimal performance.After the turning point at ten epochs, both loss curves level off, indicating that additional epochs might have a limited impact on further reducing the loss.Besides, in

Figure 9 .
Figure 9. diagrams of quantitative classification using the hybrid dataset in the source domain: (a) non-normalized graph, (b) normalized graph.

Figure 10 .
Figure 10.Training history of the rebuilt model using the dataset of target domain.

11 .
CM diagrams of quantitative classification using the dataset in the target domain: (a) non-normalized graph, (b) normalized graph.

Figure 12 .
Figure 12.Comparison between the proposed SMART approach and conventional ML algorithms.

Table 1 .
Specification of specimens used in experiments.

Table 2 .
Scales of the raw and enhanced dataset.

Table 3 .
Overall architecture of the hybrid image dataset.

Table 4 .
Optimal hyperparameters for base model using dataset in the source domain.

Table 5 .
Comparison of indicators between SMART and conventional ML methods.