Prediction of 4D stress field evolution around additive manufacturing-induced porosity through progressive deep-learning frameworks

This study investigates the application of machine learning models to predict time-evolving stress fields in complex three-dimensional structures trained with full-scale finite element simulation data. Two novel architectures, the multi-decoder CNN (MUDE-CNN) and the multiple encoder–decoder model with transfer learning (MTED-TL), were introduced to address the challenge of predicting the progressive and spatial evolutional of stress distributions around defects. The MUDE-CNN leveraged a shared encoder for simultaneous feature extraction and employed multiple decoders for distinct time frame predictions, while MTED-TL progressively transferred knowledge from one encoder–decoder block to another, thereby enhancing prediction accuracy through transfer learning. These models were evaluated to assess their accuracy, with a particular focus on predicting temporal stress fields around an additive manufacturing (AM)-induced isolated pore, as understanding such defects is crucial for assessing mechanical properties and structural integrity in materials and components fabricated via AM. The temporal model evaluation demonstrated MTED-TL’s consistent superiority over MUDE-CNN, owing to transfer learning’s advantageous initialization of weights and smooth loss curves. Furthermore, an autoregressive training framework was introduced to improve temporal predictions, consistently outperforming both MUDE-CNN and MTED-TL. By accurately predicting temporal stress fields around AM-induced defects, these models can enable real-time monitoring and proactive defect mitigation during the fabrication process. This capability ensures enhanced component quality and enhances the overall reliability of additively manufactured parts.


Introduction
In recent years, deep learning models and convolutional neural networks (CNNs) have emerged as powerful tools for field predictions in the domains of engineering and material science [1,2].The potential of deep learning to uncover intricate patterns and spatial dependencies within complex datasets has enabled end-to-end field prediction outputs such as damage, stress, and strain from image datasets of material microstructures [3][4][5][6] or heterogeneous geometries [7][8][9][10].Data-driven models trained using computer vision and semantic segmentation techniques [11] have utilized datasets with both paired [12,13] and unpaired [14,15] images from physics-informed simulation approaches like molecular dynamics (MD) [6,14] and finite element method (FEM) [16][17][18][19].Machine learning models, like the ones presented in this work, offer enhanced computational efficiency and reduced simulation time [17] complementing the precision of physics-informed simulations.
Within the context of field predictions (e.g.stress, strain, or damage distribution in a material) with data-driven models, the integration of semantic segmentation stands out as a major advancement.Semantic segmentation focuses on the pixel-level classification of objects within images [20] and leverages deep learning techniques, such as SharpMask [21], fully convolutional network (FCN) [22], U-Net [23], DeepLab [24], and PSPNet [25].These algorithms, renowned for their CNN-based architectures, excel in image-to-image predictions and have found utility in the analysis of solid structures in tasks ranging from field predictions [16] and defect detection [26] to comprehensive fracture pattern analysis.More advanced methods like Generative Adversarial Networks (GANs) [27], cycle-consistent adversarial neural networks (CycleGAN) [28], and conditional generative adversarial networks (cGAN) [29] have also been used for the generation of images that predict field data [12,[30][31][32].As an example, Hoq et al [12] used several machine learning methods such as artificial neural networks (ANNs) [33], CNNs, and cGAN for the prediction of stress fields in structures with random heterogeneity and concluded that CNNs and cGAN can outperform classical machine learning methods (e.g.random forest and K-nearest neighbors) for such tasks.CNNs were also used by Sepasdar et al [13] and Chen et al [17] for full-field predictions of failure patterns and stress in unidirectional composites using microstructure images.Buehler and Buehler [14] utilized a dataset that was generated through MD simulations and presented a model using CycleGAN to predict mechanical fields using microstructural data that contained defects.The model in Buehler and Buehler [14] showed acceptable generalizability and could do forward (stress field from microstructure image) and backward (microstructural defects from stress fields) predictions.
While CNNs have exhibited promise in doing field predictions [3,13,14,16,17,34], the majority of existing CNN-based models have primarily concentrated on spatial information (limited to 2D with some recent 3D models showing promise [35][36][37]), often disregarding the integration of temporal aspects.This oversight in data dimension and time-evolution becomes especially apparent when considering the complexities of real-world systems, where temporal evolution plays a pivotal role in understanding mechanical behaviors [38].Temporal convolutional networks (TCNs) [39][40][41] have emerged for modeling using special and temporal data and have surpassed traditional approaches such as recurrent neural networks (RNNs) [42].Wang et al [43] used TCNs for capturing the constitutive behavior of granular materials by doing discrete element modeling to generate datasets for training and testing.They evaluated TCN's predictive capability and compared it with a RNN and showed acceptable accuracy of TCN and its ability to filter randomly generated noise in data.Despite their potential, TCNs have received relatively minimal attention for field prediction tasks, often being applied primarily to predict curves.In addition, researchers have been exploring multi-decoder architectures for temporal predictions [44,45].For instance, Wang et al [45] introduced a multi-task model called MultiDeT, utilizing a single encoder and multiple decoders for joint multienergy load prediction, achieving improved accuracy through multi-head attention mechanisms.Chen et al [44] designed a multi-memory state LSTM (MM-LSTM) unit with global memory states and utilized a multi-decoder structure to enhance prediction accuracy for vessel trajectory prediction tasks, showcasing improvements over standard LSTMs.
To address the challenges about prediction dimensionality and time evolutions posed by the studies mentioned earlier in this article, we present two distinct temporal 3D data-driven models, trained using full-scale 3D FE simulation data, to predict stress fields around manufacturing-induced porosities.The first proposed model is a multi-decoder CNN (MUDE-CNN) employing a U-Net structure and the second one is a multi-encoder-decoder model utilizing transfer learning.The efficacy and prediction capability of these models is substantiated through a case study involving the prediction of a time-evolving 3D stress field around a manufacturing-induced isolated pore.The contribution of our research includes three aspects: firstly, the introduction of a data reconstruction method tailored to facilitate the training of CNNs utilizing full-scale 3D FE simulation data; secondly, the presentation of a MUDE-CNN, adeptly suited for 3D temporal predictions; and thirdly, the unveiling of a multi-encoder-decoder model, enriched by transfer learning for the same task.Understanding these methodologies is crucial not just from a computational perspective but also from an application point of view, particularly, when considering applications such as multi-scale modeling [46,47] and in-situ monitoring of additively manufactured parts [48,49] these methods can provide fast and reliable results.

Proposed deep learning modeling approaches
This section introduces two novel models designed to address the challenges in predicting time-dependent mechanical fields such as stress, strain, or damage fields.The first model is a MUDE-CNN based on the U-Net architecture (see section 2.1), and the second model is a multi-encoder-decoder architecture that leverages transfer learning (see section 2.2).

MUDE-CNN
The encoder-decoder architecture is a common model configuration extensively applied to semantic segmentation and image analysis tasks [50].For instance, in semantic segmentation, the encoder captures contextual information from input images, and the decoder generates pixel-wise class predictions [50,51].This approach has demonstrated success in diverse fields, such as medical image segmentation [52], scene understanding [53], and object detection [54].Depending on the specific application, the encoder and decoder components can be chosen from a variety of deep learning models, including CNNs [37], RNNs [42], and LSTM [44].
We leverage CNNs to tackle the tasks of temporally-evolving 3D field predictions by employing a single encoder to extract hierarchical features from input data (shared among multiple time frames).These features are then shared across multiple decoder branches, each adept at decoding information required for 3D prediction of stress field at a distinct time frame, thereby facilitating the concurrent prediction of stress evolution over various temporal instances.Figure 1(a) illustrates the schematic of the multi-decoder model (MUDE-CNN).Such an architecture is especially advantageous when addressing problems characterized by interrelated tasks [44].The shared encoder allows the extraction of features that capture complex hierarchies in data, shared among multiple tasks [44].
The inputs of the multi-decoder model of figure 1(a) are tensors X ∈ R H×L×m with the shape of H × L × m, where H, L, and m represent the length, height, and width of the 3D geometry, respectively.The encoder E 1 extract features X i ∈ R H i +1 × L i +1 ×m i by changing the spatial dimension of the data at different stages by passing the data through Maxpooling layers following equation (1) [55]: As shown in figure 1(a), the model has a total of d decoders { D j |j = 1, . . ., d } and each of them are designed to predict the 3D stress field σ t 1:d (equation (2)) at a distinct time frame.In the decoding layers the historical features from the encoder in fed into convolutional and Upsampling layers to do temporal predictions with the same spatial dimension (H × L × m) of the input tensor.In addition, skip connections are defined between the shared encoder and individual decoders, following the architecture utilized in U-Net [23]  The iterative training of multiple encoder-decoder models allows for the weights of each encoder-decoder to be inherited from its predecessor.Formally, let us denote the weights of the encoder-decoder model at time step t as Θ t .The transfer learning strategy updates these weights in the following equation (3): where ∆Θ t represents the fine-tuning applied to the weights based on the newly available data and the task-specific loss.This approach stands as a testament to the iterative nature of the training, where the predictions from the previous time step enrich the learning process of the next.By gradually adapting the weights to the evolving task requirements, the information accumulated across multiple time frames is put into use.The intuition behind this technique is to exploit the similarity of the prediction task across consecutive time steps.In this case, the stress field predictions at time t share intricate relationships with the predictions at time t − 1, as the underlying physical processes often exhibit temporal coherence.This temporal dependency enables the capitalization of the knowledge learned by the model in the earlier steps.

Case study: stress field prediction around an additive manufacturing (AM)-induced pore
The two frameworks presented in section 2 were utilized to predict the temporally evolving three-dimensional stress distribution in a representative volume element (RVE) containing an isolated pore.
The case study focuses on defects in materials originating from AM processes, as their presence significantly impacts mechanical properties and structural integrity, driving the need for a comprehensive understanding and characterization of the role of such defects in components [58,59].For further information, AM processes and related phenomena are described using the terminology specified in ISO/ASTM 52900 [60], ensuring adherence to international standards and facilitating a clear understanding of the technical aspects discussed.Various research studies have dealt with the flaw formation in AM parts [61][62][63][64].Among the various potential defects [65,66] arising from AM processes [67], pores or voids are consistently recognized as the prevailing imperfections [68][69][70] that hold the potential to significantly compromise the functionality of the manufactured components.An example of such a pore in a selective laser melting (SLM) Ti6Al4V part is shown in figure 3(c) [71].
The pore defects in AM parts can appear due to several mechanisms such as keyhole pore formation [72,73], pores formed during gas atomization [74,75], pores resulting from lack of fusion [76,77], and escaping gas bubbles from the melt pool [78].For instance, in laser power bed fusion, gas porosity typically originates from gas entrapment in powdered material or bubble formation during rapid solidification in the melt pool [79].Despite careful parameter optimization, lack-of-fusion defects may occur, while melt pool instabilities can also introduce material voids [79].The resulting pores with varying sizes from nanoscale or macroscale [80] have a significant influence on the mechanical and thermal properties of additively manufactured components [81][82][83].Such a defect may originate from in-service operations [84,85], manufacturing [65], or material processing [86], exerting substantial influence on mechanical performance by acting as regions of stress concentration and crack initiation.Therefore, this case study is designed to analyze the inhomogeneous distribution of stress around a pore which has been commonly observed in structural components [87][88][89], especially in the case of additively manufactured parts [71,[90][91][92].It is worth mentioning that manufacturing defects in additive processes can be of various shapes [93][94][95][96][97]; however, in our current models, we have primarily considered spherical pores.While spherical pores are commonly reported in the literature [71,98,99], this choice represents a simplifying assumption and does not fully encompass the diverse stress impacts of irregularly shaped pores, a complexity inherent in these manufacturing processes.
To generate the dataset, FE models are auto generated through Python scripting in Abaqus [100], and subsequently subjected to quasi-static unidirectional tensile loading.The resulting stress fields are then extracted from Abaqus result files and transferred to Python for further analysis.

Numerical simulations
To train and evaluate our proposed multiple decoder CNN network for time-dependent three-dimensional stress field prediction around a single isolated pore, we first generate a synthetic dataset using the FE simulations.Figure 2 provides an overview of the automated process executed within Abaqus to generate a dataset of synthetic RVEs.Each RVE in the dataset contains a centrally located pore.The depicted process showcases a sequence of steps that eliminate manual intervention, ensuring consistency and efficiency in dataset creation.Starting from the initial model creation, the automation extends to model generation, mesh application, and subsequent simulation runs.Finally, data reconstruction is facilitated to transfer the data to the ML models in a desired format, as discussed in section 3.2.The geometry of the spherical pore followed the observations from Biswal et al [71] with an average diameter of 34 µm captured from scanning electron microscopy.They also reported that the microstructure surrounding the pore is similar to the overall microstructure.An example of a gas pore in Ti6Al4V material is shown in figure 3(c) [71].For modeling purposes, we only considered the geometry of the pores to be perfectly spherical which is in good agreement with experimental observations [98,99].The behavior of additively manufactured Ti6Al4V material was expressed using J 2 plasticity with an isotropic hardening model following equation ( 4) where σ 0 is the yield strength, ε p is the plastic strain, and K and n are constants estimated from fitting the experimental stress-strain curves [101,102] with equation ( 4).The material properties are reported in table 1.The FE models are meshed with C3D10 (10-node quadratic tetrahedron) elements following a mesh convergence analysis.The finite element model has been validated by comparing with the experimental data of additively manufactured Ti6Al4V fabricated through SLM [102,104] and direct laser deposition (DLD) [101], as well   5) and ( 6)) and numerical predictions of stress concentration factor (c): a spherical gas pore defect in a SLM Ti6Al4V part.Reprinted from [71], Copyright (2018), with permission from Elsevier.as wrought material [105].Quasi-static uniaxial tensile simulations were conducted with a strain rate of 0.001 s −1 following the experimental setup in Sterling et al [101].The comparison of the experimental and numerical stress-strain data is shown in figure 3(a) indicating good agreement in the prediction of the elastic-plastic response of the material.Another important issue when dealing with internal defects is the stress concentration factor (K t ) around the defect, which is defined as the ratio between the maximum local stress and the applied stress.The FE models were validated in terms of predicting the stress concentration factor by comparing them with analytical solutions [71,106].It has been observed that for high length-to-diameter ratios, the change in pore size does not affect the stress concentration factor around the pore [71,106].Equation ( 5) estimates the stress concentration factor of a spherical cavity subjected to tension in an infinite body [106].For a central spherical pore in a finite radius cylinder subjected to remote tension, the stress concentration factor can be estimated using equation ( 6) [107].Both equations ( 5) and ( 6) are introduced as: where a is the diameter of the spherical pore and c is the diameter of the cylinder.Figure 3(b) shows the comparison of the stress concentration factors predicted from the FE model with the analytical solutions from equations ( 5) and ( 6) indicating that the finite element model setup is capable of accurately predicting the K t .

Data reconstruction
The time-dependent stress field data obtained from Abaqus simulations undergoes preprocessing to prepare it for training the ML models.This step includes data normalization, scaling, and ensuring the consistent formatting of the data into a structured array suitable for training deep learning models.The initial stress data, extracted at integration points, provides stress values and their corresponding spatial locations.To transform this raw data into a format compatible with deep learning frameworks, a multi-step process is implemented.The process commences with spatial interpolation using the griddata function.The griddata function facilitates the creation of slices by discretizing the continuous spatial domain following equation ( 7): where σ t 1:d (x, y, z) represents the interpolated temporal stress value at a given spatial point (x, y, z), (X, Y, Z) denote the coordinates of the integration points where stress data is available, and σ t raw represents the raw stress values extracted from the FE solver.This results in structured data that can be ingested by deep learning architectures bridging the gap between raw stress information from Abaqus and the requirements of deep learning models.Each slice encapsulates stress values across a specific spatial section, contributing to the training dataset's structured array, as shown in figure 4(a).Figure 4(b) shows the visual representation of the final structured data array in a typical slice.The input array contains the model's geometry, using '1' to signify material regions and '0' for pore locations, while the target array contains normalized stress data corresponding to each spatial point.A sensitivity study of the prediction capability of CNN models with different numbers of slices is performed in section 4.

Deep learning model setup
Figures 5(a) and (b) show the network structure for the MUDE-CNN and MTED-TL, respectively.In both models, the same architecture is used for single blocks of the encoder or decoder as summarized in tables 2 and 3.The encoder is responsible for learning spatial representations from the 3D stress field data.As shown in table 2, starting with a Conv2D layer, the encoder processes the input data, where each layer utilizes distinct convolutional filters to extract progressively abstracted representations.Three skip connections integrated at the encoder (see figure 5) allow the preservation of spatial information to the decoders.Three Maxpooling layers reduce the spatial dimensions, enhancing computational efficiency, and dropout layers contribute to preventing overfitting.Conversely, the decoder mirrors the architecture of the encoder, albeit in reverse order, to reconstruct the desired target from the compressed representation.Beginning with a Conv2D layer, the decoder gradually expands the spatial dimensions while maintaining a focus on feature extraction.Concatenation operations combine features from earlier layers, allowing for the integration of high-resolution information.The three Upsampling layers that are placed within the decoder restore the dimensions lost during the encoding phase, refining the reconstructed output.The decoder concludes with a final Conv2D layer employing a Sigmoid activation function to produce the model's output.Figure 5(a) illustrates the MUDE-CNN model comprising one encoder and nine decoder blocks.A key implementation involves incorporating skip connections from the encoder to all decoders.This strategy enables information sharing, enhancing each decoder's ability to leverage both high-level abstractions and fine details for accurate output generation.
In figure 5(b), the MTED-TL architecture is presented, a significant advancement built upon the foundation of nine encoder-decoder blocks.This model employs transfer learning to enhance training efficiency and performance.Each encoder-decoder block operates as an integrated unit, leveraging transfer learning to pre-train on a related task and subsequently fine-tune on the specific task of interest.The proposed networks employed various activation functions, namely ReLU, Leaky ReLU, and Sigmoid, each serving a distinct role.The selection of an activation function is a critical decision as it significantly impacts the model's ability to grasp intricate data patterns [108].Sigmoid, ReLU, and Leaky ReLU Functions used in this article are described by equations ( 8)-( 10) [109], respectively The dataset containing stress fields of 100 models, each with varying characteristic lengths (pore diameter over RVE length, d/L) ranging from 0.12 to 0.35 was created using a uniformly distributed sampling method.The RVE geometry and the average pore diameter were adjusted to follow the experimental observation of an average diameter of 34 µm in the literature [71,110].The dataset was divided into an 80-20 split for training and testing, where 80% of the data was allocated for training purposes, and the remaining 20% was reserved for testing and evaluating the models' performance.
To optimize the model's parameters and minimize the loss function, we employ the Adam optimizer [56] which is based on the stochastic gradient descent method.The learning rate of 0.001, β 1 of 0.9, and β 2 of 0.999 were tuned to achieve the best performance.The MUDE-CNN model, featuring its one encoder and nine decoders had 27 734 435 trainable parameters and underwent simultaneous training for 1800 epochs.In the MTED-TL model, encoder and decoder blocks (having 5136 203 trainable parameters each) are individually trained for 1800 epochs.The previously learned weights from each block serve as initial weights for the subsequent one.The choice of the mean squared error (MSE) loss function was motivated by the objective of minimizing the difference between the models' predicted outputs and the ground truth values.This alignment was pursued throughout the training process, following equation ( 11) where Ŷi are the predicted values by MUDE-CNN and MTED-TL models, and the Y i are the ground truth values from FE simulations.

Results and discussion
In the Results section, we begin by investigating the sensitivity of the encoder-decoder models to variations in slice number and training range.Subsequently, we present the outcomes of the time-dependent 3D stress field (MUDE-CNN and MTED-TL) evaluations.

Single step model
In this section, a single encoder and decoder model (time-independent) is trained focusing specifically on capturing elastic stresses to conduct a sensitivity analysis on the influence of data, reconstruction, slice numbers, and training time on the model's performance.Such a model can be used for the estimation of stress concentration factors around three-dimensional defects and its applications extend to many  engineering fields such as AM [96,111], biomechanics [112,113], and aerospace [114].Figure 6  Here, all the 2D and 3D stress contours are from test data.Visually, a strong agreement between the ground truth and the models' prediction can be observed regardless of the number of slices used in the data reconstruction process.As expected, increasing the number of slices in the model increased the computation time required for the training of the models, however, it offers a significant advantage in achieving higher-resolution 3D field predictions, as demonstrated in figure 7 for the same models.
In figure 8(a), the plot of MSE loss against epochs for train and test data of models with different slice numbers is shown.Notably, the training loss consistently remains marginally lower than the test loss for all models.This behavior is expected and signifies the models' ability to accurately predict stress field values in both known and novel scenarios.During the initial 200 epochs of training, the loss demonstrates a sharp and steep decrease, highlighting the model's rapid adaptation to the dataset.Subsequently, the loss reduction becomes more gradual, indicating a smoother and consistent refinement of the model's predictions.Furthermore, table 4 shows that an increase in the number of slices used in training is associated with a  Train loss 0.000 166 0.000 233 0.000 237 Test loss 0.000 463 0.000 556 0.000 457 slightly higher loss, which can be attributed to the additional trainable parameters in the model that require fine-tuning.Figure 8(b) compares the predicted and ground truth stress values for all points in a single geometry (model with m = 61 in figure 7).Most points exhibit proximity to the x = y line and fall within a 10% error range, demonstrating the model's accurate stress prediction.However, certain points show deviations, which especially occur near the pore's edge, highlighting the model's potential difficulty in capturing stress variations with steep gradients.In addition, figure 8(c) shows 3D stress field predictions at different training epochs (600, 1200, and 1800).Despite the variations in training time, the models consistently capture stress field features, resulting in visually similar predictions.The improved loss over epochs signifies the model's ability to refine its predictions during training.

Temporal-3D stress field predictions by MUDE-CNN and MTED-TL
This section presents the results obtained from the MUDE-CNN and MTED-TL temporal models.These models perform predictions for the 1st to 9th time frames illustrated in figure 3(a).Figures 9(a)-(e) shows an examination of the loss trajectory against epoch progression for both the MUDE-CNN and MTED-TL models across distinct temporal frames.The figures show accurate predictions and acceptable loss values from both models, highlighting their proficiency in handling the complexities of the temporal problem.Significantly higher accuracies were achieved by the MTED-TL model as indicated by notably lower loss values exhibited by MTED-TL compared to MUDE-CNN.Both models show similar loss curves for the first time frame (t 1 shown in figure 9(a)), however starting from this time frame, MTED-TL is showing loss values that were unprecedented in the whole training process in the MUDE-CNN model.MTED-TL showed better efficacy at t 1 compared to MUDE-CNN due to the lower number of trainable parameters required to train a single encoder-decoder network in contrast to an encoder-multiple decoder network.The moments in which these unprecedented losses were observed in the MTED-TL model are marked with vertical solid and dashed lines in figure 9 for train and test loss, respectively.
As we go through different time frames, it is clear that MTED-TL consistently outperforms the MUDE-CNN, with a significantly high margin in later frames, making the longer training cycles of MUDE-CNN look less effective in comparison.Notably, as shown in figures 9(d) and (e), in later time frames, MTED-TL's initial predictions (at epoch = 1) surpass the accuracy achieved by MUDE-CNN after 1800 epochs.This is because, after the 1st time frame, MTED-TL starts with a boost, using the knowledge it gained from previous tasks through transfer learning.This pronounced discrepancy highlights the inherent proficiency of MTED-TL in capturing intricate temporal dependencies with remarkable fidelity, yielding a notable accuracy profile.Moreover, MTED-TL exhibits smoother loss curves, indicative of consistent weight and bias adjustments that capitalize on the knowledge transferred within the network architecture.
Figures 10 and 11 show the 3D temporal predictions of stress fields of isolated pores for two distinct pore geometries ( d / L = 0.18 and d / L = 0.34) drawn from the test dataset of the MTED-TL and MUDE-CNN models in comparison to the 3D field from FE simulation (ground truth).Additionally, these figures also display contours representing the absolute error between the model predictions and the FE simulation results.Both models successfully captured the temporal stress fields, but notably, the error analysis demonstrates that the predictions generated by the MTED-TL model exhibited a higher level of accuracy compared to those of the MUDE-CNN model.The examination of the 3D field predictions at initial time frames (t 1 and t 2 ) reveals a higher degree of consistency between the MTED-TL and MUDE-CNN models, but as the time frame increases, the benefits of transfer learning become increasingly evident for MTED-TL.Specifically, at later time frames such as t 4 , t 6 , and t 8 , as depicted by contours in figures 10 and 11, the MTED-TL model exhibits a marked improvement in similarity to the ground truth and a notable reduction in prediction error compared to the MUDE-CNN model.
Figure 12 shows the error percentage distribution of the isolated pore geometries of figure 10 ( d / L = 0.18) and figure 11 ( d / L = 0.34) based on predictions by the MTED-TL and MUDE-CNN models for various branches.Across all branches, the majority of errors fall within the range of under 4%, signifying reasonably accurate predictions based on previous papers [35,115,116].Remarkably, many branches exhibit a pronounced peak around zero error, indicative of the models' proficiency in predicting 3D stresses.By comparing the two models, it was observed that the MTED-TL model generally outperforms MUDE-CNN, with its error distribution more densely populated toward zero.Moreover, smaller pore sizes (figures 12(a) and (c)) tend to yield more accurate results, and this can be attributed to smaller pore surface area that required less edge detection by the ML models.These observations collectively reveal valuable insights into the predictive capabilities of the models and the impact of varying conditions on the accuracy of stress predictions.
Figure 13 presents an examination of the training times for the showcased models.Comparing the training times of MTED-TL with the MUDE-CNN showed remarkably shorter training times for MTED-TL, indicating its accelerated learning process.This observation is particularly noteworthy considering the unique training approach employed in this study.The models, MTED-TL and MUDE-CNN were trained using two different strategies: one involved sequential training of individual branches, while the other trained all branches simultaneously.The model utilizing transfer learning demonstrated greater efficiency, in line with existing literature and highlighting the potential for transfer learning to reduce computational time [117,118].Another key factor contributing to this phenomenon is MTED-TL's individualized branch training approach.This strategy enables the model to benefit from specialized knowledge acquired during sequential training, resulting in faster convergence and improved efficiency.

Further discussion: new training strategy
To enhance prediction accuracy for temporal predictions, we employed an autoregressive training framework to train the MTED-TL model.This approach allowed the model to incrementally refine predictions by utilizing previously observed target values as inputs, thereby significantly improving its capacity to capture temporal dependencies within the data.This approach, as shown in figure 14, involves utilizing the target values of the current step as input for the next step following equation ( 12) where Ŷt i is the model prediction at the time frame of t and Y i t−1 is the target value at the time frame of t − 1.This autoregressive framework allows our model to incorporate its predictions into subsequent predictions, fostering a feedback loop that can capture dependencies within the data.
Figure 15 shows the minimum loss achieved by three models (MUDE-CNN, MTED-TL, and the autoregressive trained model) across all time frames (t 1 -t 9 ).Notably, the autoregressive trained model consistently outperformed the other two models, highlighting its superiority as a training strategy.Specifically, both MUDE-CNN and MTED-TL models faced challenges in achieving low loss at t 6 , while the autoregressive trained model demonstrated remarkable performance in this time frame, underscoring its effectiveness in handling this particular temporal scenario.In addition, table 5 provides a comparative analysis of the number of epochs where MTED-TL and the autoregressive trained model achieved unprecedented levels of accuracy compared to MUDE-CNN.This analysis serves to underscore the superior accuracy of the autoregressive training approach, with specific emphasis on its performance in contrast to the MUDE-CNN model.

Future work and limitations
The deep learning models presented in this paper promise both rapid and precise predictions, crucial in scenarios like real-time structural assessments [119,120] or during novel manufacturing processes [48,49].The authors are actively working on the application of these models for digital twin-based high-throughput modeling.This endeavor aims to make these models suitable for real-time monitoring of AM processes, ensuring immediate and accurate structural assessments that are essential for quality control.The authors are also progressing towards integrating x-ray computed tomography scans for the generation of synthetic datasets of defects, which will be instrumental in training models that are more tuned to real-world applications (e.g.diagnostics and performance prediction in AM).This approach is part of our broader effort to apply these models for digital twin-based high-throughput modeling.By doing so, these models will have the potential to significantly reduce the time and cost associated with traditional mechanical property analysis methods like experiments, FEM, and MD simulations.Their high throughput and speed are crucial for handling large datasets and facilitating quick decision-making in design iterations and real-time operational environments.In fracture mechanics and stress-field analysis, the model's ability to analyze microstructure-based 3D stress fields is invaluable for identifying potential stress concentrations and fracture initiation points, paving the way for data-driven fracture mechanics and the development of materials optimized for specific performances.Additionally, its real-time functionality makes it ideal for in-situ evaluations in manufacturing, enhancing quality control by allowing immediate detection of material defects.Furthermore, the models can be used to bridge the gap between micro-scale structures and macro-scale mechanical properties, providing a holistic understanding of materials and serving as a critical tool in multi-scale modeling.
While the results showed the promise of deep learning models for temporal 3D stress field prediction, it is crucial to recognize the presence of certain limitations that may temper the enthusiasm for deploying such models.The deep learning models, MUDE-CNN and MTED-TL undoubtedly demonstrated proficiency in their stress field predictions, but they excel in familiar scenarios and their performance in uncharted territories remains unexplored.Model interpretability, explainability, and generalization are critical dimensions that necessitate further exploration and refinement.For these models to transition from research achievements to dependable tools in engineering and related domains, a comprehensive understanding of their limitations and a concerted effort to address them is essential.

Conclusions
In this study, we employed deep learning models to predict 3D temporal fields from full-scale training data of FE simulations.These deep learning models, namely the MUDE-CNN and the Multiple Encoder-Decoder Model with Transfer Learning (MTED-TL), offer innovative solutions to the challenging problem of predicting stress distribution in complex materials and geometries over various temporal instances.Both models follow a comprehensive data processing and reconstruction and demonstrate promising potential for accurate stress field predictions in geometries containing an isolated pore.
A sensitivity analysis was performed, exploring the influence of data variations, reconstruction techniques, and training parameters on model performance.This research revealed several key findings.Firstly, the single-step model demonstrated robust generalization and effective stress field predictions across different slice numbers, with lower slice counts yielding higher-resolution predictions.Secondly, the temporal models, particularly MTED-TL, excelled in capturing complex temporal dependencies, outperforming MUDE-CNN with notably lower loss values.MTED-TL's superior performance was particularly evident in later time frames, emphasizing its proficiency in leveraging transfer learning to enhance predictive accuracy.Additionally, MTED-TL exhibited faster training times, highlighting the efficiency of this training approach.
Furthermore, we introduced an autoregressive training framework to improve temporal predictions.The autoregressive training framework further improved the temporal predictions, consistently outperforming both MUDE-CNN and MTED-TL.This approach enabled the model to refine predictions by incorporating previous time step information, resulting in superior accuracy, particularly in challenging temporal scenarios.
In conclusion, this study showcases the potential of deep learning models for predicting 3D stress fields in a pre-containing media, with a particular emphasis on temporal predictions.Unlike MM-LSTM models, which are excellent for time-series predictions but may not fully capture spatial relationships in three dimensions, the two models presented in this paper are inherently designed to process and analyze spatial data.This capability allows the models (MUDE-CNN and MTED-TL) to accurately predict stress fields around voids in three dimensions, while also making predictions of temporal evolution which is a novel approach in this field of research.Moreover, the results highlight the importance of considering various factors such as data variations, reconstruction techniques, and training strategies to optimize model performance.The transfer learning and autoregressive training framework was a promising approach for improving temporal predictions.While these models hold great promise, further research is necessary to enhance their interpretability and generalization, ultimately facilitating their transition from research tools to practical applications in engineering and related fields.

Figure 1 .
Figure 1.Schematic representation of proposed architectures for 3D temporal predictions, (a) multi-decoder CNN (MUDE-CNN) based on U-Net structure, enabling accurate 3D temporal predictions.(b) Multiple encoder-decoder model: illustration of the hierarchical model employing transfer learning, showcasing the shared knowledge transfer across encoders for enhanced 3D field predictions.

Figure 2 .
Figure 2. Automated generation of synthetic RVE dataset with central cavity using Abaqus.

Figure 3 .
Figure 3. Validation of the progressive response in comparison to AM [101, 102] and wrought [105] material, (a): comparison of the experimental and numerical stress-strain curves, (b): comparison of the analytical (equations (5) and (6)) and numerical predictions of stress concentration factor (c): a spherical gas pore defect in a SLM Ti6Al4V part.Reprinted from[71], Copyright (2018), with permission from Elsevier.

Figure 4 .
Figure 4. Illustration of final 3D structured data array, (a): data reconstruction to transfer raw data into structured arrays, and (b): material geometry, pore locations, and normalized stress data for a typical slice of the final arrays.

Figure 5 .
Figure 5. Architectural overview of proposed models (a): MUDE-CNN model made of one encoder and nine decoder blocks, and (b): MTED-TL model with nine encoder-decoder blocks.

Figure 6 .
Figure 6.Comparison of stress prediction in the middle slices from the ground truth and models with varying slice numbers of m = 21, m = 41, and m = 61.

Figure 7 .
Figure 7. Stress field predictions with different slice numbers (m = 21, m = 41, and m = 61) demonstrate increased spatial resolution with an increase of slice numbers.
shows a side-by-side comparison of the ground truth stress field and the predicted stress field for the slice in the middle of the 3D geometries with different pore sizes.Three different models, each utilizing a different number of slices (m = 21, m = 41, and m = 61) explore how the variations in slice count may affect the model's performance in predicting 3D stress fields.Considering that the models were trained using a dataset from 100 3D models with different pore diameters, more than 34.4,67.1, and 99.9 million stress data points were taken from the FE model and were used to train the models with m = 21, m = 41, and m = 61 slices, respectively.

Figure 8 .
Figure 8. (a): Loss during training and testing over epochs for models with different numbers of slices, (b): the predicted and ground truth stress values for all points in a single geometry with m = 61, and (c): 3D stress field predictions at different training epochs (600, 1200, and 1800).

Table 4 .
Final train and test losses for models with different numbers of slices.Model m = 21 m = 41 m = 61

Figure 10 .
Figure 10.Temporal prediction vs. ground truth showing stress field predictions of MTED-TL and MUDE-CNN models for a pore with d / L = 0.18.

Figure 11 .
Figure 11.Temporal prediction vs. ground truth showing stress field predictions of MTED-TL and MUDE-CNN models for a pore with d / L = 0.34.

Figure 13 .
Figure 13.Time prediction efficiency of the MTED-TL and MUDE-CNN models.
[56]ls, to facilitate the fusion of hierarchical features and accurate segmentation by preserving spatial information across multiple resolution scales.Algorithm 1 summarizes the workflow of the proposed model structure that has been implemented using Tensorflow 2.0[56]

Table 2 .
Network structure for encoder blocks.

Table 3 .
Network structure for decoder blocks.

Table 5 .
Epochs when superior accuracy in MTED-TL and autoregressive training models in comparison the MUDE-CNN model was observed.