Supplementation of deep neural networks with simplified physics-based features to increase accuracy of plate fundamental frequency predictions

The training and testing datasets for this study were generated through normal modes analysis using the finite element software ANSYS Version 2021, Release 1 [20]. Thin rectangular plates with four simply-supported edges having varying dimensions and material types were modeled, and their fundamental natural frequencies calculated. Figure 2 shows an example finite element mesh for the plates under study, as well as a sample contour plot of the fundamental mode shape corresponding to the frequency of interest. The mesh itself was scaled for each individual plate size and adjusted to minimize discretization and other element approximation errors. Table 1 provides relevent mesh metrics used to assess the finite element models.

Zoom In Zoom Out Reset image size

As well as serving as inputs to the finite element models, the minimal dataset input parameters used for all ML-based model generation comprise three plate dimensions (thickness, width, and length), weight density, and two elastic properties (Young's modulus Poisson's ratio), for a total of six features. To provide a somewhat diverse—but limited—set of data used to train the models, plates were simulated using aluminum, FR-4, copper, magnesium, and stainless steel. Intrinsic and elastic properties for these materials are provided in table 2, while the diversity of dimensions used is given in table 3.

Table 3. Dimensions of simply-supported rectangular plates used in the generation of training data. In total, five different combinations of planar dimensions were combined with each increment of plate thickness to give a total of 500 datasets. These 500 datasets were split into 261 for training/testing and 239 for independent testing.

Dimension	Set	Value
	i.	0.0508 m $\times$ 0.0508 m
	ii.	0.0476 m $\times$ 0.0762 m
Planar length and width, $w\times l$	iii.	0.133 m $\times$ 0.178 m
	iv.	0.152 m $\times$ 0.0672 m
	v.	0.267 m $\times$ 0.222 m
Plate Thickness, t:		0.762 to 3.18 mm; increments of 0.127 mm

Using the aforementioned parameters, a total of 500 solutions were calculated. From this, 261 datapoints are reserved for model generation (training and validation), while the remaining 239 datapoints are retained for independently assessing model predictive accuracy.

One final note regarding the training/testing dataset is the application of uncertainty. Since the data generated for this study are computational rather than physical, to simulate a possible 2% (+/−1%) variation due to meaurement uncertainty and imperfect physical sample, the final output feature—i.e., natural frequency—is adjusted as follows:

$$\begin{eqnarray}&&{f}_{n,{adj}}={f}_{n}\left(0.99+0.02R\right)\end{eqnarray} \tag{ 1 }$$

where R is a random number between the values of 0 and 1 having an approximately Gaussian distribution, and ${f}_{n,{adj}}$ is the final adjusted natural frequency used at the output parameter. (For convenience, the adjusted natural frequency is simply referred to as ${f}_{n}$ for the remainder of this study.) This addition of uncertainty was simply to emulate measured data to a degree and analysis of its effect is not a focus of the study.

Simplified physics used for integration with the deep learning model is partly based on small-deflection plate theory, adapted from Steinberg [21]. Using this approach, the fundamental natural frequency, ${f}_{n},$ of the rectangular plate simply-supported along four edges shown in figure 1 is as follows:

$$\begin{eqnarray}&&{f}_{n}=\frac{\pi }{2}{\left(\frac{{Dt}}{\rho }\right)}^{1/2}\left[\frac{1}{{w}^{2}}+\frac{1}{{l}^{2}}\right]\,\end{eqnarray} \tag{ 2 }$$

where $t$ is the plate thickness, $\rho$ is the material mass density, $w$ and $l$ are the plate width and length, respectively, and $D$ is flexural rigidity. Flexural rigidity is computed from thickness, Young's modulus, $E,$ and Poisson's ratio, $\nu :$

$$\begin{eqnarray}&&D=\frac{E{t}^{3}}{12\left(1-{\nu }^{2}\right)}\end{eqnarray} \tag{ 3 }$$

Note that this term—calculated independently from planar dimensions and boundary conditions—becomes the primary physics-based relation used to augment DNN-based model generation. This is important because while the plate equation is simple enough to use to calculate fundamental frequencies for our current configuration, for the majority of real-world applications, physical boundary conditions are not easily classifiable and may be a mix of simple, clamped, or elastic, to name a few. However, flexural rigidity is relatively easy to estimate for any plate based on material elastic properties and plate thickness. Therefore, this approach can be adapted to any dimensional and boundary condition configuration.

Since flexural rigidity is only dependent on three features—plate thickness, Young's modulus, and Poisson's ratio—an attempt is made to involve the remaining dimensions and material density into simplified physics by considering the plate weight, $W,$ calculated as

$$\begin{eqnarray}&&W=t\times w\times l\times \rho \end{eqnarray} \tag{ 4 }$$

Shear modulus, $G,$ is used as a third and final supplemental term and is calculated as:

$$\begin{eqnarray}&&G=E/\left[2\left(1+\nu \right)\right]\end{eqnarray} \tag{ 5 }$$

Since equation (2) is based on thin plate theory for small deflections, shear modulus has no direct bearing on natural frequency calculations where such assumptions are present; rather, it is chosen purely for its inclusion of both Young's modulus and Poisson's ratio, providing a relationship between the two. Particularly, its presence is intended as an additional datapoint to inform the DNN of the relationship between elastic parameters.

Based on early experimentation, the basic predictive model chosen and developed for this study is a deep neural network comprising four hidden layers and one output layer, as diagrammed in figure 3(a). The optimization algorithm chosen for this study is Adam, introduced by Kingma and Ba [22]. Implementation of each DNN was conducted using Python 3.8.8 64-bit through the Spyder 4.2.5 interface, as provided by Anaconda Navigator 2.3.2 [23]. The code used for implementation was initially patterned after that developed by Pawar et al [17].

Zoom In Zoom Out Reset image size

Data used for training and validation of the neural networked-based models were configured into comma-separated-value(s) (CSV) format. The resulting data files contain labeled column headings corresponding to the six input parameters (plate thickness, length, width, density, Young's modulus, and Poisson's ratio) and one output parameter (fundamental resonance). For iterations where augmented data were used, additional columns were added, as necessary. Specifically, these include flexural rigidity (D), plate weight (W), and material shear modulus (G). In all cases, data rows—each representing a different plate configuration—were randomized within the CSV files in order to remove biasing and allow representative diverse sampling to be used for validation.

Prior to model generation, preprocessing was conducted to condition the data for use with the neural network. Specifically, values for all parameters (input and output) were normalized to a range of 0 to 1 using min-max scaling, as follows [24]:

$$\begin{eqnarray}&&{X}_{{SC}}=\left(X-{X}_{\max }\right)/\left({X}_{\max }-{X}_{\min }\right)\end{eqnarray} \tag{ 6 }$$

where $X$ is the datapoint undergoing scaling for a given parameter, ${X}_{\max }$ is the maximum value for the given parameter within the dataset, ${X}_{\min }$ is the minimum parameter value for the given dataset, and ${X}_{{SC}}$ is the final scaled value.

Hyperparameter selection was decided by trial and error rather than through a rigorous optimization process. For consistency, all DNN-based model generation is conducted using the parameters provided in table 4. Note that the training/testing data was parsed such that 20% was reserved for validation, with the balance used for model training.

Table 4. Selected hyperparameters and their values. Note that for performance comparison purposes, hyperparameters remained constant throughout the study.

Hyperparameter	Description/Value
Hidden layer activation function	Rectified Linear Unit (ReLU)
Output layer activation function	Linear
Learning rate, ${\alpha }_{t}$	0.01
Batch size	80
Epochs	500
Number of neurons per layer	40
Numerical stability parameter, $\hat{\epsilon }$	1e-6
Loss function	Mean-Squared Error (MSE)

As previously discussed, the methodology chosen for integrating physics into the process of machine learning is to insert physics-based quantities as augmentative data into various DNN layers. Four basic approaches are taken, represented in figure 3:

• 1.  
  Neural network with no physics terms (figure 3(a)),
• 2.  
  insertion of physics into Layer 1 (figure 3(b)),
• 3.  
  insertion of physics into Layers 2, 3, 4, or 5 individually (figure 3(c)), and
• 4.  
  insertion of physics into multiple layers for each run (figure 4(c)).

These four strategies represent nine total DNNs: one basic network without additional physics terms and eight different ways of integrating physics into the baseline DNN.

Along with varying locations of physics integration within the DNN model, six different combinations of calculated parameters are examined, listed in table 5. The purpose behind this is to observe whether single or multiple variables show the most improvement when integrated into the DNN.

Table 5. Various models developed to predict natural frequency. Specifically, this entails the injection of different combinations of weight (W), flexural modulus (D), and shear modulus (G) at each DNN layer.

Model	Physics	Code file name
Baseline Deep Neural Network	None	NNML.py
Physics-Guided Deep Neural Network 1	$W,$ $D$	PGML.py
Physics-Guided Deep Neural Network 2	$W$	PGMLa.py
Physics-Guided Deep Neural Network 3	$D$	PGMLb.py
Physics-Guided Deep Neural Network 4	$G$	PGMLc.py
Physics-Guided Deep Neural Network 5	$W,$ $D,$ $G$	PGMLd.py
Physics-Guided Deep Neural Network 6	$D,$ $G$	PGMLf.py

The final consideration for model development is amount of training data available, for which the following dataset sizes are used: 261 datapoints, 117 datapoints, 60 datapoints, and 30 datapoints. The largest dataset containing 261 points serves as a superset from which the 117, 60, and 30 are derived. The goal in examining different training dataset sizes is to determine how much—if any—impact physics has on improving accuracy of the DNN when only scarce data are available.

Based on the various combinations described herein, the total number of models generated for this study is

$$\begin{eqnarray}&&\begin{array}{c}\left(4\,{baseline}\,{DNN}\,{Models}\right)+\left(9\,{insertion}\,{options}\right)\\ \times \,\left(6\,{physics}\,{combinations}\right)\times \left(4\,{training}\,{data}\,{sizes}\right){\boldsymbol{=}}{\bf{220}}\,{\boldsymbol{models}}\end{array}\end{eqnarray} \tag{ 7 }$$

Each of the final 220 models is used to compute multiple predictions of fundamental natural frequency while initialized with different random seeds, where the final output value is realized as an average of all computed values (each having equal weight). The purpose for this is to reduce variation in model predictions. In total, fifty values of natural frequency are computed for each model. This number of averages was chosen based on early trials demonstrating that—while not strictly true—the overall average would tend toward a quasi-stable value between 20 and 30 iterations with the baseline (non-augmented) neural network and for all four training dataset sizes. However, since not all datasets in the final study are reviewed for this particular trend, a factor of approximately 2x was chosen as the standard number of averages to better ensure stability. An example of the stability trend using the training dataset size of 261 is shown in figure 4.

Zoom In Zoom Out Reset image size

The final term of high importance computed for assessment of model accuracy is percentage error. Once a solution of fifty averages is attained, the error for each resultant datapoint is calculated as

$$\begin{eqnarray}&&Error=\displaystyle \frac{\left|{f}_{n.a}-{f}_{n,p}\right|}{{f}_{n,a}}\times 100 \% \end{eqnarray} \tag{ 8 }$$

where ${f}_{n,a}$ is the actual natural frequency of the plate as determined through finite element modeling and ${f}_{n,p}$ is the natural frequency predicted by the DNN-based model. Based on this information, several basic quantities are available, such as mean error, median error, standard deviation of error. Thus, these are investigated for each predictive model.

The error terms for each physics-guided DNN model are evaluated against those for the baseline DNN without integrated physics. This provides insight into the overall improvement a particular physics-guided DNN has in predicting plate natural frequency. To ensure any conclusions are statistically significant, the two-sample t-Test assuming Gaussian distribution but unequal variances is conducted between the baseline DNN and each physics-enhanced DNN individually. Results are considered statistically significant for a two-tail p-value of 0.05 (5%) or less.

Following development of predictive models through training and testing with original data having parameters from tables 2 and 3, each is tested using two independent datasets in order to assess prediction accuracy. The first dataset independently tested with each model—henceforth referred to as Test Dataset 1—contains the remaining 239 datapoints developed from finite element modeling of the aluminum, copper, FR-4, magnesium, and stainless steel plates. The second independent dataset—Test Dataset 2— consists of a series of 101 plates constructed from composite printed wiring board (PWB) material having the isotropic property values and random dimensions within the extreme parameters provided in table 6.

Table 6. Independent Dataset 2 input features. Material elastic properties taken from Steinberg [21], while density is estimated. Planar length (l), planar width (w), and plate thickness (t) for the 101 different plates are generated at random within the bounds shown.

Feature	Value
Density	4,200 kg m−3
Young's modulus	21 GPa
Poisson's ratio	0.18
Planar length, $l$	0.0518 to 0.250 m
Planar width, $w$	0.0512 to 0.250 m
Plate Thickness, t:	0.610 to 5.49 mm

Selection of printed wiring board material properties for independent Test Dataset 2 was undertaken for two primary reasons. First, systems containing PWBs are often subjected to vibration environments in practice. As such, their use in this study provides a real-world rationale for development of a methodology to accurately predict natural frequencies of plates. Second, PWBs are often fabricated from a layup of copper and FR-4 material, both used in generation of the training dataset. As seen in a comparison of tables 2 and 6, PWB material properties lie between those for FR-4 and copper and therefore lie within the values used to develop the predictive DNN-based model.

Figure 5 shows mean predictive error for each physics-guided DNN model when compared with the baseline DNN containing no physics. Test Dataset 1 is used for independent testing of each model. Note that mean error for the baseline DNN without physics is 2.1% for the largest training size of 261 datapoints, which is approximately equivalent to the range of uncertainty assumed for plate natural frequency measurements. As expected, baseline DNN models generated with decreased training dataset sizes of 117, 60, and 30 datapoints exhibit increasingly higher mean error, with values of 5.0%, 6.4%, and 19.3%, respectively. Figure 6 shows the corresponding median predictive error and variability for each physics-guided DNN model when compared with the baseline DNN containing no physics. Note that median error for the baseline DNN without physics is 0.95% for the largest training size of 261 datapoints. As with mean error, baseline DNN models generated with decreased training dataset sizes of 117, 60, and 30 datapoints exhibit increasingly higher median error, with values of 2.0%, 3.6%, and 8.3%, respectively.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 7 compares mean predictive error for the physics-guided DNN models to the baseline DNN when independently tested with Test Dataset 2. Baseline mean error values are 33.2%, 34.9%, 35.8%, and 43.0% for dataset sizes of 261, 117, 60, and 30 points, respectively. Figure 8 shows the corresponding median predictive error and variability for the physics-guided DNN models when independently tested with Test Dataset 2. Baseline median error values are 30.4%, 30.8%, 32.0%, and 36.5% for dataset sizes of 261, 117, 60, and 30 points, respectively.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

One further evaluation of the methodology and models developed section 2 was conducted to assess their performance against various other regression approaches. Using the Matlab 2024a Regression Modeler application [25], the following models were trained validated with the same datasets of size 261, 117, 60, and 30:

• Linear Regression
• Cubic Support Vector Machine (SVM)
• Trilayered Neural Network

These models were subsequently tested with independent Test Dataset 1 (materials and dimensions similar to the training/validation set) and Test Dataset 2 (the printed wiring board material dissimilar to the training/validation set). Figure 9 shows mean prediction error for each of these three models when compared with mean error for the baseline DNN of figure 3(a) and the best performing physics-enhanced DNNs (figures 3(b)–(d)).

Zoom In Zoom Out Reset image size

One major item immediately stands out from the data. The first observation is that for data of similar dimensions and materials as the training set, insertion of physics into the DNNs results in no statistically-relevant improvement for models generated with 261, 117, and 60 datapoints. In fact, for these simulations, the t-Test shows that the only statistically-significant changes in predictive power are detrimental. This effect actually becomes more apparent for models generated using scarcer training data. For example, figure 5 demonstrates that, in general, inclusion of the weight term in the model (alone or in combination with modulus of rigidity and shear modulus) results in significantly worse predictive power than the simple non-enhanced DNN. Overall, therefore, no notable or consistent benefit is evident from the inclusion of physics-based parameters when testing with data similar to that used for training.

In contrast to independent Test Dataset 1, although less accurate overall, Test Dataset 2 sees moderate but clear improvement to natural frequency predictions for certain combinations of physics parameters and insertion layers covering all four training dataset sizes. In particular, solutions where flexural rigidity is included as a physical parameter almost universally demonstrate better accuracy of natural frequency prediction over the baseline DNN without physics. For example, the greatest increase in accuracy occurs with inclusion of weight and flexural modulus in DNN Layers 1 through 4, exhibiting a total predictive error reduction of about 15% (from 33.2% to 19.6%) when the training dataset contains 117 points.

Interestingly, while not as effective when inserted into the DNN with larger training datasets, the addition of shear modulus to the W-D combination exhibits similar performance to just weight and flexural modulus alone for the training dataset of size 60, and the W-D-G combination actually outperforms all others when the model is trained with only thirty 30 datapoints. Although not conclusive, this may indicate that certain combinations of physics-based parameters are more effective than others when applied to different training dataset sizes during model generation. In particular, those models with a greater number of terms creating parameter relationships appear to be effective in improving predictions.

As illustrated in figure 3, supplementation of physics-driven data was implemented with the neural network in several different layers, both singularly and simultaneously. Similar to results overall, figures 5–7 show no discernable improvement or detriment with any particular combination of layer augmentation for predictions made with using Test Dataset 1 (i.e., input parameters similar to the training dataset). However, when making predictions with Test Dataset 2, notable differences in accuracy are observed. In particular, when modulus of rigidity (D) and plate weight (W) are used as augmentative data parameters, the effect of injecting physics into Layer 1 or Layer 5 appears to be least impactful to accuracy. However, for these cases, the supplementation of a combination of inner layers or inner with Layer 5 provides a lower overall predictive error for the model. While further study is required to better understand this effect, it indicates that the supplementation of a DNN with physics-derived data may be most effective when conducted for multiple layers.

One aspect of this study not yet discussed in detail is the comparison of training and validation loss. During the course of the investigation, it is observed that as the training dataset sizes decrease, the training and validation loss (based on mean-squared error, as indicated in table 4) increasingly separate from one another, as is evident in the example shown in figure 10. Although expected behavior for the DNN where all hyperparameters are held constant, it likely indicates an increasing degree of overfitting. As such, the number of epochs used for this study is too large and reduction to something less than 100 would improve computational efficiency while not likely deteriorating accuracy. Overall, further investigation to simplify the DNN while optimizing hyperparameters would benefit the approach to reduce the overfitting effect.

Zoom In Zoom Out Reset image size

As described in section 3.3, the baseline deep neural network and corresponding physics-enhanced models developed for this study were compared with various other regression approaches, specifically linear regression, cubic SVM, and a trilayered neural network. As figure 9 demonstrates, the baseline DNN significantly outperformed Linear Regression and Cubic SVM for all training/validation dataset sizes when independently tested against Test Dataset 1 and Test Dataset 2. In contrast, the baseline DNN showed similar performance to the Trilayered DNN approach. However, in all comparisons the best-case physics-enhanced models were found to provide the lowest average percentage prediction error.