Statistical moments for simulation calibration with model-bridge

Computer simulations are actively used for analyzing complex phenomena, especially in fields where access to their real-world counterparts is not feasible, such as physics, chemistry, material science, and others. The key to executing successful simulations is making sure that the parameters of a simulator reflect real-world scenarios, a tedious and error-prone effort addressed through simulation calibration. Recently, several methods have been proposed to automatize this task by learning from previously calibrated simulations using a model-bridge paradigm: a complex simulation is replaced by a simpler surrogate model, which can then be bridged to the calibrated simulation parameters. However, designing the surrogate model is a non-trivial problem involving trade-offs between simplicity of representation, interpretability and calibration accuracy, as well as the complexity of the bridge model required to map the surrogate to calibrated parameters. Further, while effective, such approaches can be non-intuitive for practitioners due to their distance from the simulation. In this paper, we view a simulation as a distribution of output variables, which can be easily represented by statistical moments. This yields a very simple and interpretable surrogate that can be bridged to calibrated parameters with a simple linear regression. We show that our method outperforms previous approaches, in terms of calibration accuracy and time, through experiments on simulations of turbulent flow dynamics and synthetic signals.


Introduction
Computer simulations have become ubiquitous in the world of science and engineering; they are used to model, reproduce and examine the behavior of complex phenomena, allowing practitioners to gain valuable insight unobtainable otherwise [1,2].Their use is no longer limited to hard sciences like physics [3,4,5], chemistry [6,7], biology [8,9,10], genetics [11], climate science [12] and others [13,14,15], but is now commonly seen in industrial manufacturing [16,17], logistics and planning [18,19], as well as in various engineering [20,21] and social disciplines [22,23].However, to be used effectively, a simulation model need to be calibrated in order to accurately reflect its real-world counterpart.This task is not trivial as it usually requires domain expertise, experience with the simulator itself, and a great deal of trial-and-error.
The task of automatic simulation calibration [24] is, therefore, of interest to practitioners that, if solved and implemented effectively, will greatly reduce the time and effort required to calibrate a simulator.One of the latest proposed approaches for this task is the Kernel Approximated Bayesian Computation (KernelABC) [25,26], a kernel method which approximates the posterior distribution of simulation parameters based on samples from a prior distribution [27].From this posterior distribution it is then possible to predict simulation parameters using kernel 2 Figure 1.Overview of proposed simulation calibration method herding [28].However, establishing the connection between prior and posterior distributions of simulation parameters requires many actual simulation runs, which, in cases of complex simulations, requires significant processing power and time.
A recently introduced simulation calibration paradigm that avoids costly simulation runs, but is instead based on learning from past experience, is called model bridge.This idea was first explored by Kisamori et al. [29], with two key components: a surrogate model and a bridge model.In general, the surrogate model should be a simple approximation of the simulation model, meant to represent it in a convenient way.The parameters of the surrogate model serve as an input into the bridge model, which is then learned given a dataset of previous simulation runs.This model acts as a "bridge" from the intermediary "surrogate" representation to the actual simulation parameters, given alongside the training dataset.The model bridge approach depends on the existence of a dataset of previous simulations and is used under that assumption; however, the key design choice for the model bridge is the choice of the surrogate model, and, naturally, the bridge model that follows it.
Several surrogate models have been proposed, with different ideas on how should the relationship between the input and output of the simulation model be modeled.Kisamori et al. [29] propose Gaussian processes and Bayesian neural networks as non-parametric and parameteric surrogate models, respectively, bridged with kernel ridge regression.The output of this approach is a predicted distribution of simulation parameters for the target simulation model.An alternate approach based on geometric surrogate has been proposed [30]: first, principal curves are calculated to approximate simulation systems and then, vectors comprising the slopes and intercepts of its tangent lines are used as the final surrogate model.A simple regressor such as shallow neural network or a linear regression is used as a bridge model, and unlike the previous approach, the output of the calibration is a point estimate.
Though model bridge has shown success for the task of simulation calibration, the design choices it imposes can make it difficult to implement in real-world scenarios.Further, proposed surrogate models still tend to be fairly complex models, often requiring computation of sophisticated (and potentially unstable) algorithms such as neural network training or the bending algorithm for the principal curve [30].
In this paper, we propose a simulation calibration method directly influenced by the modelbridge paradigm, with the goal of creating a very simple surrogate model that can be bridged to simulation parameters via linear regression; the overview of our proposed method is illustrated on Figure 1.Our main idea is that the probability distribution of the outputs of the simulation can be represented with statistical moments such as mean and variance well enough for the task of simulation calibration.More broadly, we explore the use of a combination of notable moments, as well as central and standardized central moments as candidates for the surrogate model.Such a surrogate model would be very small in size, greatly reducing the computational time and complexity, while retaining and even increasing calibration accuracy.The surrogate model is bridged to simulation parameters using multivariate linear regression, allowing for calibration of multiple simulation parameters, while retaining the simplicity and low data requirements of the linear regression.
We evaluate out proposed method through simulation calibration experiments on physical simulations of turbulent flow dynamics, as well as synthetic signal simulations representative of common simulation behavior.We also investigate the relationship between the complexity of the proposed surrogate model and simulation calibration accuracy, as well as data requirements to successfully train the bridge model.The rest of paper is organized as follows: in Section 2 we introduce our proposed method, followed by its extensive experimental evaluation in Section 3. Finally, Section 4 concludes the paper.

Proposed method
In this section we formulate our problem and introduce the proposed method for simulation calibration by describing the surrogate model and bridge model parts.

Problem formulation
The objective is to predict the simulation parameter vector θ given simulation system data D. Here, simulation parameters θ ∈ R p are the underlying values that control the behaviors of the simulation system f (x, θ).Simulation system data is defined as a set D = {(x j , y j ) j } n j=1 of n pairs of corresponding inputs x j and outputs y j of the simulation for the given parameters θ.
A key assumption is that we have a training dataset of N calibrated simulation systems , where D i is the simulation system data corresponding to calibrated parameters θ i .Our proposed solution is divided into solving two problems: (1) estimate the input simulation system f in (x), represented by system data D in = {(x j , y j )} n j=1 , with a surrogate fin comprising statistical moments, and (2) perform regression from the moment surrogate fin to the parameter θ using a bridge model.

Surrogate model
Let q(x) and p(y) be the probability distributions of input x and output y comprising simulation system data D = {(x j , y j ) j } n j=1 .In our problem setting, q(x) is uniform.We seek to represent distribution p(y) with its notable statistical moments, which we approximate using Monte-Carlo method.By doing this, we create a simple and interpretable representation of the original simulation system, which can be directly bridged to the calibrated parameters.
We propose three main surrogate models: the mean-variance (MV) surrogate, the central moment (CM) surrogate, and the standardized central moment (SCM) surrogate.Further, we propose three additional surrogate models by combining MV surrogate with CM and SCM surrogates, aiming to best capture different information contained within individual moments.
2.2.1.Mean-variance surrogate.The mean-variance (MV) surrogate comprises of mean, standard deviation, variance, skewness and kurtosis.Excluding standard deviation, these features are considered notable moments of a distribution and are commonly used statistical tools.We postulate that when used together, a simple but powerful descriptor of simulation system data can be created and used for the task of simulation calibration.
The mean µ is the approximation of the expected value of a probability distribution, formally µ ≡ E[Y ], and corresponds to the first raw moment of said distribution; it is also the first notable moment of a distribution.Mean can also be considered the simplest representation of a distribution, and we choose it as a starting point for our surrogate.In our work we use arithmetical mean as follows: Standard deviation σ is often used alongside the mean in statistical analysis; it indicates the tendency of sample proximity to the mean.Though not considered a statistical moment, we include it for its representational ability and the fact that it is expressed in the same unit as original data, making it valuable for direct interpretation.Similarly, variance V ar(Y ) = σ 2 is also a measure of dispersion, and is the second central moment.As such, it is considered one of the notable moments.Standard deviation and variance are closely linked, and are defined as: V ar(Y The two final notable moments are skewness γ and kurtosis κ; they are also the third and fourth standardized central moment.Skewness measures the asymmetrical deviation of the distribution; it indicates the existence of distribution tails.On the other hand, kurtosis measures the extremity of distribution tails.Skewness and kurtosis are respectively defined as: The proposed mean-variance (MV) surrogate vector ζ mv is comprised of the above-mentioned notable moments, with standard deviation included.It is a simple but powerful representation of the simulation system, and is formally written as: 2.2.2.Higher-order moment surrogates.Central moments are moments of a probability distribution centered around its mean, and they describe the shape of the distribution.Because they are centered around the mean and not around zero, they more accurately represent distribution's shape.Further, central moments are also translation-invariant which may offer additional robustness to the variation of input values to the simulation.The central moment (CM) surrogate is comprised of first n central moments.The main idea is that the shape of any distribution can be uniquely described with a collection of all moments; therefore, we postulate that a distribution can be sufficiently accurately approximated by a collection of first n central moments.The n-th central moment µ n is defined as: Thus, the proposed central moment (CM) surrogate vector ζ cm is defined as: Standardized central moments are central moments normalized by the power of standard deviation of the probability distribution.In addition to being able to describe the shape of said probability distribution, due to normalization they are scale-invariant, allowing for proper comparison of shapes of different distributions.In the context of our proposed method, this quality is interesting due to its ability to reduce different probability distributions to an underlying shape, which greatly helps training of the bridge model for final parameter calibration.
The standardized central moment (SCM) surrogate is comprised of first n standardized central moments, where one such moment s n of n-th degree is defined as: Thus, the proposed standardized central moment (SCM) surrogate vector ζ scm is defined as: 2.2.3.Mixed moment surrogate.Previously defined surrogate models can be mixed to better capture varied information they represent.This is especially useful for the bridging part of our proposed method, where richer expressiveness of the surrogate model provides better features for regression to the simulation parameters.
We propose three mixed moment surrogates with differing levels of complexity.The first is a mix of the mean-variance surrogate and the central moment surrogate, denoted as ζ mv+cm .The second is a mix of the mean-variance surrogate and the standardized central moment surrogate, denoted as ζ mv+scm .Finally, we mix all three basic surrogate models, and denote this as ζ mix .
Formally, we define them respectively as follows: With this, we have defined our surrogate models to represent simulation systems f (x, θ) through their data D = {(x j , y j ) j } n j=1 as a surrogate vector ζ.

Bridge model
The main goal of the bridge model is to enable a regression between the surrogate representation and simulation parameters, effectively completing the task of simulation calibration.In principle, any regression algorithm can be used as a bridge model for the surrogates defined in previous section; however, in this paper we use multivariate linear regression as a simple and effective choice.
Given a dataset of calibrated simulation systems In the simplest case, only one simulation parameter needs to be calibrated, in which case θ ∈ R. For this special case, we use multiple linear regression model.This model factors multiple independent variables paired with their unknown parameters, which are fitted using the least squares method.In context of our problem, we define multiple linear regression model as: where θ i is the calibrated simulation parameter, ζ i is the surrogate model of system data D i , β = [β 0 , β 1 , . . ., β n ] are parameters of the linear model, and ϵ i is the error term.Using matrix notation, the previous equation can be written simply, for all N simulation systems, as: Fitting the parameters β of the linear model is done using ordinary least squares method, which can be solved analytically as: If there are more than one parameters to be calibrated, meaning θ ∈ R p , p > 1, we instead use multivariate linear regression, which is essentially a sequence of multiple linear regression models; one for each simulation parameter in vector θ.
An input simulation system represented by its data D in = {(x j , y j )} n j=1 can be calibrated by representing D in with the chosen surrogate ζ in , plugging the surrogate into the bridge model and predicting the simulation parameters θ.

Experiments and discussion
In this section we evaluate our proposed method for simulation calibration through experiments on four simulation datasets:: turbulent flow simulation dataset from OpenFOAM ® , and synthetic simulation datasets based on three families of functions.First, we introduce the datasets.We then show the general behavior of statistical moments in our datasets and argue their viability as surrogate models.Next, we perform simulation calibration experiments using our proposed method, and compare them to previous simulation calibration methods.Finally, we evaluate the robustness of the proposed method to training data size and hyperparameter choice.

Turbulent flow dataset
OpenFOAM ® [31,32] is an open-source tool for computational fluid dynamics (CFD) which mostly relies on finite element methods (FEM) to simulate the flow of fluids in various geometrical environments.In our work, we use the simple case of a two-dimensional square cavity, in which the initial state of a fluid is stationary.The fluid on top of the cavity starts moving with initial velocity x, and over time the fluid in the cavity moves, ultimately reaching the state of turbulent flow.We measure velocity y in the center of the cavity after 5 seconds have elapsed.The fluid in the cavity is characterized by its Reynold's number Re.
To collect our dataset, we vary initial velocity x in from 0.65 m/s to 1.0 m/s with a step of 0.1, and Reynold's number Re from 10, 000 to 70, 000 with a step of 100.This approach yields 600 simulation systems, one obtained for each fluid characterized by its Re using the varied x and collecting the y.We create three synthetic datasets in order to evaluate the proposed method in a variety of typical simulation system behaviors involving trade-offs between input and output.In general, given simulation parameter vector θ ⊤ = {θ 1 , θ 2 , θ 3 } ∈ R 3 , data is generated as y = f (x, θ) + ϵ, where x is the input of the simulation system, y is the output and ϵ is random noise drawn from N (0, σ 2 s ).We sample the parameter vector θ as θ = τ h + ω, where h is a constant weight, ω ∈ R 3 is a random vector with entries sampled from N (0, σ 2 p ), and τ is a uniform random value in the range [−1, 1]; these could be considered meta-parameters for generating our datasets.Concretely, individual datasets are generated based on the underlying behavior of three families of functions: logistic, polynomial and sinusoidal.For each of the three datasets we generate N = 100 simulation systems with n = 100 data points per system.
In the case of the logistic dataset, formally the simulation system is given as y = a/(1 + e (−b(x−c)) .Parameters θ ⊤ = {θ 1 , θ 2 , θ 3 } correspond to {a, b, c}, which are amplitude, steepness and midpoint of the logistic function, respectively.We fix the midpoint c at 1, making this a dataset where we estimate two parameters -θ 1 and θ 2 .
These simulations are less demanding in terms of computational resources required, but potentially yield more varied output with respect to input.Additionally, synthetic simulations are governed by multiple simulation parameters, in contrast to only one parameter for the cavity turbulent flow dataset.

Visualizations of moments
To give some intuition for their choice as surrogate models, we show the behavior of various statistical moments with respects to given datasets on Figures 4 and 5.
In Figure 4 we plot the mean values for all simulations of turbulent flow in the cavity, along with their standard deviation and variance and two notable moments -skewness and kurtosis.It can be clearly seen that in most cases, the mean provides a one-to-one relationship with the Reynolds number, the parameter to be calibrated.This gives us some confidence for using the mean of output values as a surrogate.Intuitively, this is exactly what we are looking for -we aim to find an appropriate surrogate such that it is relatively 'easy' to create a direct mapping (i.e.function) to the parameter to be calibrated.From a mathematical perspective, the plot shows that the mean could be considered a bijective function of the Reynolds number.
On the other hand, standard deviation is not as descriptive, especially in the Reynolds range of [20000, 70000], while variance remains near constant across the entire range due to values very close to 0. Similarly to mean, skewness shows a possible one-to-one relationship with Reynolds number, revealing another strong candidate for the surrogate model.Finally, kurtosis may be a strong surrogate in the range of [10000, 35000], after which it exhibits some problems.
On Figure 5 we show the same plots for the synthetic datasets.In all three cases, mean appears as a decent surrogate candidate, followed by the standard deviation.For logistic simulations variance seems like the best choice, while for polynomial and sinewave simulations it falls slightly behind mean and standard deviation.Behavior of skewness and kurtosis varies between the three datasets: it tends towards a constant value for logistic, seems very noisy for polynomial, and exhibits sinusoidal behavior in case of sinewave data.This indicates that choosing the appropriate surrogate for a given dataset is essential, as including non-informative moments might lead to degraded calibration performance.However, in all cases mean seems like a strong candidate.
Variance, as the second degree central moment, has shown potential for logistic data.Likewise, skewness and kurtosis, third and fourth standardized central moments seemingly work well for cavity data.This motivates us to examine the behavior of higher degree central and standardized central moments, to potentially uncover better surrogate candidates.Figure 6 illustrates first five moments of both types for all four available datasets.
For cavity data, it seems that the second central moment contains most information; however, it does not exhibit a nice one-to-one relationship with the Reynolds number which characterizes a good surrogate candidate.On the other hand, standardized central moments, besides the first and the second which are 0 and 1, respectively, are able to capture different degrees of curvature information, especially in the lower ranges of the Reynolds number; this gives some motivation for their use as surrogates.
However, for synthetic datasets it seems that non-standardized central moments offer better surrogate candidates.This might be due to the fact that standardization removes linear scaling  Both types of moments are made around the mean; and are therefore translation invariant.In addition, standardized moments are scale invariant and don't depend on the linear change of scale.These two qualities bring a type of robustness to a surrogate composed of central moments as they would be able to generalize the underlying nature of data regardless of scale or translation.
It is worth noting that in some cases, not using the mean removes a key piece of context which might significantly degrade the performance.Therefore, we claim it is prudent to use notable moments such as mean and variance in conjunction with central and standardized central, which are better able to describe the shape of the data.

Baseline comparison
In this experiment, we compare our proposed method to several baseline methods for the task of simulation calibration.Calibration performance is evaluated in terms of regularized mean squared error (RMSE), a common metric for regression problems.Additionally, we measure training and evaluation times for all methods to compare computational and time complexity via empirical means.
For baseline methods, we use KernelABC, a classic simulation calibration method, and tangent slope-intercept (TSI), a model-bridge approach with a geometrical surrogate.KernelABC estimates the posterior of θ; hyperparameters of KernelABC include number of samples from prior B, Gram matrix regularization parameter 1e−6, and bandwith parameter σ y of the Gaussian kernel function.In our experiments, we set B = 1000 and σ y = 1.As KernelABC outputs a posterior distribution, we take its expected value as the calibrated parameter.This is done in order to facilitate comparison with our method, because our method outputs a single point estimate instead of a distribution.TSI is also a single point estimate calibration method.Its main hyperparameters is the number of surrogate points for the principal curve, which we set at m = 34 for the cavity dataset, and m = 98 for synthetic datasets.
For our proposed method, we have five different choices of surrogates.The first is the simplest Simulation calibration results can be seen in Table 1.The results showcase the quality of proposed method compared to baselines in terms of calibration accuracy.Depending on the choice of surrogates and the target datasets, results vary.On turbulent flow data, MV seems a relatively poor choice for the surrogate, falling behind both KernelABC and TSI.This is most likely due to fairly 'flat' graphs of notable moments (see Figure 4), which lead to difficulty in accurate calibration.CM is also unable to capture the cavity data; this is probably because the only significant moment is the second one (corresponding to variance), while the higher order central moments are very close to zero and provide no information.On the other hand, SCM seemingly best captures the essence of the data, and yields an order of magnitude better results than previous methods, at 5.90 compared to 134.83 and 38.52 of KernelABC and TSI, respectively.However, the importance of mean should not be completely disregarded; when combining MV and SCM surrogates, best calibration performance is achieved.In this case, mean and standard deviation provide important context, while higher order standardized central moments describe the shape of the underlying function.
For synthetic datasets, the simplest proposed surrogate, MV, achieves orders of magnitude better results than baselines and almost perfectly calibrates the parameters.These results are somewhat surprising considering the simplicity of the MV surrogate; they show that there is a lot of room for improvement in model-bridge methods when it comes to compactness.Higher order CM and SCM yield competitive results, though they offer no significant improvements compared to using only the notable moments contained within MV.For synthetic data, as well as for the cavity data, combining the context provided by MV and shape information of higher order CM and SCM achieves the best calibration results.
Calibration time may be an important factor when choosing simulation calibration methods.We measure training times for methods (where applicable) and evaluation (calibration) times and report them in Tables 2 and 3, respectively.Note the key difference between standard simulation calibration approaches (KernelABC) and model-bridge approaches.KernelABC has no training stage, because it is an online calibration method; starting from an initial set of parameter, simulation is run multiple times and the parameters are adjusted after each run until convergence.This possibly leads to extremely long calibration times that directly depend on the length of the actual simulation.In contrast, model bridge approaches assume a dataset of previous simulations from which a machine learning model is constructed within a reasonable time frame, followed by extremely fast calibration times.The results show that our proposed method is orders of magnitude faster both than KernelABC and TSI, a competing model-bridge approach.For the cavity dataset, training time of TSI is 2926.52 [m/s], while MV requires only 3.79 [m/s].CM and SCM training times vary depending on the number of higher order moments selected; however, a linear increase in time can be expected for higher number of moments, and the training time would remain within the same order of magnitude.Choosing whether to use MV, CM, SCM or a combination of these as a surrogate for our proposed approach is a trade-off between time constraints and desired calibration performance: MV is the simplest and fastest, but may not achieve the best results; on the other hand, combining MV, CM and SCM yields the best performance but at the cost of additional processing time.However, results suggest that perhaps, in practice, the difference in calibration time is negligible and choosing a combination of MV, CM and SCM is a safe choice in most use cases.

Method robustness
In this experiment, we investigate how much data is required to train the bridge model which would connect the representation made by our proposed surrogates to the actual simulation parameters.We aim to show that with our proposed method it is possible to successfully calibrate simulation parameters without requiring significant amounts of prior simulations, thus greatly reducing the overhead of the entire task of simulation calibration, one of the key points of the model-bridge approach in general.

Figure 7. General dataset robustness
Figure 7 shows the change in RMSE for the three types of surrogates: MV, CM and SCM, on all four available datasets.We change the number of available training samples from 1 to 400 and from 1 to 50, while fixing the number of testing samples at 200 and 50, for Cavity and Synthetic datasets, respectively.Overall, the plots indicate that the proposed surrogates require very little training data to achieve good calibration results.On the cavity dataset, all three methods require less than 30 samples (out of 600 total) for the RMSE to stabilize, while on the synthetic datasets usually less than 10 samples (out of 100 total) is enough.
The MV surrogate exhibits sharp and consistent drops in error as more training data is introduced.To achieve good error rates on Cavity and Logistic datasets, MV requires the least amount of data, out of total data available, at around 15 and 4 samples, respectively.On Polynomial and Sinewave datasets, MV requires a a slightly higher number of training samples, but very good error rates are already achievable with only 10 to 15 samples.The Sinewave datasets contains the most complex data, therefore more training samples are required to best capture such simulation systems.
CM and SCM surrogates show similar trends on all datasets.On the Cavity dataset, 30 samples guarantee very good error rates, and increasing the number of training samples does not greatly improve performance.On all synthetic datasets, the error rate behaves stochastically under low sample conditions; however, having at least 15 training samples guarantees lowest error rates.When implementing a machine learning model to address a problem, a major design decision is the choice of hyperparameters.In the case of our proposed surrogates, the only adjustable hyperparameter is the number of moments used to construct the surrogate.A surrogate using more higher order moments will, in principle, be able to express increasingly complex simulations, but might be prone to overfitting if there is not enough training data.Further, higher order moments may start introducing noise, thus reducing the quality of the simulation calibration.In Figure 8, we examine the calibration error rate while varying the number of moments for MV, CM and SCM surrogates, on all available datasets.
Experiments on the Cavity dataset are a good example of how more complex models are better able to capture the underlying nature of data.For every surrogate, increasing the number of moments results in better calibration error rate.MV requires only three notable moments (mean, standard deviation, and variance), while adding kurtosis and skewness does not improve results drastically.CM and SCM start achieving good accuracy at around 5 moments, and converge at 7 and 9 moments, respectively.
On synthetic datasets it is possible to see the overfitting effect of complex models, especially in the case of SCM.For all three synthetic dataset, SCM achieves increasingly worse error rates when using more than 10 moments.CM exhibits similar behavior for Logistic and Polynomial datasets, where using more than 6 and 11 moments, respectively, results in worse calibration performance.Further, centralized and standardized centralized moments essentially describe the shape of the curve of the underlying data; very likely logistic and polynomial data of second degree does not require moments of higher order to effectively describe their shape; in these cases such moments most likely are capturing only the noise, leading to poorer performance.
In the case of Sinewave data however, CM and SCM have opposing behaviors: CM achieves better performance with increasing number of moments, while SCM does not.This is likely related to the fact that one of the parameters in Sinewave data is amplitude, which is ignored in SCM as it does not take scale into account.
Note that even if increasing the number of higher order moments would guarantee increased performance, it is computationally unfeasible to do so; at a certain point, computing the higher order moment will result in division with zero.In our experiments, it was not possible to obtain moments of a degree higher than 25 in most datasets.In general, hyperparameter selection should be done for each individual dataset either through trial-and-error or by relying on using train, validation and test splits, or, in cases of low sample datasets, cross-validation approaches.
In general, machine learning models require more training data as their complexity increases.The complexity of our surrogates is directly proportional to the number of moments used, as it corresponds to the number of features in the linear regression bridge model.Therefore, in addition to previous experimental setup, we vary the number of moments and check its effect on the error rate for all three surrogates on all available datasets, and plot the results on Figure 9. On the first glance, it seems that to achieve good error rates, surrogates using more moments do not require additional training data compared to surrogates using less moments.However, it should be noted that the error rates are much higher for a complex model with low training samples, but quickly drop with increased number of training samples, a trend which can be observed for all three surrogates.In contrast, simple models make less extreme errors with few training samples, but even with additional training data, they cannot reach the performance of higher complexity models.
These results show that, in practice, unless under extremely restrictive data conditions, choosing MV, CM or SCM surrogates with more moments will yield better results.This is

Conclusions
In this paper, we proposed a new simulation calibration method based on the model-bridge paradigm.We use statistical moments such as mean, variance, central and standardized central moments and their combinations to create a simple surrogate model to represent simulation data.This representation is bridged to the actual simulation parameters using a multivariate linear regression, allowing for efficient and accurate calibration.We have extensively evaluated our method on OpenFOAM ® fluid dynamics simulation data and three synthetic signal simulation datasets.Our experiments show that the proposed method outperforms previous related simulation calibration methods, while achieving a significant increase in calibration speed.

Figure 2
shows examples of simulation systems with different Reynolds numbers.

Figure 2 .Figure 3 .
Figure 2. Change of fluid behavior based on Reynolds number

Figure 6 .
Figure 6.Central and standardized central moments of all datasets

Figure 9 .
Figure 9.Effect of the number of moments on data requirements

Table 1 .
and is comprised of mean, standard deviation, and notable moments (variance, skewness and kurtosis), further termed the mean-variance (MV) surrogate.Next are two surrogates based on central moments (CM) and standardized central moments (SCM), respectively.Finally, we combine the MV surrogate with CM and SCM surrogates.CM and SCM surrogates have one adjustable hyperparameter: the number of moments.This number depends on the dataset and is best chosen experimentally.RMSEs of all methods. 11

Table 2 .
Train time of all methods