Opportunities and prejudices in synthetic experiments

Synthetic experiments, meaning experiments performed using models of reality, are used in a variety of contexts. In recent years, advances in computing capability have led to the development of modelling capability that replicates reality to such an extent that in some circumstances it is difficult to distinguish the model from reality. From a science or engineering perspective, computational analysis models are used to replicate or predict phenomena such as fluid flow or stress in a machine component, but the capability has also crossed over into the arts enabling the computer generated imagery (CGI) used in films, and the augmented reality (AR) of video games. This paper reviews the application of synthetic experiments to replace physical experiments and discusses some of the advantages and pitfalls of such an approach in the context of education. The trend for democratizing analysis is considered, and the dangers of misinterpreting results are discussed. A discussion of the issue of model over–simplification is given, and finally the use of synthetic experiments as part of a systematic scientific investigation is also considered.


Introduction
In many scientific contexts it is either impossible, or rather infeasible, to perform a physical experiment, or sufficient physical experiments, to be able to bring an investigation to a firm conclusion. In place of physical experiments, it is increasingly likely that computational models will replace or extend the investigation to provide access to insights that would not otherwise be available. Likewise, in engineering product design practice, it is becoming increasingly common to replace verification testing with modelling and rely on a smaller number of physical product validation tests. In both science and engineering the principle is the same: the model used is believed to contain all the pertinent physics principles that apply in the real world example: all the good practice principles that used for that type of modelling are scrupulously applied, for example, mesh size control, time step size, and other such methods for ensuring numerical stability and accuracy; and therefore the outcome of the model analysis is then believed to be correct within some precision tolerance band. Let us call such experiments synthetic experiments.
Clearly, the computer age provides a great opportunity for experimental study and engineering verification using models. Nevertheless, it is inevitable that even the most diligent researchers will approach modelling with a degree of prejudice, and it is important to recognise this and avoid complacency. For example: one might be diligent enough to select a suitable mesh, which is a mathematical analysis stability or convergence requirement; but fail to recognise some other special features, such as a requirement to represent some pertinent physics within the model. In the context of education, there is a growing trend towards teaching through simulations, and a reduction in laboratory practical classes. In the post-COVID-19 era, the move away from the laboratory will be accelerated. The fallacy is this: it is generally believed that the laboratories merely confirm theory and develop the students' laboratory skills. This is simply not the case: the fundamental role of laboratory experiment is to show that even the most meticulous experimental work will not replicate theory perfectly, because there will inevitably be other physics principles involved. It is in recognising this that enables a good researcher to avoid prejudice in modelling work.

The avoidance of prejudice
The principle of avoidance of prejudice is nothing new. Even as long ago as in Ancient Greece, Aristotle [1] listed thirteen fallacies, of which the eighth, Secundum quid et simpliciter concerns the confusion between that which is true within a particular context with that which is absolutely true. This is precisely to the point of modelling: models must be recognised as being simplifications of reality, and understood to be applicable only within a particular regime.
Somewhat more recently, in 1267, Roger Bacon [2] listed four hindrances, the second of which relates to entrenched custom and practice, while the fourth warns against hiding personal ignorance and giving an outward display of knowledge. In today's highly disciplined and controlled engineering industry, companies that design and manufacture safety critical machines and structures have detailed written procedures and processes for every aspect of the design, validation, manufacture and inspection of their products. The computational modelling work has become an increasingly significant part of the validation process, and is rightly tightly controlled by the procedural documentation; however, observing custom and practice alone is insufficient as a guard against a failure to capture all the relevant physics in the model. Furthermore, the beauty and apparent certitude of a rainbow coloured stress analysis model image provides misleading evidence for those with insufficient knowledge or experience.
In 1620, Francis Bacon [3] illustrated the problem of prejudice in science by means of his Idola mentis -false images of the mind. The first of these, Idola tribus, relates to prejudices shared by all humankind, and arising from our shared limitations. The metaphor Bacon used to explain this is that what we understand of reality is distorted or discoloured, by details not apparent to us, in the same way that an imperfect mirror can provide a distorted or discoloured image. Later in the second book of his Novum Organum, he describes the process of listing instances of a phenomena, and from this identifying causes: this seems to be the first scientific publication in which it is recognised that for any (experimental) observation the result might comprise effects from more than one cause, and that one can gain value from designing experiments in such a way as to be able to separate those effects.
In modern times, we are becoming increasingly aware of human factors [4] in science and engineering. Even diligent researchers will approach modelling with prejudice: we remember to make a suitable mesh, but do we fail to recognise some other special features? It is important to recognise this and avoid complacency.

Definitions and motivations
In the context of the present paper, the focus is on synthetic experiments, i.e. experiments made on the basis of computational (or possibly other) models. Where modelling can be considered a useful and faithful representation of reality, the use of models as proxies for experimental test specimens can be advantageous. This is precisely the role of models in component design verification and validation [5]: modelling enables design verification at lower cost and more quickly than used to be possible with the complex test programmes that were necessary before the advent of the sophisticated Finite Element Analysis (FEA) software available today. Nevertheless, the process of validation still IOP Publishing doi:10.1088/1757-899X/1060/1/012018 3 requires a final physical test proof to confirm the model predictions. The scepticism implied by the physical test process is justified: it is understood that a model is based on assumptions, and while those assumptions might be justifiable on the basis of prior experience within a similar regime (a process known as read-across), it is possible that for the particular regime of the new design, the model is less than adequate. Product development is an expensive process, the modelling work can be very useful in reducing the risk that the design would need to be changed, and this is a matter of importance to the finances of a company engaged in product design and manufacture. Failure of a product in-service, particularly if that failure were to be catastrophic and lead to loss of life, is a matter of far more importance, and has severe implications for reputation, as well as stringent financial penalties. It is for this reason that the final physical test programme is, and will continue to be, required.
In terms of science exploration, one might think that there is nothing more of novelty to do after the modelling capability is developed. Such a model might represent the requisite physical phenomena, and perform in a mathematically and computationally stable way. Nevertheless, from the point of view of research opportunities, this is only the beginning. Unless a model is extremely simplistic it cannot be explored in its entirety using mathematical analysis techniques alone, and thus must be explored by computational analysis with a myriad of choices of variable parameters and applied conditions (e.g. loads, temperatures and boundary conditions). Surprising results may ensue, and these might be described as emergent phenomena. If the model is an appropriate representation of reality, then the same phenomena might then be discoverable in reality. In this way, we might facilitate the process of real science discovery using the computational model as a tool.

Why do we make models?
Model-making is probably one of the very first tools of humanity. It allows us to stand outside our world and theorise about it. Understanding our world is the first step in being able to control it. Originally, more control meant a life more free from predators and reduced uncertainty about food supply. Over time this enabled the increasing ability to devise more complex physical tools, which has become a virtuous circle leading all the way through to modern civilisation.
But we are not gods: we know that models are limited. For example, the cycle of the day and the year are fixed with the precision of celestial mechanics and provide a basis for expectations about weather in particular seasons, but predictions of what the actual weather would be on a future day is limited, even with the powerful forecasting capability that is available in modern times. In the same way, engineering product design validation must make some assumptions about the loads and conditions that a product might be expected to withstand during a reasonable working life, but the individual product might be used in a wide variety of circumstances.

Confusing models with reality
Unfortunately, a time is approaching where many people become so inured with living with models, that the distinction between reality and the capacity of a model to represent reality becomes difficult to gauge. In the past, models were basic and therefore obviously approximate. In the model results of early FEA and computational fluid dynamics (CFD), the approximation was made obvious by the crudeness of the visualisation imagery. More recently, and to an ever-greater degree, FEA and CFD models look like the real problem. For example, in a 1D beam finite element model, many FEA results viewers will render the image using the geometrical information provided for the calculation beam bending stiffness. Similarly in CFD, the representation of particles entrained in flow can mislead understanding about how those particles interact with the flow, each other, or other physics not in the analysis. In modern films, computer-generated imagery (CGI) monsters look and move like real monsters, and the image manipulation algorithms which provide realistic textures and shadows demonstrate incredible capability. The cross-over between games and real applications using combined computational analysis and augmented reality (AG) [6] has already begun. These image presentation capabilities are gradually being applied in computational models, as a means to drawn in the non-expert, and democratize analysis [7]. While it can be helpful for the modelling specialist to convey to company senior management the significant aspects of the modelling work undertaken, it can also instil the erroneous expectation that such models provide an entirely faithful representation of reality, not merely a useful one.

Typical uses for computational modelling
Computational modelling methods are typically used when it is difficult to perform a physical experiment and when the can model provide insights that would not otherwise be available. Judicial use of modelling can lead to a reduction of the costs in engineering product design by reducing the number of physical tests needed for product validation. The same principle can be used in science discovery, where modelling can anticipate the physical test result, and help in the design of an appropriately sensitive physical test.
In using such a model there is a belief that it will contain all the pertinent physics principles. This belief can be tested by assessing candidate physical principles and estimating the extent of their influence. This is not a fool-proof method, but it is a common process in model-building to test a model before and after adding such an enhancement to establish whether the change is negligible or provides a modification to the result of the type and magnitude expected.
In modelling, the recognised and established good practice procedures are applied. In terms of computational mechanics modelling, this will include adequate checks on parameters such as mesh size, time step size, and other such methods for ensuring numerical stability and accuracy. After this then the model analysis can be considered repeatable meaning that there should be no errors arising from mathematical or computational inadequacy or instability, but this provides no assurances as to the adequacy of the physics represented in the model. Thus, it must be understood that however much we might increase reliance on computational modelling, and reduce the need for testing, there will remain a need for some testing.

Models in the context of STEM education and training
In the UK, most STEM degrees are accredited by professional bodies. An accredited degree programme provides prospective students an assurance that the content of the programme meets the specification set by the professional body that accredited it. The degree then forms part of education and training route toward professional qualifications. In almost all cases, laboratory based practical work forms a part of the specification [8,9] required for accreditation.
It is generally believed that student laboratory work serves to confirm theory, consolidate understanding, and to provide the student with laboratory skills. For a campus-based student, it is certainly true that time spent in the laboratory allows for personal interaction with fellow students and laboratory demonstrators, thereby providing the student with an opportunity to ask questions and improve understanding. The performance of prescribed laboratory experiments is somewhat artificial: the apparatus and test pieces are generally pre-designed, so that the student follows a standard recipe. Generally, the laboratory experiments are designed in such a way that the student is unlikely to break anything significant, and if an accident were to happen then the consequences would be minimal. Familiarity with handling apparatus, leads to confidence, develops hand skills, and might also serve to inspire the student to think beyond the particular experiment set.
The Open University, and other universities which offer a distance learning option, manage the laboratory experience requirement by holding some residential practical sessions. For many IOP Publishing doi:10.1088/1757-899X/1060/1/012018 5 of those students, travel can be a hindrance, and allocating blocks of time away from home might be difficult. For the simpler laboratory experiments, laboratory kits can be sent by post to perform at home, and more recently, the Open University has developed remotely operated physical laboratory facilities, which the students can access and drive over the internet [10]. In addition to these practical solutions, there is a growing trend of teaching practical skills using simulation tools and CGI models. This is driven in part by cost reduction, in part by convenience, and in recent months by COVID-19 and the need to reduce social interaction. After the pandemic, it is likely that this trend will be accelerated.
The advantages of such simulation-based models is that they are clean, safe, unbreakable, require minimal space to store or to operate, and that they behave consistently. Such advantages are also disadvantages in that they conceal reality: one important role of the laboratory practical is to show that, even with the most meticulous experimental practice, there will be occasions when experimental results will not replicate theory. There will always be some other physics principle involved, and sometimes that will be significant. A good researcher learns to recognise a when an experiment is a good differentiator of the significant effects and the negligible ones. Thus, although there are many advantages to teaching through simulated laboratory experiments, it is clear that students will require some real, physical laboratory experiences as part of their professional formation.

Models in the context of STEM research and product development
We live in a macroscopic world, which informs our common sense. Furthermore, our education conditions us to believe books not our own eyes, and we have become like rats running around in the intellectual model mazes that we have learned to navigate. The discipline boundaries have become comfort blankets protecting our minds from model regimes that clash, and while we are being exhorted to embrace interdisciplinary studies, there is little recognition of the additional intellectual burden of navigating through multiple discipline areas [11]. Here are some examples.

Material is "Bulk"
Classical physics and most of modern-day engineering is built on the notion of bulk materials. Almost everything that we interactive with daily involves Avogadro numbers of atoms, molecules or electrons. Most of the time most material (solid or liquid) behaves more like a bulk material than a surface, line or individual particle, and this reinforces our natural prejudice for classical thinking. While this might be expected to be the case for engineers, since they have relatively little exposure to modern physics and chemistry concepts, physical scientists can be just as trapped by classical or common sense thinking. The construction of mathematical models such as quantum mechanics and condensed matter provides prediction capability but does relatively little to shape human insight.
Two examples, spanning the last century, of such an intellectual struggle, include: in physics, the ever-continuing debate over what quantum mechanics really means; Physicists do not believe quantum mechanics because it explains the world, but because it predicts the outcome of experiments with almost miraculous accuracy John Horgan [12] and in engineering, the development of the theory of fracture mechanics from Griffiths crack surface energy argument through to the misunderstanding and mis-application of the theory which is still common and embodied in engineering standards even now, 100 years later [13].

The "Perfect" geometry
In performing engineering component stress analysis, the initial starting point is the specification of the component geometry. Generally, this is a Computer Aided Design (CAD) model which specifies the geometry through the definition of a number of reference points fixed in space, from which solid geometry is defined using splines and parametric curves. In practice, for some complex geometric shapes, the initial CAD models created by the design engineers are inadequate for use in stress analysis because of geometrical flaws such as mis-match of the edges of adjoining surfaces leading to tiny steps or curvature that are not intended features in the real component. Such geometry flaws must be inspected and healed.
The healed geometry can now be meshed: the term meshing really means subdividing the entire modelled component into sub-volumes, called elements, and these elements are defined by geometric points called nodes. The surface geometry of the component is thus defined by the surfaces of the elements which are in the outer layers of the mesh, and hence by the geometric points of the nodes and the interpolated element surface, which is usually linear or quadratic with respect to the nodes. Thus, the analysis geometry is similar to, but not the same as, the CAD geometry.
Even if we are prepared to accept these subtle differences, we should observe that this is still an idealised mathematical model of an object with a perfect geometry, comprised of perfect bulk material, and subjected to idealised loads. Such a component has no polycrystalline substructure, no unintended porosity or foreign material inclusions, nor does it have a surface texture. In such an idealised component, the stress state within the material is perfectly varying with no localised extremes, and is an object through which only bulk stress waves propagate.
One particular problem in mechanical analysis concerns the assumption of geometric linearity. For small loads and small deflections, the assumption is reasonable, but problems can arise when a structure is slender. Within the linearised model for a beam or shell, the mathematics does not allow for the eventuality of buckling under compressive loading. In order to model for buckling, one first has to recognise that buckling is possible, and then apply and appropriate mathematical model. In solving using FEA, it is usually necessary to introduce some artificial asymmetry to the geometry to precipitate the analysis to model the buckling mode. Since it is generally possible that a structure may have more than one buckling mode, the modification to the geometry has to be considered carefully, as otherwise it is possible that the lowest buckling mode would not be excited, and the load predicted for buckling initiation would be too high. In today's modern light-weight structures, there is increasing reliance on designs including slender features and the use of features like corrugations to inhibit buckling. Small geometrical variations, or in-service damage, can lead to substantial variation in the loads that can be supported by such structures.
A similar issue arises in modelling fluid flow. A fluid flowing along the idealised smooth surface of a component encounters nothing to trigger turbulence. In order to help the model represent observed reality, additional mathematical artefacts are added: hence the turbulence models used in CFD [14]. In both cases, the existing model is simplified, essential physical effects have been excluded, and then put back in using a different, and possibly rather arbitrary, mathematical form.

Dimensional reduction of the mathematical model -The Elastic Beam
The beam is a particular case of a model simplification process in which the real-world component is first modelled in mathematical form using a governing equation, and then the physical dimension of the problem is then reduced, based on some aspect of the geometry or symmetry of the component.
In such cases, there might be a ranking of plausible equations, based on how well they match a particular class of problem, and the mathematical elegance of the equation. For example, equation (1) shows the general equations for linear elasticity in 3D, equation (2) gives the Euler-Bernoulli representation for wave propagation through a 1D beam, and equation (3) shows the Timoshenko beam with a transverse shear correction term. Each equation is correct, but only in the particular context of its applicability.

Physically inadequate mathematical model -Navier-Stokes Equations
Until recently, most CFD packages were based on numerical solution of the Navier-Stokes equations for fluid dynamics. In the derivation of the Navier-Stokes equations [15], the following process took place: first all the classical physical effects were written down and assembled into an equation, then all the mathematically difficult terms were removed, and this was justified by a reduction of the context of applicability. In CFD packages, some of those difficult terms have been re-introduced, or they have been modified or replaced by empirical terms to make them more numerically amenable. An alternative approach is to develop from the fundamental physics principles of the mean free path of a particle travelling and interacting with other particles. This method is based on the Boltzmann equation [16]. Equivalence between this formulation and Navier-Stokes has been shown, for the context at which the latter is applicable. By basing numerical solution on the Boltzmann equation, a different numerical scheme is required -Lattice Boltzmann CFD, and this circumvents the various limitations of model building using CFD based on Navier-Stokes.

Results demonstrating model prejudice
The purpose of this section is to provide some illustrations of some large-scale effects arising from physical details that are usually excluded from analysis models. It is not immediately apparent that these details are significant, and their exclusion is frequently left unremarked. On the basis that these are significant effects arising from seemingly insignificant details, we might call these effects emergent.

Model of a test specimen with imperfect geometry included
The Saint-Venant principle is a guiding principle in stress analysis, which indicates that small geometric or stress distribution variations are a local phenomenon only, and their effect diminishes to insignificant at a sufficient distance from the disturbance.
The rationale for this study was the observation that for a perfect material, with uniform properties and perfectly smooth geometry, and when loaded to a level below the elastic limit can then be unloaded and return to the original state without damage or introducing a residual stress-strain state. On the other hand, it is well known that a real component, when cyclically loaded to a moderate fraction of the elastic limit, will eventually fail though fatigue. Generally the process of fatigue is characterised by test or empirical formulae derived from test data. In these studies [17,18,19], it is argued that fatigue must be a physical process, and as such should be possible to model directly. Figure 1 shows a simple test specimen, and sequence of section details leading to a local model of the gauge section. A cyclic load is applied to 90 % of the elastic limit at the gauge section, and the material properties include an isotropic strain-hardening plasticity model. Figure 2 shows von Mises stresses for three models: where the gauge section has surface roughness only; where the gauge section has sub-surface porosity only; and where the gauge section has both roughness and porosity. The interface between pale and dark blue represents the perfect model "nominal stress" at 90 % of the elastic limit stress. The green represents stresses above the elastic limit leading to plastic deformation. Note that for a model that does not include surface roughness nor sub-surface porosity, the nominal stress over the gauge section would be uniform, and always below the elastic limit. In such a model, the cyclic loading would be unable to generate damage accumulation, potentially leading to fatigue. Notice that the effects of surface roughness and porosity are different. Surface roughness introduces high stresses at deeper roughness troughs and tends to link troughs. The high stress region penetrates into the surface by about 1.5 times the peak to trough height in the roughness. The effect of porosity is to form diagonal bands extending from the surface and penetrating quite deeply into the sub-surface. The combined effect of roughness and porosity is significantly greater than either effect considered individually.

Low Reynolds number and steady laminar flow
In this example, the purpose is to illustrate that the simplistic view that low Reynolds number alone implies that the flow is steady and laminar can be misleading. Take the simple example IOP Publishing doi:10.1088/1757-899X/1060/1/012018 9 of plane Poiseuille flow [15] between two parallel walls, as illustrated in figure 3. In this case, the walls are perfectly smooth, and the analytical solution predicts a velocity distribution that is of parabolic form. This problem can be replicated as a simple 2D CFD model [20], the results of which are shown in figure 4(a). In this case, the walls were considered perfectly smooth with a no-slip condition. The inlet condition at the left-hand side is for a uniform velocity distribution equal to the mean flow velocity. The outlet condition at the right-hand side is that of zero pressure difference. Applying the usual due diligence one would evaluate the Reynolds number, and on confirming that it is sufficiently low for the fully developed flow to be laminar, one would then consider the size requirement for the lead-in region. A lead-in region is required because the inlet conditions represent the mean flow, and not the velocity profile of the fully developed flow. The velocity profile image shows that the fully developed flow develops within the first 5 % of the model geometry. Figures 4(b-f) show similar models, but where the lower wall has some irregularity. In the case of figure 4(b) the gap narrows, but away from the gap step changes, the velocity profile 10 and pressure profile are very similar to the perfectly smooth wall case. In figures 4(c-e), the gap varies over a shorter region, and is comprised of several inward or outward repeated steps which are closer together than the lead-in length scale. The flow treats these as if they were a single step, and the presence of the outward steps seems almost unimportant to the velocity profile. Figure 4(f) shows a continuously varying gap profile, with features that a comparable or somewhat larger than the lead-in length scale. In this case, the velocity profile is also continuously varying, but consistent with what might reasonably be expected.
In all the above cases, the walls were modelled as rigid, but in figure 4(g) the lower wall was modelled as a flexible elastic structure that can flex or vibrate at low amplitude as a response to the fluid pressure. The analysis made use of co-simulation methods, with the fluid flow modelled in CFD, in the same way as previously, and the wall modelled in FEA. The structure model is of a 1D shell and represented in the lower two figures as a strip of one element in height. Numerical stability of the combined analysis system was difficult to achieve, except for an elastic structure with high stiffness. In the result presented here, the upper two figures show the pressure and velocity profiles in the fluid.
The higher pressure at the inlet end extends over a larger distance. The velocity profile suggests that on average, across the whole stream, the flow is faster towards the right-hand end of the figure. This can only be explained by a pumping effect driven by the expansion and contraction of the fluid volume and resulting from the lower wall flexure. Thus, when the elasticity of the wall is taken into account, the fluid flow becomes pulsed. This is again, an emergent phenomenon, since the set-up of the model did not include any explicitly time varying parameters, yet a time varying result was obtained.

Discussion
The observations and analysis results presented in the previous sections have highlighted the importance of understanding the limitations of models, while recognising that the pertinent limitation is removed, models can provide new insights not previously observed, or recognised for their significance.
In the context of education and training, the use of simulations in place of real experiments can save time and cost, as well as greater convenience for both the institution and the student. The two significant disadvantages are, (i) that the student misses the interaction with other students or laboratory demonstrators, and (ii) that the results of the simulated experiment can be too consistent, and as such do not provide the student with the problem-solving experience of working out why his or her results might not have been the same as those of other students.
In the context of industry, the use of models as a verification strategy is highly useful and underpins and de-risks the product development process. At the end of the development process, the product is then certified for operational use based on real proof validation tests.
Finally, in the case of science research, the use of models is not just restricted to the development of new modelling methods and the validation of those against previously validated models or experiments. A fundamental science opportunity exists in using the models in unexpected ways, to challenge the accepted assumptions that were made in earlier times, when it was not possible to run more complex computational models.

Conclusions
Although models are not real, and are only representations of reality, they can be very effective in many usage scenarios. Models should be useful representations reality: to investigate that reality in new ways, the models can be used for synthetic experiments. When making use of the excellent modelling capabilities that are available today, one should also challenge the accepted ideas about what features or physics should be included in the model. This is an especially important lesson for students and should be explored through a combination of simulation IOP Publishing doi:10.1088/1757-899X/1060/1/012018 11 and physical laboratory practical classes. Finally, one should be aware that models might not always be sufficiently trustworthy, and validation through physical testing will continue to play an important role in engineering product development.