Hierarchical structure analysis in verification and validation of aerodynamic numerical simulation

Verification and validation is a basic way to ensure the credibility of numerical simulation. The realization of verification and validation of aerodynamic numerical simulation requires scientific and rigorous processes. At present, long-term research on verification and validation at home and abroad has summarized the whole verification and validation process into four stages: problem analysis, verification, validation and credibility evaluation. In each stage, it is of great significance to use the hierarchical structure to sort out key information and clarify tasks. This paper summarizes the application of hierarchical structure in the verification and validation process, and gives examples of aerodynamic numerical simulation with application value.


Introduction
With the development of physical models, numerical algorithms and computer technology, Computational Fluid Dynamics (CFD) has played an increasingly important role in aerospace, energy power, transportation and other fields.In the CFD 2030 vision released by NASA, requirements and expectations are put forward for the credibility of CFD [1].In order to ensure the credibility of CFD, verification and validation has become a necessary way.In the past 20 years, American Institute of Aeronautics and Astronautics (AIAA) and American Society of Mechanical Engineers (ASME) have published a series of standards that have greatly promoted the scientific development of verification and validation [2,3].At present, the National Numerical Wind Tunnel (NNW) Project has established the process and methods for the verification and validation of CFD field through the construction of the numerical simulation verification and validation system [4,5].The process of CFD verification and validation is divided into four stages: planning stage, verification stage, validation stage and credibility evaluation stage.
In the methodology of CFD verification and validation, Analytical Hierarchy Process (AHP) is an important method and means, which is strengthened in standards of AIAA [2] and ASME [3].At present, AHP has been widely applied to all stages of verification and validation, and corresponding changes have been made for specific tasks in different stages.Chapter 2 of this paper will specifically discuss the hierarchical structure in Phenomena Identification and Ranking Table (PIRT) analysis, verification, validation and credibility evaluation.Chapter 3 will give a summary.

Verification and validation process
CFD verification and validation process includes four stages and several nodes, which ensures the orderly implementation of verification and validation activities, as shown in Figure 1.

Hierarchy in PIRT process
The task of PIRT analysis in the process of verification and validation is to create a PIRT table to identify the key phenomena, key processes and key factors in the software, model or simulation.Usually, the PIRT process has five steps: organize a PIRT team, define the objectives of the PIRT process, specify the environment and scenario, identify plausible physical phenomena, and establish a PIRT [6].Hierarchical structure is required in multiple steps of PIRT analysis.
(1) Define the environments and scenarios of PIRT Environment refers to the external condition or situation that the system may be exposed to, while scenario refers to the possible events and sequence of events that the system may be exposed to in a specific environment.Usually, during the PIRT process, only one environment is specified for analysis: normal, abnormal, or hostile.The designated scenario should be based on the technical expertise and academic background of PIRT members.The process of specifying the environment and scenario should specify key parameters, and if possible, include the range of parameters [6]. Figure 2 shows the hierarchical structure of environment -scenario -SRQ (System response quantities) tree.(2) Establish hierarchical structure of complex flow The hierarchical structure of complex flow has been emphasized many times in verification and validation standards and documents.It is generally divided into four tiers, from top to bottom: complete system tier, subsystem tier, benchmark case tier and unit problem tier.The focus shifts from complete system to single physical phenomenon.Figure 3 shows an example of the hierarchical structure of the transport aircraft from the perspective of application problems.The system tier consists of complete hardware, geometry, and complete physical and chemical characteristics, such as transport half-mode.The subsystem tier consists of the functional system hardware, relatively independent large block geometry, and some related physical and chemical characteristics, such as the wing subsystem; The benchmark tier consists of specific flow characteristics and simplified geometry, such as attached flow; The unit problem tier consists of very simple geometric structure and flow characteristics, such as fuselage attachment flow.For more complex flow systems, tiers can be added between the complete system tier and the subsystem tier.

Hierarchy in verification process
The purpose of verification is to prove that there are no errors in the software operation logic, algorithm implementation and solution process, estimate the numerical error in the simulation process, ensure the correctness of the input and post-processing process, and ensure that the numerical solution is physically reasonable.Verification is generally divided into code verification and solution verification.Code verification is generally divided into software quality assurance and numerical algorithm verification.The main activity of solution verification is numerical error estimation.Figure 4 shows the hierarchical task structure of verification.The second tier consists of two parts, numerical algorithm verification and numerical error estimation.For numerical algorithm verification, the third tier includes 6 indices, mainly choosing simple or accurate solution tests.For details, please refer to unit problems in Figure 3.The numerical errors consist of round-off error, iterative error, discretization error and statistical sampling error, among which iterative error and discretization error are mainly considered in general tests.

Hierarchy in validation process
The purpose of validation is to quantify the difference between the calculation results and the highly credible reference data, evaluate the adequacy of numerical simulation, and modify the model if necessary.Validation includes validation experiment and validation simulation, and the consistency of experimental results and calculated results is quantified by validation metric.Validation hierarchy has been discussed in the literature mostly from the perspective of complex flow, and the hierarchical structure is similar to Figure 3.The objects of verification and validation can be models, software or numerical simulation.The validation hierarchy is not exactly consistent for different objects, but the overall structure is similar.A brief hydrodynamic model validation hierarchy is used as an example in Figure 5.The subsystem tier consists of three parts, and the benchmark tier corresponds to specific forms, such as turbulence model corresponding to SST model and SA model.Three specific flow problems are presented in the unit problem tier, each of which points to more than one benchmark tier block diagram, meaning that each unit problem can test several benchmark tier models.For example, SA model, SST model, ideal gas model and Newtonian fluid model can be test by calculating the turbulent plate case.For numerical simulation validation, the hierarchical structure is established based on flow characteristics, so as to simplify complex flow problems and reduce the simulation difficulty of benchmark tier and unit problem tier, as show in Figure 3.The difficulty in establishing a hierarchy is that from the subsystem tier to the benchmark tier, this process requires the identification of a series of typical physical characteristics that are often only qualified by experienced experts.For software validation, hierarchy is established based on software function.This issue can be converted into software credibility evaluation, which will be discussed in the next section.The validation hierarchy guides both validation simulations and validation experiments.The numerical simulation and experiment should maintain communication, but also maintain necessary independence.

Hierarchy in credibility evaluation process
The purpose of the credibility evaluation process is to evaluate the credibility of the software, which is based on the verification and validation of the software.The core task of credibility evaluation is to establish credibility evaluation indicator system.The evaluation indicator is closely related to the simulation capability of the software, and the complex flow is decomposed through the setting of hierarchical indicators.To achieve the credibility evaluation of software, we should grasp the credible attributes and feature flow of software, and use standard examples and credible data.Figure 6 shows a default credibility evaluation indicator system of steady aerodynamic software.In the figure, the first level indicator includes two indicators, namely basic flow indicator and engineering simulation indicator.The second level indicators are classified from flow velocity and aspect ratio respectively.The fourth tier of the indicator system is the evaluation cases tier, some benchmark cases are listed to test the capability of the software.

Conclusion
This paper discussed the hierarchical structure of each stage in the verification and validation activities of aerodynamic numerical simulation, and analyzed the position and function of hierarchical structure in the verification and validation activities.Final conclusions: 1) The establishment of the hierarchy precedes the beginning of verification and validation activities.
2) The hierarchy of verification and validation of software, models and simulations should be adjusted accordingly.
3) The hierarchical structure facilitates the analysis of complex flow characteristics and the implementation of validation experiments and validation simulations.
4) Establish hierarchical structure requires expertise and often require iteration.

Figure 1 .
Figure 1.Verification and validation process

Figure 3 .
Figure 3. Example of hierarchy of complex flow system

Figure 4 .
Figure 4. Hierarchy of verification indicator system

Figure 5 .
Figure 5. Example of fluid mechanics model validation hierarchy

Figure 6 .
Figure 6.Example of credibility evaluation indicator system of steady aerodynamic software