Research on the Multi-probability of a Nuclear Reactor Core Digital Twins in the HeTu platform

Digital twin (DT) technology plays a fundamental role in innovation in the energy field. At present, research of nuclear reactor DT is mainly focused on methodologies of multi-disciplinary, multi-physical, and multi-scale coupling simulation. However, the lack of research on multi-probability seems to limit the DT application. Multi-probabilities are widely found in reactor operation, such as process measurement uncertainty, equipment degradation uncertainty, theoretical model uncertainty, and so on. How to realize the DT of the reactor system based on the multi-probability principle to achieve the real-time operation state and the remaining useful life estimation, and to realize the autonomous operation control under the uncertainties, are the current hotspot and difficult issues. The solutions of a reactor core DT platform HeTu are discussed in this paper for probability estimation and probability control based on probability programming methodologies. The results show that Bayesian estimation and model prediction control methods proposed in HeTu could obtain satisfactory results for practical application.


Introduction
Digital and intelligent upgrading of the energy industry plays an important role in the construction of a modern energy system.This role is highlighted again in the 14th Five-Year Plan, from 2021 to 2025, on energy technology innovation to propel green growth and digital transformation of the energy sector.This five-year plan was focused on developing new technologies to render a more efficient, cost-effective, and reliable supply of renewables, which was released in April 2022.In this plan, an overall blueprint for digitalization and intelligentization of the nuclear power plant was presented, which would achieve key technology breakthroughs in 2023, achieve demonstration applications in 2025, and make application and promotion come true before 2030.Digital twins (DT) [1] technology is becoming a new foundation technology for the digital and intelligent upgrading of the energy system.Meanwhile, an overall blueprint of advanced nuclear energy systems based on a lead-based fast reactor was presented, which focuses on technical systems established by 2023 and commercial applications after 2030.Therefore, how to support and promote the development of advanced nuclear energy system technology and realize unprecedented technology innovation by using digital twins (DT) technology, is a pivotal problem needed to be solved.
To support the full-life period management of reactor equipment, a digital twin platform (named HeTu) prototype was developed in our work to cover all stages of advanced reactors, including the R&D, design, construction, operation, and maintenance, decommissioning.The HeTu platform would provide functions of operation based on real-time measurement information, such as equipment fault diagnosis, remaining useful life prediction, intelligent autonomous control, and remote management.As shown in Figure 1, HeTu is the DT platform containing the fundamental model and algorithm of monitoring, prediction, diagnosis, and autonomous control of a lead-bismuth fast reactor system, while Luoshu is an augmented reality-based application platform for on-site operation and maintenance, and CangQiong is a virtual reality-based application platform for design verification and training purpose.Digital twin makes full use of the physical model, sensor update, operation history, and other data, and integrates the multi-disciplinary, multi-physical, multi-scale, multi-probability simulation process to realize the mapping in the virtual space to reflect the whole life period process of the corresponding real reactor.There has been abundant research for multi-disciplinary, multi-physical [2] , multi-scale coupling simulation [3] .However, there are relatively few studies about multi-probability problems.The algorithm of specific probabilistic estimation and control in the HeTu platform will be proposed in this paper.

Multi-probabilistic perspective
Stochastic phenomena exist widely in nature.According to the quantum mechanical theory describing the behavior of physics at the microscopic scale, nature is random at the basic level.For example, due to the influence of noise, the measurement of any physical quantity of the process parameters is uncertain, and the degree of the uncertainty is usually expressed in the 95% -95% confidence interval of the probability distribution.The failure or remaining useful life of the equipment also follows the probability distribution and may change over time.Specifically, in the field of nuclear reactor neutron science, various reactions (such as   , n f ,   , n  ) of neutrons and materials also occur with a certain reaction cross section (probability), so the state of the reactor would also be uncertain.
In the perspective of probability theory, there is a difference between certainty (frequency perspective) and confidence (Bayesian perspective).From a certain perspective, the state of a reactor system is a knowable and fixed constant at a certain moment.Since the parameters themselves are determined, fluctuations in the direct or indirect measurement of these parameters do not arise from uncertainty in the parameters themselves but arise from interference caused by the limited detection means or the limited number of observations in time or space.Therefore, the exact values of the unknown reactor state parameters can be effectively inferred based on these inaccurate or incomplete detection data, but with estimation error.However, from the perspective of confidence, the operation state of the reactor is uncertain in nature, and any state of the operation reactor has a subjective degree of credibility.This level of credibility is based on the existing measurement and theoretical cognition of the state.
So far, the monitoring, control, and protection technologies of existing reactor systems are mainly based on the certainty perspective.This may lead to some deficiencies: (1) it is unable to directly estimate and control the safety margin of nuclear reactors, resulting in an increase in operation conservatism.Existing operating procedures are controlled based on state parameters that can be directly measured (or simply derived), without considering the operating limitations of the implied unmeasurable important safety state variables.Using indirect control will result in the narrowing of its operating boundary and lead to operating flexibility reduction.(2) Uncertainty reduces the effectiveness of control planning.A series of action suggestions could be determined based on the offline theoretical calculation combined with the planning optimization algorithm.However, offlinegenerated schemes may be failed due to observation uncertainty and theoretical model error.
The HeTu platform would solve the DT multi-probability problem from the perspective of confidence, which assumes that the running state is not a fixed constant at a certain time, but changed by probability with a certain confidence interval.

Probabilistic monitoring
2.2.1.Probabilistic monitoring for process system.Usually, there are many state parameters that cannot be directly measured in process systems, such as the effective neutron proliferation coefficient keff, nuclide density, etc.On the one hand, these unmeasurable state variables are crucial for the safe operation of the reactors.For example, temperature limits for fuel rods or fuel cladding are vital to avoid material performance failure.On the other hand, these unmeasurable state variables can only be derived from mathematical models based on the measured values [4] .The accuracy of the model and the uncertainty of the measured data have a great influence on the estimation or precision control of these unmeasured variables.
As shown in Figure 2, given the current reactor state t s , the corresponding confidence (covariance) The DT of important equipment, as shown in Figure 3, includes several steps: 1) monitoring all related important parameters of the equipment, 2) detecting the change of equipment parameters, 3) ascertaining the root cause of change, and clearing the fault mode, 4) based on the specific failure mode, predicting the remaining useful life (RUL) or probability of failure (POF).In [5][6][7][8] , the necessity of predicting the remaining useful life is pointed out, and the challenges in the development of the lifepredicting model and many potential applications of the RUL model are analyzed.However, how to make RUL predictions in practical application is still difficult.Different parameters which characterize the degradation performance could be proposed.Appropriate degradation trend parameters do not have to be directly measured parameters, and can also be obtained by a function of several measurement variables, or as an empirical model that cannot be measured directly.
through the idea of probabilistic programming and update the 95% -95% confidence interval of RUL.

Probabilistic control
In environments where incomplete observations and measurements are uncertain, the estimated state can be a value between the model predictions and the measurements.HeTu Platform uses the model predictive control [9] (MPC) algorithm to realize the full-state-space estimates that run along a predetermined trajectory, solve the optimization problem online in a rolling way, and produce a series of optimal control schemes.The MPC uses the loss function to describe the difference between the real state and the expected control state and then allows the intelligent algorithm to minimize the global loss function.Traditional control methods (such as a simple PID controller) are local methods, with a small convergence range, which may cause overshoot.The trajectory-based optimization control is to find a control sequence that can minimize the loss function.Given the target state trajectory ,target t s , the general form of MPC is as follows: ,Target 1: where w is the weights for different state variables.Reactor control is essentially a nonlinear control problem, while the HeTu platform will find a fixed point for linear approximation through the Taylor expansion, thus equivalent to a quadratic programming problem.A fixed point is easily found if the reactor system operates in a relatively small local range.Differential dynamic programming (DDP) [10] is proposed for the problem of state drastic changes.The thought of DDP is a linear approximation of multiple tangent points of the state transition function.It starts with a non-optimal policy used to control in a virtual way to generate nominal tracks.Each nominal trajectory is an approximation of the state and action and can serve as a linearized fixed point.And then, these linearized functions are used for the next iteration to generate the next nominal trajectory until the nominal trajectory converges to the target trajectory.

Calculation results
Data from the data challenge competition on Prognostics and Health Management (PHM08) [11] were used for RUL prediction in the HeTu platform.The predicted failure probability curve for engine # 1 is shown in Figure 5. .In order to reflect the effect of the predictive model error, a perturbed point core model is also constructed, where all the parameters of the core model are superimposed with the relative Gaussian error of (0, 2%).Obviously, the state prediction error of the perturbed core model will be gradually amplified compared with the real point core model.As shown in Figure 6, Due to the lack of information in the initial stage, the confidence interval of the prior probability may be large, so a large initial    The optimal estimation and modeling of all parameters in 3D state space based on the Bayesian principle will be a challenging problem.Furthermore, as shown in Figure 8, the probability distribution representation of 2D or 3D fields is investigated to enrich the cognition of probability confidence intervals further.

Conclusion
This paper introduces some solutions to the digital twin multi-probability problem in the HeTu platform, including probability estimation and probability control based on public data sets and a point reactor core model.The remained useful life prediction method based on equipment degradation parameters and probability programming is established, and the state estimation and control simulation of the typical reactor power load-follow curve is performed by using the MPC algorithm based on DPP methods.The results show that the proposed method will help to construct a more robust MPC in

Figure 3 .
Figure 3.The basic process of equipment health management

Figure 4 .
Figure 4.The basic process of equipment health management

Figure 8 .
Figure 8. Core power distribution probabilities in 2D or 3D (six sampling superimposed) an environment of uncertainty.Finally, a probabilistic digital twin system based on probabilistic programming, Bayesian estimation, and model predictive control methods could be developed in the HeTu platform.