Surface temperature measurement from infrared synthetic diagnostic in preparation for ITER operations

The protection of ITER in-vessel components and the plasma-wall interaction studies will be based on a large network of infrared (IR) cameras covering 70% of the tokamak. The surface temperature measurement from IR images remains challenging due to the presence of metallic targets, with changes in surface thermo-radiative properties (emissivity) and the presence of multiple reflections. The paper provides an overview of major progress to improve the interpretation of IR image and to get more reliable surface temperature from IR synthetic diagnostics. The paper presents the latest development of (1) the forward model to include the modelling of the edge localised modes and a new advanced camera that is better adapted to experimental data (2) the inverse model to retrieve the emissivity of the targets and the surface temperature from a neural network trained exclusively from synthetic IR images. Promising results have been obtained both from simulated test images with an estimated emissivity better than 0.05 and a surface temperature better than 10%, and from WEST experimental images of ITER-like wide-angle to filter reflection patterns.

Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence.Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

Introduction
The infrared (IR) thermography system on ITER has been designed to monitor ∼70% of the first wall and divertor surfaces.It should be capable, of measuring surface temperature with sufficient accuracy to detect anomalous behaviour of Plasma Facing Components (PFCs) between 200 and 3600 • C.However, there are issues with IR measurement in metallic environments due to low and variable emissivity of targets and reflection contributions, causing major errors in temperature measurement.This calls into question the reliability of IR camera to prevent overloads and material failure during ITER operations.Related R&D is considered as a high priority in the ITER research plan (issue A.14 of [1]).
The aim is to develop and provide advanced data processing for robust IR temperature measurements to address this issue.This novel numerical approach is based on an IR synthetic diagnostic (or forward model), capable of providing realistic simulated IR images of a given plasma scenario.As explained in section 2, synthetic diagnostic is first and foremost a practical tool for understanding and quantifying the impact of various disturbances in the measurement chain.It is also used to develop inverse processing, using unique new machine learning (ML) methods to achieve reliable surface temperature measurement in this harsh environment.
Section 3 presents new advances in achieving more realistic synthetic images-including modelling transient events such as edge localised modes (ELMs), a new advanced camera model including distortions and a tool for automatic adjusting camera parameters from experimental images.Section 4 deals with the last results of inversion algorithms based on ML techniques, to remove reflection features of IR images and estimate target emissivity and surface temperature.

Estimation of temperature measurement error from IR imaging systems
IR systems are subject to several errors, not all of which have the same effect on temperature measurement, depending on the target temperature.Figure 1 shows an estimate of the total error on the measured temperature including the errors coming from the measurement chain (transmission, parasitic light) and coming from the observed thermal scene (emissivity, reflected flux) for the equatorial wide-angle viewing systems.We consider an uncertainty on the emissivity value ∆ε of 0.15 for a tungsten target around of 0.2 emissivity, which is in agreement with laboratory and tokamak measurements [2].The total transmission of optical system is fixed to 10% with an error ∆τ of 50%, including the calibration uncertainties and the flux losses due to spatial resolution [3].The contribution of the parasitic light from the instrument (due to hot optics) and from the thermal scene (i.e.due to multiple reflections in the metallic environment) is fixed at 100% of according to [4].Without correction of the target emissivity and the parasitic fluxes or without significant calibration effort (specifically for monitoring optical transmission and assessing straylight during operations), the temperature errors are outside the ITER specification (10% for high temperatures and 20% for low temperatures) [5].There are also two operating regime: the lowest temperatures (<400 • C) will be more sensitive to the presence of parasitic flux (flux reflected coming from the observed thermal scene or coming from the instrument itself, also so-called stray light) while the highest temperatures (>400 • C) will be sensitive to uncertainties in emissivity and calibration errors.To achieve the specifications required by ITER, all errors must be minimised.Typically, this requires an absolute error on the target emissivity of less than 0.05, a relative calibration error of less than 10%, and a correction of instrumental background and reflecting environment of less than 20%.We are working on this integrated approach through the development of a complete and robust simulation, capable of dealing with all contributors from the thermal scene to the instrument.

Simulating augmented IR images
For better interpretation of IR images and more reliable temperature measurements in complex environments such as tokamak, the developed approach is based on a robust and faithful simulation of the entire IR measurement chain-from the observed thermal scene to the detector.Such an end-to-end simulation is used to predict IR measurement performances of upper and equatorial wide-angle viewing systems of ITER [4,6] and initial applications to experimental data from WEST and AUG have already shown that simulation can reliably discriminate reflections patterns from real thermal events [7].Now, to go one step further and make a quantitative comparison for temperature measurement, reproducing realistic IR images or 'augmented' images-close to the real worldis a key point in processing experimental data.This implies efforts throughout the IR measurement chain to incorporate more complete physical models, while gaining more knowledge about the range of model parameters.In the paper, we start with new advances in the geometrical calibration of the camera, which is the first and mandatory step for the recognition of hot patterns and their localisation in the thermal scene (2D-3D mapping).This is also necessary to adjust the camera viewpoint (line-of-sight, field-of-view, etc) so that the simulated images match with the experimental data with an accuracy of a few pixels (∼5 pixels) for advanced analysis.Another point of progress is modelling of the thermal scene in tokamaks.This is important to predict the IR camera response for different plasma scenarios and check the camera performances (spatial, temporal resolution).A new advance in this area is achieved by simulating ELMs in the IR images.

Geometric camera calibration
In computer vision, the geometric camera calibration (or also called to camera resectioning) is the process of estimating the parameters of camera model from a single input image.This includes the linear parameters such as the pupil position, line of sight (i.e. the camera extrinsic parameters), the detector size, focal length (i.e. the camera intrinsic parameters) and the non-linear intrinsic parameters such as lens distortions.This does not include diffraction and aberrations.Camera geometric calibration is the first and essential step in comparing experimental images with simulated images to obtain reliable analysis and accurate measurements.This is particularly true for a fully reflective scene, where the flux collected per pixel is highly dependent on the viewing angle, and therefore the camera position and orientation.These tasks is also mandatory to ensure the protection of individual in-vessel components (for 2D-3D mapping).
Two methods-semi-automatic and fully automatic-have been developed for the estimation of geometric camera parameters (the parameters related to diffraction and aberrations should be characterised in the laboratory and assumed not to change during the reactor life).The first semi-automatic method is based on a dozen pairs of points in the 2D image and the 3D world that have been previously selected and defined by hand.A gradient descent algorithm is then used to minimise the loss function, which calculates the mean square error between the calibrated points and those predicted by the camera model.This method must be able to estimate between 10 and 16 camera parameters: seven parameters are related to the pinhole camera parameters (position, orientation/direction and focal); the others are related to the lens distortions (up to nine distortion coefficients in case of radial and tangential distortions).This method has made it possible to achieve an accuracy of better than 10 pixels for high-resolution cameras (1024 × 768 pixels) and 5 pixels for lower resolution cameras (512 × 640 pixels).This is an interesting method for checking the accuracy of the geometrical calibration, as we have access to ground truth through the calibrated points.But it is not designed to process large amounts of data (and even less in real-time), as such a method would require manual processing of each experimental image to select the dozen reference points.The second method aims to determine the camera parameters fully automatically using ML techniques, without user input.A neural network model is built based on a convolutional neural network (CNN) coupled to a multiperceptron (MLP), using the camera image as the input layer and the camera parameters as the output.The neural network is trained from a data base of synthetic images obtained by varying the camera parameters (direction, orientation, focal length and distortions parameters).The NN performance is first evaluated on a test set of 600 simulated images.The camera parameters are predicted very well with less 1 • error in the field of view and 0.5 • deviation in the line-of-sight.In a second step, the NN is applied on experimental image of WEST wide angle view.To visualise the results, a ray-tracing code then uses the NN-predicted camera parameters as input to calculate the mask of the main in-vessel components.Figure 2(a) shows the experimental image superimposed with the mask contour calculated from the NN-predicted camera parameters.In this case, the difficulty is to quantify the accuracy achieved with the experimental image, as no ground truth is directly available.The validity and performance of the method are then evaluated on several experimental images with manual segmentation.Intersection Over Union (IoU) is the metric used to quantify the degree of overlap between the predicted mask and the ground truth mask.An IoU of one indicates that the predicted mask is completely overlaps with the ground truth mask.Lower IoU indicates poor segmentation.An IoU greater than 0.8 is found for large components (lower and upper divertor, baffle, inner and outer wall) and lower IoU (0.5-0.6) for antenna and tubes.The camera parameters predicted by CNN are then used as input to a 2D-3D mapping algorithm to compute geometric maps, giving, for example, the target distance to the pupil, the viewing angle, and pixel resolution (figure 2(b)).

Thermal scene modelling
The modelling of the thermal scene includes (1) the modelling of photon-wall interaction, (2) the modelling of all IR emission sources.The challenge in this area is to obtain sufficiently accurate physical models while limiting the range of parameters values for measurement inversion.To this end, efforts are being made to better characterise the optical properties of materials (emissivity and reflectivity) and to diversify the IR source emission models including steady-state heat loads and transient events.

Material properties characterisation.
The Bidirectional Reflectance Distribution Function (BRDF), which describes the photon behaviour on a surface for any incident and viewing direction, is a determinant factor for the fidelity of ray-tracers and the understanding/simulation of the optical measurements.Systematic measurements and experimental characterisation of the BRDF for tungsten samples (before plasma exposure) have been performed in order to establish reliable BRDF models (to be used in the synthetic IR diagnostic) for different surface states (roughness, deposit, etc) [8,9].The experimental results demonstrate the high specular behaviour of W samples, in particular in the IR range [3-5 µm], fitting roughly with classical models of BRDF implemented in ray tracers (as Gaussian, micro-facet models).The detailed dependences on roughness and surface state before and after exposure are also being studied.We aim to find parameters that are easier to measure (e.g.topographical parameters) that can be linked to BRDF and used to constraint it.In particular [10,11] investigate new metrics such as surface inclination angle or the power spectral density analysis to link BRDF parameters with topography measurements from tungsten and beryllium samples.

Assessing transient events on IR measurement.
One goal is to have a comprehensive library of simulated images for different operating scenarios to build inverse models (as discussed in section 4).This means simulating the observed thermal scene in the tokamak for different plasma scenarios (from standard plasma scenarios to more exotic scenarios).In most standard plasma scenarios, the main IR source comes from heat load deposited on PFCs.In steady-state, a magnetic field line tracing code, based on optical approximation, is used to compute steady-state heat flux on 3D mesh geometry of the first wall and divertor, from the magnetic plasma equilibrium [12,13].To include the modelling of transient events likes ELMs or disruptions, the non-linear extended MHD code JOREK is used to simulate density distribution and the resulting heat loads on PFCs [14].The resulting 3D temperature field is then used as input of ray tracing code, the core of synthetic diagnostic simulation tool, able to propagate the rays within 3D geometry taking into account the materials optical properties (i.e.emissivity and reflectance).
Figure 3(a) shows an example of an ELM simulation in ITER using the JOREK code.During ELM, plasma filament cross the separatrix, while fast transport along magnetic field lines drive most of the energy from those filaments down to the divertor, where an increased heat-flux can be observed.Figure 3(b) shows the time profile of heat flux during ELMs and the resulting surface temperature computed from 1D thermal model, representative of monoblock geometry (including material layers and water cooling).Figures 3(c) and (d) show this ELMs observed by the equatorial Wide Angle Viewing System (WAVS) taking into account pixel resolution and assuming high temporal resolution (0.5 ms/ 2 kHz).Figures 3(e)-(h) show the apparent temperature (assuming blackbody target) profile along the inner and outer vertical target and during the pulse (t = 0 ms-pre-elms; t = 1 ms max; t = 2 ms-decay).For example, the ELMs cause a real increase in surface temperature of around 200 • C on the inner vertical target (IVT), whereas the system measures an increase of 100 • C due to the reflection contribution and the angle of view from divertor view.In the second step, thermal analysis based on Finite Elements or Finite Volume Methods should be performed to convert the power deposition on PFCs into the surface temperature while considering the realistic boundary conditions (thermal properties of materials, cooling system) [15].We see the importance of obtaining accurate physical models (SOLPS, thermal model, BRDF models).

Thermal scene assessment from IR images
From IR synthetic diagnostic (forward model), inversion algorithms are developed in order to retrieve surface temperature and/or emissivity from experimental images.Such techniques should be able to handle parasitic flux coming from reflections and uncertainties on targets emissivity.The construction of inverse models is based on synthetic images, able to simulate all possible scenarios.A novel method based on the supervised deep fully CNN has been investigated.Unlike the iterative inversion strategy based on minimisation between the experimental images and images predicted by the forward model [16], supervised deep-learning methods are based on big-data training set that enables calibration of the weights of the neural networks.The calibrated network is then ready to process an unseen camera image.Such a method has been applied and tested on a WEST-like numerical prototype to assess target emissivity from a dedicated scenario and surface temperature during pulses.

Emissivity assessment
WEST experience has shown that the emissivity of metallic targets changes after plasma, mainly due to the erosiondeposition phenomenon that occurs during the reactor lifetime [2].A challenge for getting more reliable IR measurements and safe operation is to be able to follow emissivity change during the experimental campaign.We propose here to use the experimental images of baking operations, when the temperature of the divertor and the first wall are different and spatially uniform, in order to assess and monitor the emissivity of the observed thermal scene.These thermal scenes have already been used to identify emissivity profiles either experimentally with the two-temperature method [17] or numerically by processing a single image with a gradient descent method [18].We propose here to process a single baking image but with ML tools.A large database of 3000 simulated images is first created by varying the optical properties of the wall (emissivity and BRDF), assuming an uniform temperature on the divertor and first wall.The spatial distribution of emissivity on lower divertor is based on a nominal radial profile measured in [17] to which Perlin noise is applied to randomly vary the emissivity values in the poloidal and toroidal direction.This synthetic data base is then used to train a CNN having as input the image of wall baking and as output the emissivity map.Five cases were tested from WEST numerical prototype by changing the temperature difference between first wall and divertor (∆T = 10, 25, 50, 80, 100, 160 • C).If the first wall and divertor are at the same temperature, the reactor behaves like a blackbody without sufficient contrast to assess emissivity.Figure 4 demonstrates that targets emissivity could be assessed with a high accuracy of 0.03 ± 0.03 as long as a temperature difference of 80 • C is achieved in the WEST case.It should be noted that the required differential temperature between the first wall and the divertor depends on the sensitivity of imaging systems and should be evaluated for ITER systems.But this proves that wall baking could greatly help monitor emissivity, provided that different baking temperatures on the first wall and divertor are applied.

Temperature assessment
In [19], it is shown that an inverse model based on ML techniques is able to estimate the temperature with an error better than 6% on a thermal scene considering pure specular surfaces.In [20], new results have been obtained on complex thermal scenes, taking into account realistic surface properties (BRDF) and extended temperature distribution including local and unpredicted hot spots (e.g.due to transient events).A huge data base of 60 000 simulated images is used to train a CNN (U-net).This involved the development of a novel Monte Carlo IR ray tracer code on the graphics processing unit (GPU) [20], which can simulate the complex photon behaviour in 3D geometry, while significantly reducing computation time compared to classic ray tracers.This approach has been demonstrated on simulated images of ITER-like wideangle view of WEST in the case of standard plasma scenario (lower single null).The U-Net performances have been evaluated on a test dataset of 100, 000 simulated images including noisy images.For the lower temperature range (<400 • C), the average error in temperature predicted by the U-net is 9%, with a total of 89% of predictions within 20% error.For the higher temperature range (>400 • C), the average error is 4%, with a total of 90% of predictions within the 10% error required by ITER [19].Applied to experimental images of WEST angle viewing system, the U-net is first able to remove most of the reflection patterns as shown in figure 5. U-Net also predicts the real surface temperature from the apparent temperature map measured by the camera without knowing emissivity.Apparent temperature is the default value provided by the camera.It assumes an emissivity of 1 which leads to underestimate the temperature of the hottest targets, while the NN predicts a higher temperature, which is in good agreement with metallic targets of low emissivity.At this stage it is still difficult to confirm U-net's temperature prediction from experimental images, without ground truth.This will require additional efforts to quantify uncertainties in the neural network's prediction, considering measurement uncertainties (e.g.calibration, stray light, etc).The next step is to validate the proposed methods for retrieving the emissivity or the surface temperature on images obtained from well-controlled and dedicated experiments in laboratories.Finally, this approach will be tested in a complex tokamak environment such as WEST using embedded diagnostics (thermocouples, Bragg fibre) to cross-check the temperature measurements.

Conclusion
Measuring the surface temperature of fusion devices is very challenging because we have to deal with multiple sources of perturbation/errors, both from the instrument and from the observed thermal scene.The integrated numerical approach we propose aims to incorporate all these contributions to provide more reliable image interpretation and accurate surface temperature measurements.This is based on a detailed simulation of IR imaging diagnostics, which is constantly evolving to take into account new and more accurate physical models.At this stage, IR synthetic diagnostic can simulate stationary plasma scenarios, ELMs, complex photon-wall interactions and advanced camera models.In parallel, inverse models are developed from synthetic diagnostics to retrieve targets' emissivity and to predict the surface temperature of in-vessel components after filtering reflections.A first neural network, based on the Unet architecture, has been trained on a large database of simulated images and has given very good results in predicting the temperature from simulated (noisy) images of an ITER-like wide-angle camera of WEST (with an estimated error of less than 10% for higher temperature (>400 • C) and 20% for lower temperature within the ITER specification in 90% of cases).The U-net has also proved highly effective in removing reflection features in WEST experimental images.
The ML methods have certain limitations.Unlike traditional iterative methods (Gauss-Newton) for finding input model parameters, the optimised neural network is not powerful enough to predict situations it has never seen or learned about.As a result, these methods require a large training database of simulated IR images covering all the situations that may occur during operations.That means, first of all, working to develop accelerated algorithms (thermal model, ray tracing codes).Secondly, that means working on all IR sources emissions to cover the maximum number of possible plasma scenarios.Another challenge is the assessment of uncertainties due to the limited amount of ground truth data.In terms of ITER operations, the feedback of experience from this new approach applied to current facilities in operation will be of great benefit to generate optimised training databases for a better interpretation and exploitation of IR measurements.
European Commission.Neither the European Union nor the European Commission can be held responsible for them.

Figure 1 .
Figure 1.Total error on temperature measurement from IR imaging systems in fully metallic environment.The temperature error is expressed as a percentage in degrees Celsius.The second vertical axis gives the percentage contribution of each source of error (emissivity, transmission, instrumental background and reflecting environment).

Figure 2 .
Figure 2. (a) Overlay of a predicted mask on the experimental image of WEST wide angle tangential view (pulse#55 567).The mask is computed from predicted camera parameters by the neural network.The large in-vessel components (lower and upper divertor, baffle, bumpers, inner and outer panels) are well segmented (with IoU > 0.8).(b): Geometric maps computed from deterministic ray tracer using predicted camera parameters as input: geometry Id, target distance, pixel resolution, angle, etc.

Figure 3 .
Figure 3. (a) ELM simulation using JOREK code (b) Time profile of heat flux during ELMs and the resulting surface temperature (time step = 24 µs) (c) and (d) Simulated IR images of divertor and left tangential view of ITER E-WAVS during ELMs (maximum temperature).(e) and (h) The spatial temperature profile on inner vertical target (IVT) and outer vertical target (OVT) is shown during ELMs (t = 0 mspre-elms; t = 1 ms max; t = 2 ms-decay).

Figure 4 .
Figure 4. (a) Simulated image of wide-angle tangential view in the case of divertor baking at 170 • C and wall baking at 90 • .(b) True and predicted emissivity profile on WEST lower divertor from convolutional neural network trained from synthetic data base.The simulated test image are noised to account for measurement uncertainties, which causes predictions uncertainties shown as error bars.

Figure 5 .
Figure 5. Application of inverse IR radiative neural network on experimental images of WEST wide angle tangential view.(a) Apparent temperature (or blackbody temperature assuming emissivity of one) provided by WEST camera during pulse #55 567.(b) Real surface temperature of observed thermal scene predicted by U-net.The-U-net is trained from an extensive training data set of simulated images.Most of reflection patterns (coming from lower divertor and antenna on upper divertor and from glow electrode on inner wall and upper divertor) are removed by U-net.