Dimensional XCT comparison campaign on an aluminium object

An x-ray computed tomography (XCT) interlaboratory comparison campaign, involving an aluminium-machined object, whose dimensions (92 × 78 × 63 mm3) are significant for a 225 kV XCT system, was performed for the purpose of investigating the performances of industrial XCT systems for dimensional measurements in terms of accuracy, i.e. precision and trueness, and to evaluate the influence of the measurement protocol (i.e. measurement strategy), of the operator and of the software on the results by comparison to reference measurements. In this campaign, we came to the conclusion that the measurement strategy is predominant, except for distance; that the measurement process is affected by the operator only for cylindricity and coaxiality; that there is no or little influence of the software except for coaxiality and position; and that a volumetric Gaussian filter allows to improve the measurements only for some participants’ measurements Furthermore, different behaviours, in terms of precision and trueness, are observed depending on the type of measurands when performed by different operators. The diameter measurements are reproducible with XCT, lower than 30 µm which corresponds to a subvoxelic factor of 2.5 and the trueness is lower than 22 µm. The distance measurement is also reproducible with XCT, 15 µm which corresponds to a subvoxelique factor of 4.9 and the trueness is 8 µm. For these mesurands, their measurements do not depend on the used XCT system. However, the XCT reproducibility for cylindricity, coaxiality and position is worse as well as of the trueness except for the position which has a trueness of 1 µm. The process measurement should be revised regarding cylindricity and coaxiality measurements. Finally, overall, the ability of the participants to perform measurements with XCT, whatever their system, is statistically comparable except for a few measurements.


Introduction
Additive manufacturing (AM) gives the opportunity to fabricate parts with very complex geometries, including internal features. For material, but also dimensional quality inspections, of such complex parts, x-ray computed tomography (XCT) is the only adapted non-destructive method, hence the recent popularity of XCT in the AM industry. This industrial popularity brings out the need for XCT capability evaluation in terms of dimensional measurements.
The XCT working group of the French Confederation for non-destructive testing (Cofrend) [1] performed a dimensional XCT interlaboratory comparison campaign on a large aluminium-machined object. The purpose of this comparison was to investigate the performances of industrial XCT systems [2] for dimensional measurements [3,4] in terms of accuracy, i.e. precision [5] and trueness [6], and to evaluate the influence of the measurement protocol, of the operator, and of the software on the results by comparison to reference measurements.
Dewulf et al has performed an extensive review about dimensional metrology involving XCT [7]. One of the section is dedicated to 'Interlaboratory comparisons for dimensional and geometrical feature metrology' [8][9][10][11][12][13]. These interlaboratory comparisons involved several participants and several different XCT systems to perform dimensional measurements on various objects, and the comparison to reference measurements has been made. However, it seems that none of these comparison campaigns evaluates the influence of the measurement protocol, of the operator and of the software on the results by comparison to reference measurements. Furthermore, these comparison campaigns involved objects limited in dimensions to cube of 10 mm side, 15 mm diameter cylinder and 40 × 60 mm bracket, but also to external dimensions. In the present comparison campaign, the object's dimensions (92 × 78 × 63 mm) are significant for a 225 kV XCT system regarding correlated penetration power for aluminium. In addition, all measurands are internal dimensions.
The paper is structured as follows: the first section presents the different steps of the comparison campaign; the second section describes the circulating part in the comparison campaign; the third section is dedicated to the definition of the measurands; the fourth section reports on the calibration of the circulating part. Finally, the last two sections are related to the XCT dimensional measurements performed in the comparison campaign and their statistical analysis.   measurements of several measurands, on the 3D images, on their own, according to a first protocol (prot. 1). These first results were analysed and compared to reference measurements performed with a coordinate measuring machine (CMM). The high bias observed between XCT and CMM measurements led the working group to reconsider the measurement protocol to narrow the trueness. Thus, another protocol was elaborated (prot. 3) and had to be used by each participant to measure again the measurands. This new protocol (prot. 3) includes two majors changes: (1) increasing local searching parameters named 'Search distance' and 'Safety distance' in VGSTUDIO MAX to 1 mm, and (2) applying a volumetric Gaussian filter during reconstruction, before the segmentation, to reduce the noise. In addition, to evaluate the influence of this volumetric Gaussian filter, an intermediate protocol without applying the filter, was also implemented (prot. 2). Furthermore, each participant had also to provide their raw scans to VG and Zeiss such as they can carried out the dimensional measurements with VGSTUDIO MAX and GOM Volume Inspect, respectively. These different steps are summarized in figure 1 and the differences between the three different protocols are highlighted in table 1. Figure 2 explained schematically the  difference between the two measurement methods implemented in the three protocols: maximum inscribed and Gauss.

Description of the circulating part
An aluminium-machined object (figure 3), named CAD Cube, was provided by Zeiss.

Definition of the measurands
Seven measurands were selected for the comparison campaign, including one distance, three diameters, one cylindricity (figure 4(a)), one coaxiality (figure 4(b)) and one true position. They are listed and described in table 2 and in annex 1.

Calibration of the circulating part
The CAD Cube was calibrated, by Zeiss, with a tactile CMM Contura G2 using the measurement software Calypso 2021. An acquisition step of 0.1 mm was used, and 8071, 9661 and 5911 measurement points were performed on datum A, datum B and datum C respectively. Related to the first protocol, using maximum inscribed fit for the cylinders, 7 measurements, for reproducibility, were performed before circulation of the CAD Cube and 3 measurements were performed after circulation. When it was decided to push further the investigation on the measurement to adjust the XCT measurements to the reference (CMM) measurements, the CAD Cube was measured again three times implementing a second measurement protocol, using Gauss (least square) fit for the cylinders, except for M1 for which Chebyshev fit was used. The calibration measurements, implementing protocol 1 (before and after circulation of the CAD Cube) and protocol 2 (after circulation of the CAD Cube) for comparison, are provided in figure 5. As expected, the cylindricity (M1) is similar for the two protocols (figure 5) as no change in the strategy has been made. The diameters (M2, M3 and M5) measured with protocol 2 are around 10 µm higher than those measured with protocol 1. This increase appears consistent with the change in strategy from maximum inscribed to Gaussian.
As the measurement uncertainty was not provided by Zeiss, the maximum permissible error (MPE) of the CMM was used instead: MPE = [1.9 + L/300]µm where L is given in mm. Thus, the referenced measurements (x ass_p1 and x ass_p2 ) as well as their associated uncertainty (u ass_p1 and u ass_p2 ) are given by: For protocol 2: where p1 and p2 stand for protocol 1 and 2 respectively andx represents the average of the individual x measurements.

Participants involved in the comparison campaign
The campaign involved seven French XCT users and system suppliers: 3D Casting, Baker Hughes, Cetim, Nikon Metrology, Safran Composites, Yxlon, and Zeiss but one of the participants performed the measurements with two different XCT sources, one microfocus but also one minifocus sources, thus the comparison campaign involved eight participants. As presented in table 3, six different XCT brands (Nikon, NSI, RX Solutions, Waygate Technologies, Yxlon, Zeiss) were used. In order to preserve anonymity, each participant was designated by a number from 1 to 8 not related to the order in table 3.

XCT scans and 3D image reconstruction
A template table was sent to all participants beforehand, specifying the information to give on the XCT system, on the XCT scanning parameters and on the reconstruction and dimensional software used (Annex 2). It was asked to carry out three independent scans on the CAD Cube for repeatability. Thus, the object had to be removed from the XCT system in between each scan. A support holder, in polystyrene foam, was provided which tilted the CAD Cube in one direction ( figure 6). It should be noted that participants 4 and 8 did not   removed the CAD Cube from the XCT systems between each measurement and did not use the support. However no significant difference in terms of dispersion was observed on their measurements compared to the other participants, so their results were kept in the statistical analysis. As a result, the CAD Cube was not tilted during the scans for these two participants.
The participants were free to apply or not a beam hardening correction numerical filter on the reconstructed 3D image. Figure 7 provides a 3D XCT image of the CAD Cube.

Dimensional measurements
The XCT dimensional measuring process includes the scans of the object, the 3D image reconstructions, the segmentation of this 3D images and then the dimensional measurements of the measurands with a dedicated software.

By each participant.
The dimensional measurements performed by each participant on the 3D XCT images with their dedicated software, using protocols 1 and 3, are displayed in figures 8 and 9, respectively. All participants used VGSTUDIO Max except participant 7 who used GOM Volume Inspect. For comparison, the reference CMM measurements are also displayed as well as their expanded measurement uncertainty U = 2xu ass .
With protocol 1, there is a high repeatability between participants with a few exceptions. The trueness seems to be better for position (M6) and distance (M7) measurements than for the other measurands (M1, M2, M3, M4, and M5). The dimensional measurements obtained implementing protocol 1 provide a significant bias between XCT and CMM measurements. This is mainly due to the fact that (1) the search distance was too small which filters a lot of real points from the analysis (in particular when the deviation between the numerical model and the scan surface part is important), (2) the safety distance was too small which can lead to bias due to points located at the edge of the cylinders or planes, points not considered in CMM measurements, and (3) the maximum inscribed strategy for the cylinders is too sensitive to marginal points (and therefore to artefacts).
Thus, at this stage of the comparison campaign, considering the gap between CMM and XCT measurements, protocol 1 was abandoned and it was decided to correct the measurement protocol in order to minimise this gap due mainly to an error in the measurement process defined in protocol 1. Thus, protocol 3 was elaborated.   was made, Chebyshev method was implemented in both protocols 1 and 3. However, a volumetric Gaussian filter was applied in protocol 3 compare to protocol 1 and it seems that this filter has a slight influence. Indeed, a small improvement is observed with protocol 3 for most participants in term of dispersion or trueness.

By one operator with VG.
The dimensional measurements performed by Volume Graphics with VGSTUDIO MAX version 2022.1 on the 3D XCT images provided by each participant, using protocol 3, are displayed in figure 10. For comparison, the reference CMM measurements are also displayed as well as their expanded measurement uncertainty U.
When the measurement process on the 3D image is made operator independent, the dispersion of the coaxiality measurement (M4) is improved and a non-uniform difference is observed for cylindricity (M1), otherwise the other measurands are not affected. Thus, one can conclude that the measurement process is only slightly affected by the operator for diameter, position and distance measurements.     measurement uncertainty U. The protocol had to be slightly changed to process the data with GOM Volume Inspect to overcome some issues.

Measurement analysis
As can be observed from figures 9-12, there is a bias between XCT and CMM measurements with protocol 3 for cylindricity (M1) and coaxiality (M4) measurements except when measurements are performed by VG for M4. For the other measurands (diameters, position and distances), the bias are low.
In order to compare the effect of the software on the results, the measurements performed with GOM Volume Inspect and VGSTUDIO MAX were plotted on the same graph ( figure 13).
In addition, in order to compare the effect of the volumetric Gaussian filter (reduction of the noise and artefact at the expense of the resolution) on the results, the measurements performed with protocols 2 and 3 were plotted on the same graph ( figure 14).
As can be observed there is no influence of the software neither of the volumetric Gaussian filter on the diameter (M2, M3, and M5) and distance (M7) measurements. There is a high influence of both the software and the volumetric Gaussian filter on the coaxiality (M4) measurement, however the volumetric Gaussian filter allows to improve the measurements for few participants. There is an influence of the software and not of the volumetric Gaussian filter on position (M6) measurement. The opposite is observed for cylindricity (M1) measurement.

Comparison campaign statistical analysis
The results of the comparison campaign were analysed to evaluate the performance of XCT systems implementing ISO 5725-2 related to accuracy, and more specifically to one of the two components of accuracy: precision [5], and ISO 5725-4 related to the other component of accuracy: trueness [6] ( figure 15).
The International vocabulary of metrology (VIM) [14] defined accuracy as 'closeness of agreement between a measured quantity value and a true quantity value of a measurand' and trueness (i.e. Bias) as 'closeness of agreement between the average of an infinite number of replicate measured quantity values (Bias =x lab − x ass ) and a reference quantity value (x ass )' such as.
Precision is defined in the VIM [14] as 'closeness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions'. Precision can be quantified by two uncertainty contributors, i.e. repeatability (S r ) and reproducibility (S R ). Repeatability represents the dispersion of the results obtained under unchanged measurement conditions. Whereas reproducibility represents the maximal dispersion due to the method (different laboratories, different instruments, different operators…).
The variances of the results were initially analysed with a Cochran test. If the test failed, the individual results associated to the maximum variance were analysed with a Grubbs test for the detection of outliers in the data set. After the outlier's removal if required (table 4), the repeatability S r was evaluated as weighted mean of the standard deviation of each laboratory's results S ri , according to: where dof i represents the number of degrees of freedom associated to the standard deviation S ri . In addition, an ANOVA test (ANalysis Of VAriance) was performed on the participant's mean values to highlight systematic differences among laboratories. Measurement results affected by a systematic behaviour could be accounted for as laboratory effect S 2 L evaluated according to: where S 2 d corresponds to n times the variance of the mean and n is the number of repeated measurements of each laboratory (n = 3 in this comparison campaign). Finally, the reproducibility standard deviation S R was computed using: Furthermore, the results of the comparison campaign were also analysed implementing ISO 13528 [15] to evaluate the performance of each participant. The laboratory's capability to have results close to the reference value within its stated uncertainty can be assessed by a Z ′ score computed using: The results of these different analysis are provided in tables 5-8 and the Z ′ scores in figures 16-19. In order to relate the reproducibility standard deviation to the voxel size (Voxel/S R ), the average of the voxel sizes over all laboratories was used.
One can observe different behaviours depending on the type of measurands. The diameter (M2, M3, and M5) measurements with XCT are reproducible, lower than 30 µm which corresponds to a subvoxelique factor of 2.5 and the trueness is less than 22 µm. The distance (M7) measurement with XCT is reproducible, lower than 15 µm which corresponds   to a subvoxelique factor of 4.9 and the trueness is no more than 8 µm. For these mesurands, their measurements do not depend on the used XCT system. However, the XCT measurements for cylindricity (M1), coaxiality (M4) and position (M6) are not reproducible, and their trueness deviates for the true value, except for the position which has a trueness lower than 1 µm. The process measurement should be revised regarding cylindricity and coaxiality measurements  as the noise on the image generates variations on the measurements. An appropriate segmentation protocol should be defined. Thus, one can conclude that in this comparison, XCT encounters difficulties in measuring cylindricity and coaxiality.
Overall, considering the Z ′ scores, one can say that the capability of the participants to perform measurements with XCT, whatever their system, is statistically comparable.

Conclusion
In this article, the results of an interlaboratory comparison campaign on XCT dimensional measurements have been presented. This campaign presented the particularity to involve an aluminium-machined object whose dimensions (92 × 78 × 63 mm 3 ) are significant for a 225 kV XCT system (regarding correlated penetration power for aluminium) and also larger than most of the other interlaboratory comparison campaigns. In addition, the campaign involved only height participants but a large panel of XCT system brands (Baker Hughes, Nikon, NSI, RX Solutions, Yxlon, Zeiss). Furthermore, two different measurement software (GOM Volume Inspect and VGSTUDIO MAX) were compared as well as three protocols which differences relied, one on the measurement strategy, and the other on the volumetric Gaussian filtering of the images. In this campaign, we have come to the conclusion (table 9) that the measurement process is affected by the operator only for cylindricity and coaxiality measurements, that there is no or limited influence of the software neither of the volumetric Gaussian filter on the diameter, and distance measurements. However, there is a medium to strong influence of the measurement strategy on all measurands, except distance, and there is an influence of both the software and volumetric Gaussian filter on the coaxiality measurement, whereas the volumetric Gaussian filter has a high influence on the cylindricity but not the software, which is the opposite for the position measurement. Furthermore, different behaviours, in terms of precision and trueness, are observed depending on the type of measurands when measured by each participant (table 10). The diameter measurements are reproducible with XCT, lower than 30 µm which corresponds to a subvoxelique factor of 2.5 and the trueness is lower than 22 µm. The distance measurement is also reproducible with XCT, 15 µm which corresponds to a subvoxelique factor of 4.9 and the trueness is 8 µm. For these mesurands, their measurements do not depend on the used XCT system. However, the XCT reproducibility for cylindricity, coaxiality and position is worse as well as the trueness except for the position which has a trueness of 1 µm. The measurement method should be revised regarding cylindricity and coaxiality measurements particularly affected by the noise on the image. Finally, the participants are statistically comparable, most of their Z ′ scores are lying inside the interval [−2, 2] except a few measurands.
This interlaboratory comparison campaign highlighted the following lessons learned : (a) the fact that three successive protocols have been required to optimize the results, is showing that dimensional measurement onto XCT volumes is complex regarding performances in terms of trueness (despite some specialists into our working group); (b) dimensional measurements onto XCT volumes is appearing as a complex process because it requires separate skills that are not yet commonly shared : on one hand, skills in XCT, on the other, skills in dimensional measurements; (c) nevertheless, with a proper and solid protocol (prot. 3 in our case), the comparison campaign shown that human factor can be put under control and rather equivalent results can be obtained whatever the measuring process (manual or batch processing), in terms of hardware (height XCT system's models and six brands) as well as measuring software (two brands); (d) current interlaboratory comparison campaign shall be considered as a starting point regarding needs that are currently incoming with AM parts where internal dimensions will be required (impossible with classical CMM); (e) last but not least, precautions shall be taken regarding the position (inclination) of the part to avoid or reduce geometric artefacts generated by 3D reconstruction.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).