The evaluation of uncertainty in mass calibration: possible approaches in a comparison study

In this paper we give the results of four methods of calculating uncertainty associated with a mass calibration problem, three based on different implementations – the first and second order law of propagation of uncertainty and the Monte Carlo method – of the general methodology described by the Guide to the Expression of Uncertainty in Measurement, the fourth based on a Bayesian formulation. The nonlinearities present in the model for the calibration problem means that the first order approach can be an unreliable method for evaluating uncertainties, relative to the other three approaches.


Introduction
The application of the GUM (Guide to the Expression of Uncertainty in Measurement) [1] uncertainty framework to the evaluation of uncertainty is accepted as adequate in most fields of metrology, including mass metrology. The methodology has, as its starting point, an input-output model describing the measurand as a function of influence quantities and the assignment of probability distributions or probability density functions (PDFs) to the influence quantities. Once these two elements are in place, the PDF associated with the measurand, the output quantity is defined. The issue is how to determine information about this PDF in a computationally convenient way.
The GUM uncertainty framework, as usually implemented, involves two types of approximation, i) linearization, i.e., first order Taylor expansion, of the function relating the output to the inputs, and ii) an appeal to asymptotic results derived from the Central Limit Theorem to associate a Gaussian distribution to the output quantity. (The GUM also provides a methodology for associating a student tdistribution to the output quantity, using a scheme to determine effective degrees of freedom of the tdistribution. We do not consider this more general scheme in this paper.) For linear models and Gaussian inputs, the GUM uncertainty framework is exact. In all other cases, it will only provide an approximate solution. The quality of the approximation is however difficult to predict, a priori. The GUM also discusses the use of a second order Taylor expansion, which should lead to more accurate approximations. GUM Supplement 1 [2] describes the use of a Monte Carlo method (MCM) to propagate the distributions associated with the input quantities through to that associated with the 1 To whom any correspondence should be addressed. output quantity. The MCM does rely on approximating the function nor on asymptotic results associated with the output distribution and can be used to test the validity of the GUM approach or as the main computational tool for evaluating the uncertainty [3]. Moreover, an important advantage of MCM is that it goes beyond the evaluation of the measurement result and its associated standard uncertainty since it propagates the probability density functions instead of just the uncertainties of the input quantities. This leads to an estimate of the PDF of the output quantity, and thus any required statistic, including the measurement result, the associated standard uncertainty and coverage intervals, can be obtained from this distribution, as represented by the Monte Carlo sample. Another important advantage of MCM is its applicability regardless of the nature of the model, e.g., those incorporating strong nonlinearities. As mentioned above, the GUM uncertainty framework does in fact encompass second order methods to deal, at least partially, with nonlinearity.
The three approaches mentioned so far, the GUM first and second order and MCM, can be viewed as computational approaches that attempt to evaluate summary information about the same PDF, that associated with the output quantity, defined in terms of the input-output model and the PDFs assigned to the input quantities [4]. In a Bayesian formulation of uncertainty evaluation, the starting point is somewhat different, in that a prior PDF has also to be assigned to the measurand. However, the MCM method coincides with the Bayesian approach for a particular choice of prior and the sample produced using the MCM approach is a sample for the Bayesian posterior distribution constructed using that particular prior [5,6]. The MCM sample can be used to generate a sample from the posterior distribution constructed using another (perhaps more appropriate) prior for the measurand through the application of an extremely simple Markov chain Monte Carlo (MCMC) algorithm [7]. In this context, the GUM and MCM methods can be viewed as approximate computational approaches to Bayesian uncertainty evaluation.

The mass calibration model
See for example [3, section 9.10]. The model concerns the calibration of a weight W of mass density W ρ against a reference weight R of mass density R ρ . It is assumed that the two masses are nominally the same. The calibration is performed using a balance operating in air of density a ρ and provides an estimate of the mass R m δ of a small weight R δ , also of density R ρ , needed to achieve a balance.
Taking into account buoyancy effects, the application of Archimedes' principle leads to the following model equation.
involving the masses and of W and R, respectively. It is usual to work in terms of conventional masses relating to a standard density from which we obtain (using an approximation adequate for most purposes) The calibration prob e of c W m , , and its associated uncertainty, from . Thus, Eq.
(2) provides the input-output model to m as a function ξ f m = of n of uncertainties .

The diff
) ( enable the GUM and MCM approaches to be executed.
In this paper, we compare the MCM against the first and second order propagatio and also compare the MCM method with a Bayesian evaluation of uncertainty

Uncertainty evaluation using the first and higher order GUM approach
If ) (ξ f = η is a function of n input parameters j ξ , the GUM gives a general approach based on the propagation of uncertainty (LPU) for propagating uncertainties associated with estima e eter j la tes of th inputs to that associated with an estimate of the In its simplest form, the approach states that if each input param w of output.
ξ is associated a probability distribution with mean and standard deviation , then the parameter , and standard deviation assuming that the j ξ are independently distributed. The expression for the uncertainty ) ( 2 y u given in Eq. 3 is derived from a first order approximation to the function f. A more accurate estimate can be made by using a higher order approximation. The general approach is as follows [3 8]. If we a ][ sume that the input quantities are independently distributed, and make the further assum distributions are mmetric, there will be important simplifications tage of, and the some of expansion terms will be zero. With these assumptions, a higher order estimate of s ption that the input sy to take advan η given by and its associated uncertainty is estimated by If j ξ Δ is associated with a normal or rectangular distribution centred at 0, then and , respectively. The calculation also requires the evaluation usually the main deterre pleme is appr j i nt in im nting th oach. For the mass calibration problem with the data as in Table 1, the only nonzero higher order partial derivatives that will be evaluated as nonzero (taking into account that x The second order estimate of η is the same as the first order estimate, since but its ssociated uncertainty is given by x mptions of normality. For many implement.

Discussion of results
As an example calculation, we assume that the information about c R m , and c R m , δ is taken from calibration certificates and for which it is appropriate to assign Gaussian distributions. We assume that the density information is given in terms of upper and lower limits for which it is appropriate to assign rectangular distributio This information is summarised in Table For the data in Table  e estimate  Compared to the second order approach (and even the GUM first order approach), the MCM method is straightforward to implement. Fig. 1 shows the normalised histogram of 100,000 MCM samples generated using the input distributions specified in Tab . The degree to which the MCMC algorithm modifies the MCM sample depends on the extent that q d varies. For the mass calibration example, these partial derivatives vary by less than 5 parts in 10 f ∂ to produce 6 , which means that the MCMC sample is essentially the same as the MCM sample. In fact, all of the 100,000 of the MCM samples were accepted so that the two samples are identical. For larger MCM samples, some would be rejected but would not introduce any meaningful differences. The good agreement between the two approaches is due to the fact that the dependence of the measurement equation on c R m , δ is almost linear. For such cases, the MCM approach is an extre proach fo pling from the Bay [6,7].  Table 2. The first three methods all attempt to provide summary information about the same probability distribution, namely the output distribution associated with Eq. 2. The differences are in the level of approximation, in the case of the GUM methods, a first or second order approximation, in the case of MCM, the approximation of a continuous distribution by a discrete sample. As the number of MCM trials increases, the discrete approximation can becomes more exact. Due to nonlinearities in the model, the first order GUM method significantly underestimates the standard deviation of the output distribution. (For a smaller uncertainty associated with the density quantity W ρ , the 1 st order GUM gives a more accurate estimate of the uncertainty.) The GUM 2 nd order method gives satisfactory estimates and can be implemented for the mass calibration problem without much complicated calculation. For a general model, there can be a considerable (or prohibitive) amount of work involved.
As indicated above, the MCM and MCMC approaches give identical results.

Conclusions
Four computational approaches have been applied to evaluating the uncertainty associated with a mass comparison. Three, the GUM 1 st order, GUM 2 nd order and the Monte Carlo Method (MCM) all involve the same distribution determined by the measurement equation and the assignment of distributions to the influence quantities. The differences between them therefore reflect the degrees of approximation implicit in the computational approaches. The GUM 1 st and 2 nd order approaches involve truncation approximations and assign Gaussians of the basis of the estimated first and second moments of the distribution. The MCM method is approximate in the sense that only a finite sample is constructed. The MCM method and GUM 2 nd order method are in close agreement for the calculated example.
The fourth method involved a Bayesian formulation of the mass comparison measurement and necessarily involves the assignment of a prior distribution for the quantity of interest. If a noninformative prior is chosen, the Bayesian posterior distribution is essentially the same distribution from which the MCM determines a sample. This means that the MCM sample can also be regarded as a sample from the Bayesian posterior corresponding to the assigned prior, as confirmed by the application of the Markov chain Monte Carlo (MCMC) scheme. Thus, three approaches give essentially the same results: GUM 2 1 ) ( , ∝ c W m p nd order, MCM and the Bayesian approach. The GUM 1 st order approach is not recommended for the mass comparison measurement problem. Of the three approaches, the MCM method is the easiest to implement. The MCMC scheme implementing a Bayesian approach requires a modest extension of the MCM approach.