For metrology to be recognized as a measurement science, it must be seen to be using the scientific method. This requires metrologists to make predictions that can be tested and validated by experiment. The fundamental testable prediction is usually a variant of 'agreement within the claimed uncertainty', and the experiment is usually a comparison of two or more nominally identical measurements. The normalized error, E_n, can be generalized as the ratio of {a difference of the two values} to {the standard uncertainty in the difference of the two values}. This definition can apply equally to the difference between a particular measurement and a reference value with an uncertainty or to an unmediated bilateral difference between two measurements considered as peers. This latter interpretation leads to the creation of a family of bilateral E_n values in a comparison, which can be aggregated by taking the root-mean-square (RMS) average. This RMS E_n is a norm that can support intuitive ideas of ordering performance in the comparison. The mean-square E_n is a chi-squared-like statistic and can be evaluated by Monte Carlo simulation to perform quantitative tests of the ideal agreement hypothesis for a comparison. The use of these statistics in broader aggregates is discussed: averaging across similar unlinked key/regional comparisons, across a range of artefact values, across different principal measurement techniques in a given metrology area or even across all major metrology areas spanning the entire International System of Units. Each of these 'averages' can be done as an overall aggregate of all participants or can focus on one particular participant's RMS E_n aggregated with respect to all its peers' results.