Technical evaluation of a simulator for accurate reproduction of oscillometric blood pressure pulses, providing traceability for automated oscillometric sphygmomanometers

Oscillometric blood pressure measurement devices are not directly traceable to primary standards. Currently, device accuracy is measured by comparison between a sample device and reference measurements in a clinical trial. We researched in this study the potential for an alternative evaluation with a simulator. Our research simulator was studied for repeatability and accuracy in delivering simulated blood pressure pulses. Clinical cuff pressure measurements were obtained, along with simultaneous recordings of oscillometric pulse waveforms, spanning the clinical range of cuff pressures, pulse intervals and pulse shapes. Oscillometric pulse peak amplitudes ranged from 1.1 to 3.6 mmHg. Simulated repeatability results showed an average Standard Deviation (SD) for pulse peaks of 0.018 mmHg; 1.0% of peak amplitudes. Comparing simulated pulse shapes, the average repeat SD was 0.015 mmHg; 0.8% of the normalised pulse shapes. The simulated accuracy results had a mean error of − 0.014 ± 0.042 mmHg with a mean accuracy of 97.8%. For pulse shape the corresponding values were − 0.104 ± 0.071 mmHg with a mean accuracy of 95.4%. The correlation between the reference and simulated pulse shapes ranged from 0.991 to 0.996 (all p < 0.00003), with a mean 0.994. We conclude that oscillometric pulses can be reproduced with high repeatability and high accuracy with our research simulator. The extended uncertainty U(p sim ) = 0.3 mmHg for the simulated pulses is dominated by the uncertainty (64%) of the clinical reference data. These results underpin the potential of the simulator to become a secondary standard for millions of oscillometric sphygmomanometers.


Motivation
In 2019 about 1.28 billion people worldwide were afflicted by hypertension, and only 23% of woman and 18% of men have it under control (WHO 2022). Usually the blood pressure is determined by noninvasive techniques, while invasive techniques are used by medical professionals in critical cardiovascular situations. Auscultation with a stethoscope at the upper arm in combination with an upper arm cuff and a pressure manometer is the typical manual noninvasive blood pressure measuring technique (auscultatory technique). The most frequent automated non-invasive technique is the oscillometric one. In 1982 Geddes and co-workers (Geddes et al 1982) published the main aspects of this technique. Due to the simple application of oscillometric sphygmomanometers, these devices can be used by a layperson as well as by professionals.
Fortune Business Insights (Fortune Business Insights 2023) reports a total global volume of 1.6 billion USD for sphygmomanometers in 2022, and about 67% for automated sphygmomanometers, 13% for ambulatory blood pressure monitors and about 20% for manual sphygmomanometers. When estimating an average price of 100 USD for digital blood pressure monitors, 2000 USD for ambulatory blood pressure Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. monitors and 60 USD for a manual device. This corresponds to about 10.7 million digital blood pressure monitors, 0.1 million ambulatory blood pressure monitors and 5.3 million manual devices sold on the global market in 2022. Another source (Global Market Insights 2023) reports two times higher numbers.
To establish confidence in the reliability of determined blood pressure values, the critical components of the sphygmomanometers have to be tested before putting them on the market (with type approval) and when in use (with frequent verification). For manual sphygmomanometers this is an easy task, because essentially the accuracy of the pressure manometer can be tested. The test on the permissible error limits for the pressure display does not require very challenging, expensive reference manometers for the calibration. In contrast to that, the performance test of automated sphygmomanometers is much more challenging. Oscillometric sphygmomanometers, which dominate the market, require special setups to test the accuracy of the displayed blood pressure values of systolic and diastolic blood pressure, and sometimes additionally the mean arterial blood pressure. Present standards or other normative documents (ISO 81060-2, International Organization for Standardization 2009, Stergiou et al 2018) describe how to evaluate automated non-invasive sphygmomanometers in a clinical trial performed usually before putting it on the market. Tests for devices in use are essentially limited to a test of the pressure sensor (ISO 81060-2, International Organization for Standardization 2009). It is expected that the software which passed the initial clinical trial is unchanged.
Initial clinical trials for the evaluation of automated non-invasive sphygmomanometers have to balance the effort to perform it and the validity for the intended use. There is a long history of recommendations on how to perform clinical investigations, starting 1987 and continuing until now (AAMI 1987, PTB-A15.4, Physikalisch-Technischen Bundesanstalt 1988, O'Brien et al 1993, EN 1060-4, European Standards 2004, ISO 81060-2, International Organization for Standardization 2009, Stergiou et al 2018. The clinical investigations are performed by comparing the blood pressure values of the device under test with manual auscultatory reference blood pressure measurements by trained observers. Usually 85 subjects are included in the trial, fulfilling distributions for age and blood pressure. These trials are not ideal. but have proven over more than 3 decades to differentiate acceptable from non-acceptable devices. There has always been a desire to use simulators (Mieke 1997) to improve performance tests of automated sphygmomanometers. When simulators are able to clone the signals analysed by the automated sphygmomanometer perfectly, this would open the door to test the devices in depth. This study investigates a simulator (PTB research simulator) which claims to have this capability for automated sphygmomanometers of the oscillometric type.

Metrological aspects
The test for accuracy of a measuring device always needs a reference that is more accurate than the tested device. For the manometers used for manual blood pressure measurement this is not very demanding, and practically all metrological institutes worldwide are able to provide calibration services in this case. Much more difficult is the situation for automated noninvasive sphygmomanometer.
In metrology it is essential to provide an unbroken chain of calibrations back to national metrology standards. This is an established procedure called traceability. Metrology institutes worldwide provide primary standards for all kind of measurands. Unfortunately this classical approach is not feasible for many medical measurands such as blood pressure measured by automated non-invasive sphygmomanometers because there is no automated non-invasive sphygmomanometer as reference. Macdonald and Mieke (Macdonald and Mieke 2010) have proposed alternatives to establish traceability in such cases. The idea is to replace the primary standard either by a database of physiological test signals or one clinically tested device. In this paper the traceability based on a reference database is investigated for automated oscillometric blood pressure measurements, see table 1.

Database of oscillometric signals with clinical validation
The PTB research simulator does require a database of validated oscillometric signals for evaluating automated sphygmomanometers. The oscillometric signals were collected at Newcastle University. The EU programme 'Simulator to test automated non-invasive sphygmomanometers on compliance with the overall system accuracy defined in EN 1060-3 (FP5 European Union programmme 'Competitive and Sustainable  Growth' 2002Growth' -2005' funded most of the data collection. At present the database includes about 550 subjects, from which more than 1300 valid data sets exist. In a first step the oscillometric waveforms were collected with a recording unit, which inflated a single bladder cuff to a manually chosen cuff pressure above the systolic blood pressure of the subject, deflated it typically at a rate of 2.5 to 3.0 mmHg s −1 and deflated the cuff manually, when the both observers had detected by auscultation the diastolic pressure, usually 10 mmHg less than the diastolic pressure. The reference systolic and diastolic blood pressures were determined by auscultation with a double-headed stethoscope for 2 trained observers. For the accurate auscultatory blood pressure measurement it was necessary to stay within the deflation rate of less than 3 mmHg s −1 recommended by medical societies (AHA 2005).
In a second step the cuff inflation and deflation pressure were removed from the oscillometric waveforms by mathematically separating the oscillometric pulses from the underlying cuff pressure.
In a third step these oscillometric pulses were split into intervals, containing only one pulse associated with a certain cuff pressure range. These pulses became part of the database, associated with specific clinical criteria.

Simulators
At present the traceability for automated non-invasive blood pressure measurement does not use simulators, but uses clinical investigations, see 1.1.
Simulators are based on the knowledge that automated oscillometric sphygmomanometers analyse the pulse peaks and pulse shapes of the detected oscillometric waveform from the cuff pressure signal (Geddes et al 1982). These characteristics change during the course of blood pressure measurement, and the changes allow blood pressure to be estimated by automated devices.
Existing simulators that generate artificial waveforms use shapes that are usually nearly sinusoidal, and for which there are no clinical reference blood pressures. The pulse peaks of those artificial waveforms are created with features that easily satisfy the analysis of the oscillometric sphygmomanometer, but do not have the complex features shown by humans. These simulators therefore allow stability (repeatability, reproducibility) to be studied, but prevent accuracy being assessed (Havlík and Fabián 2020). There are several such simulators available, including Accu-Puls (Clinical Dynamics), BP Pump 2 (Fluke), SimCub NIBP Simulator (Pronk Technologies), NIBP-1020 (BC Biomedical), SECULIFE BP PRO (Gossen Metrawatt), NIBP Simulator MS200 (Contec), vPad-BP (Datatrend Systems Inc.) and UNI-SIM (Rigel Medical). These types of simulator are employed in hospital clinical engineering departments, and when used they output a number of constant artificial pulse shapes and easy-to-analyse amplitude envelopes to enable any change in any device between tests to be detected, such as every 6 months. With no change detected, the automated non-invasive sphygmomanometer can continue to be used. SunTech Medical (2010), a manufacturer of automated non-invasive sphygmomanometers, has issued a paper showing the results of their devices in combination with 5 different simulators. Although this stability check is useful, these simulators cannot check accuracy, as the artificial pressure pulse waveforms in these devices are not similar enough to human waveforms, and therefore cannot replace clinical investigations.
Simulators are being developed to enable replay of real cuff pressure pulses with shape and peak envelopes that have been pre-recorded during clinical blood pressure measurements. These developments have primarily been undertaken by research bodies and metrological institutions with recognised device test houses that want to use simulators to perform accuracy tests in their laboratories. These simulators will also help by widening the range of subjects for use in clinical investigations, and increase the number of subjects in some subgroups, such as neonates. Institutions in Germany (National Metrology Institute, PTB), South Korea (Korea Research Institute of Standards and Science, KRISS), and Taiwan (Industrial Technology Research Institute, ITRI) have published limited results from some automated blood pressure devices (Hung et al 2007, Riedel et al 2011, Doh et al 2016. They used, respectively, the original PTB simulator (Riedel et al 2011), or a development of the PTB simulator or in one case an independent development. These have been limited assessments, primarily observing the simulator replaying to devices under test. Continued developments will ultimately allow all automated oscillometric sphygmomanometers to be evaluated with agreed and recognised extended data, enabling scientific comparisons (Gersak et al 2021). To encourage routine device testing there needs to be further assessment of the accuracy of reproduction of the clinical waveforms.
The PTB research simulator used for this study has been designed and built as part of an EU funded project (Competitive and Sustainable Growth, G6RD-CTC-2002-0076) Zheng et al 2009, Fahd 2012). This paper is the first to investigate technically, how accurate the signals of the database are generated. Our study is following the ideas and intentions of the technical standard ISO/ TS 81060-5, International Organization for Standardization (2020).
To our knowledge there has been no detailed comparison of simulator oscillometric pulse pressure outputs with the original recorded cuff pressures for their accuracy of reproduction. This is the goal of this paper. The PTB simulator investigated in this study replays clinical oscillometric data recorded from human subjects, and this paper evaluates the repeatability and accuracy of the reproduction of the waveforms generated by the simulator, allowing comparison with the original recordings and the establishment of metrology for the oscillometric blood pressure measurement.

Description of research simulator
This section describes the design of the PTB research simulator, and then in section 3.1, how it was used in this study. The simulator superimposes oscillometric waveforms from its database onto any cuff pressure produced by a blood pressure measurement (BP) device. This is technically complex, because of the need to generate waveforms with maximum amplitudes less than 5% of the cuff pressure; typically with pulse maxima around 1%-3% of the cuff pressure. The research simulator works with any oscillometric sphygmomanometer, independently of its cuff inflation or deflation format or speed, including with step changes. Since the measurement procedures for different BP devices are so diverse, the oscillometric waveform output from the simulator is determined by the cuff pressure produced by the BP device, which enables the selection of a pulse recorded at the same cuff pressure from the original reference recording.
The main components of the simulator hardware are a diaphragm pump and a lever system connected to a DC servomotor to produce the required oscillometric pressure waveform (figure 1). A pressure sensor in the pneumatic system connected to the BP device measures cuff pressure, and uses any selected oscillometric pressure waveform from the simulator's database of pre-recorded waveforms. The simulator reproduces the cuff pressure pulse waveform. Since the waveform is depending on the cuff pressure, the software of the simulator has to select pulse waveforms, representative for the actual cuff pressure generated by the sphygmomanometer under test. If the cuff pressure remains at a constant level, such as during any step of stepwise deflation of the cuff pressure, the pulse peak and shape will remain constant for each pulse for the duration of the step. The user may choose from the database different waveforms collected from different human subjects that represent different clinical conditions or subject groups. More details on technical aspects are given in (Riedel et al 2011). The method used in the simulator is shown in outline in figure 2.
3. Method for determining simulator repeatability and accuracy 3.1. Obtaining simulator pulse outputs from recorded pulses For the study presented in this paper, to assess the repeatability and accuracy of oscillometric pressure pulse reproduction, the simulator was pneumatically connected to a 200 ml enclosed volume equivalent to a Figure 1. Photo of the main components of the PTB simulator hardware. The rotation of the motor is transformed into a displacement of the diaphragm via a strip and a lever. The lever has a length of 20 cm. The displacement of the diaphragm generates the oscillometric pressure pulse in the pressure chamber and the connected pneumatic system. cuff air volume, and pulses reproduced for specific recorded cuff pressures, which was set manually for the simulator, enabling a series of repeat pulses to be studied. Oscillometric pulses to be generated were replayed from selected stored data, and the pressure pulses produced in the enclosed volume were recorded to a computer for offline comparison with the original stored waveforms. A sequence of reproduced oscillometric pulses is shown for a constant cuff pressure in figure 3.

Clinical data recordings
Oscillometric pressure pulse waveforms were obtained from cuff pressures that had been recorded from volunteer subjects to a computer via a pressure sensor and the computer's analogue-to-digital converter with a sample rate at 2 kHz. The study received ethical approval from the Newcastle and North Tyneside (UK) Research Ethics Committee, and subjects gave signed informed consent. Recordings were made during cuff deflation at approximately 3 mmHg s −1 .

Selection of oscillometric pulses for simulator evaluation
For evaluation of the simulator, clinically recorded oscillometric pulses were selected to span a range of different cuff pressures, heart rates, and pulse shapes. Seven pulses from different recordings were identified spanning cuff pressures from 72 to 121 mmHg. The pulse rates ranged from 55 to 96 min −1 , and pulse peaks from 1.1 to 3.6 mmHg. The characteristics of each pulse are given in table 2, and the different pulse shapes are shown in the Waveform column. The shape of the waveforms from above systolic to below diastolic blood pressure changes (Geddes et al 1982). The waveforms no.1 and 2 are Figure 2. Simplified workflow of the simulator. After the operator has selected, from a single subject, a data set for simulation, the simulator starts a closed loop procedure. Depending on the cuff pressure detected by the pressure sensor of the simulator, the appropriate waveform is selected from the database and generated by the diaphragm system at the pulse interval of the subject during the data recording, and superimposed on the cuff pressure output from the blood pressure device under test. close to the diastolic blood pressure, no. 6 and 7 are close to systolic blood pressure. No. 3, 4 and 5 are waveforms between diastolic and systolic blood pressure. The waveforms were taken from different subjects to cover different pulse rates and peaks to include these parameters in the test. Cuff pressure differed over the seven pulses studied with the higher cuff pressures tending to be more biphasic.

Simulation of clinical reference oscillometric pressure pulses
The simulator was used to output the pressure pulses of each reference oscillometric pulse ten repeat times, as in figure 3, and the resulting oscillometric pressure pulses reproduced by the simulator were captured via a pressure sensor and an analogue-to-digital converter (sample rate 2 kHz) to a computer for offline analysis. Table 2. The characteristics of each reference clinical oscillometric pulse used in this study to assess the simulator repeatability and accuracy are presented. They are ordered from a low to a high cuff pressure. The cuff pressures, pulse rates, pulse intervals and pulse peaks are given. The waveforms are normalised for pulse amplitude to allow easy comparison.

No.
Waveform ( In total there were 70 simulated pressure pulses used for this analysis.

Comparison of simulated pulse amplitudes and shapes with the clinical reference oscillometric pulses
The repeatability of reproduction of each pulse was assessed from each pulse's 10 repeat simulations. Pulse peak repeatability was obtained from the SD of the 10 repeats, and also averaged over the 7 pulses. For pulse shape repeatability, the SD of all sample differences between the simulated pulse and the average simulation waveform of all 10 simulations, was calculated over all 10 simulations, and then also averaged over the 7 pulses. Accuracy of reproduction was calculated as for repeatability, but also including the mean difference, with the comparison always using the reference pulse peak and reference pulse shape, rather than the simulated pulse. Both repeatability and accuracy were calculated as absolute values in mmHg, and as a percentage of the recorded reference pulse peak. Finally, the correlations between each simulated pulse and the original reference pulse were determined as an added estimate of accuracy.

Database of validated oscillometric signals
The oscillometric signals were collected by a recording unit. It recorded the data with a sampling rate of 2 kHz and a resolution of 14 bit. The frequency range of the recording unit including pneumatic tubing and cuff was 0.5-12 Hz, which included the 4th harmonic wave of the fundamental wave (0.5-3 Hz) of the oscillometric signal. The standard uncertainty of the reference blood pressure u(p ref ) is identical to the uncertainty of the recording unit u(p ru ). The uncertainty of the recording unit was determined in the project 18RPT02 adOSSIG of the European Metrology Programme for Innovation and Research (EMPIR). Based on the calibrations performed before and after the waveforms were recorded by the recording unit (RU), it is:

Simulation repeatability of pulse peak and pulse shape
Repeatability was determined as the SD of ten repeat simulations. Repeatability results for each set of repeat  Table 4. Simulated accuracy calculated by comparing the simulated pulse to the reference pulse for all 10 simulations of each pulse, with the mean and SD of each set of reference-simulation differences. There were no significant differences (p>0.05) between the simulation and reference for the pulse peak or pulse shape.
(a) Pulse peak (b) Pulse waveform shape simulations are shown in table 3, for both pulse peak and pulse shape. The repeatability measure stayed within 1.9% for the simulated pulse peak and within 2.3% for the shape, with mean repeatability values for peak and shape 1.0% and 0.8% respectively.

Simulation accuracy of pulse peak and pulse shape
The results for simulation accuracy are given in table 4. The average for all pulses resulted in an accuracy for the simulation of the pulse peaks of 97.8% and for shape 95.4%. The simulated pulse peaks were highly correlated with the reference pulse peaks (figure 4), as would be expected from the results in table 4.
It is interesting to note in table 4, that the absolute difference for the pulse peaks is always less than 0.07 mmHg, and the absolute difference for the pulse shapes always less than 0.19 mmHg. For shape these differences tended to appear in the 2nd half of the pulses with a decreased amplitude in the pulse tail, see figure 5.

Simulation accuracy by correlation analysis
The correlation coefficients between recorded reference and simulated waveforms are given in table 5. The mean correlation coefficient was 0.994 with an average SD of 0.002, with all correlations lying very close to unity.

Measurement uncertainty of the PTB simulator
For the estimation of the uncertainty according to the Guide to the expression of uncertainty in measurement GUM (2008) of the simulator using the clinical data, only the quantities having the main impact are taken into account, these are: • p pa pulse amplitude (table 4).
The data are shown in table 6 in comparison with the reference pulse The drift and temperature effects of the pressure transducer as well as environmental effects are not taken into account because the pressure transducer is calibrated frequently.
The combined uncertainty u(p sim ) c of the pressure pulse generated by the simulator is: where u(p pa ) uncertainty of the pulse amplitude, u(p ps ) uncertainty of the pulse shape, u(p ref ) uncertainty of the reference blood pressure. The value of the extended uncertainty U(p sim ) = 0.3 mmHg (k = 2, p = 95%, normal distribution) for the pulses generated by the simulator is only 0.03 mmHg higher (formula 2) than the one for the clinical reference data (formula 1). This shows that the contribution of the hardware and software of the simulator is almost negligible for this uncertainty consideration.

Conclusion
Oscillometric sphygmomanometers analyse the amplitude of the pulse oscillations superimposed on the cuff pressure. An overview of different methods can be found in (Chandrasekhar et al 2019). Since the details Figure 5. Column 2 shows all simulated pulses (continuous lines) and the reference pulses (dashed lines). Column 3 shows the differences between the simulated and reference pulses. Column 4 shows the correlations between the simulated and reference pulses. All plots are normalised to the maximum of each reference waveform amplitude.  Liu et al (2013Liu et al ( , 2016 and Raamat et al (2013) have shown that including the pulse shapes may be a way to improve the oscillometric method. Thus the pulse shapes are becoming an important parameter for testing the accuracy, repeatability and reproducibility with simulators. The international ISO document (2020) 81060-5 addresses the repeatability and reproducibility of the pulse shapes but not their accuracy. Doh et al (2016) reported an accuracy of 5% for the amplitude of the KRISS simulator; an accuracy for the pulse shape is not given.
ITRI from Taiwan used a simulator (Hung et al 2007) which replayed pre-recorded human pulses. It is basically a copy of the PTB simulator. They showed for 2 signals, that the accuracy of it stays within + 1 mmHg and − 4 mmHg, when comparing the results of measurement on humans with the same signals generated by the simulator. The blood pressure values were measured by a commercial automated sphygmomanometer (Microlife BP 3BTO-A). Gersak et al (2009) recommends that 'the repeatability of maximal values of oscillometric pulses has to be at least in the order of a couple of 0.01 mmHg over a period of several seconds'. Our results show an average value of 0.02 mmHg (0.01-0.03 mmHg) for this parameter.
To our knowledge this study is the first to investigate in detail the pulse shapes generated by a blood pressure simulator, with comparison of the simulated outputs to the original recorded pulse shapes for repeatability of reproduction and for accuracy of reproduction. Oscillometric pulses spanning the clinical range of cuff pressures, pulse intervals and pulse shapes encountered during oscillometric blood pressure measurement were studied. Any future extended database for use by this simulator will include a wide range of oscillometric waveforms in consultation with cardiologists.
The reproductions by the simulator showed only small pulse differences between the simulated pulses and the original reference pulses. For pulse peaks the difference had an average of only 0.02 mmHg, which was less than 1% of the pulse peak, and for shape the average difference was also 0.02 mmHg. In addition, the correlations between the reference and simulated pulses were very close to unity at 0.99.
The results show the potential of the PTB simulator. It can be used to substitute for clinical investigations in evaluating this approach for medical device regulatory requirements. Our results do encourage such a study. This would open the potential for substantially increasing the subject numbers and the range of clinical conditions, as volunteer subjects for each device evaluation would no longer be required, and oscillometric recordings would need only to be pre-recorded Under metrology aspects the calculation of the measurement uncertainty shows that the dominating quantity is the uncertainty of the clinical reference data, and not the hardware or software of the simulator.
Our technical tests show very good results for repeatability, reproducibility and correlation of the generated pulses for the PTB simulator. The extended uncertainty (with k = 2 ), for the pulses generated by the PTB simulator, were determined to be U(p sim ) = 0.3 mmHg. This is dominated by the uncertainty (64%) of the clinical reference data, and not by the hard-and software of the simulator. Considering the concept explained in section 1.2 regarding the traceability of oscillometric sphygmomanometers, these results encourage the use of this simulator as a secondary standard.