Clinical precision, accuracy, number and durations of exhalations for a novel electrochemical monitor for exhaled nitric oxide

Exhaled nitric oxide (FeNO) is an established marker of airway inflammation in allergic asthma. The major benefits of using FeNO include helping diagnose asthma, predicting and confirming the response to medications that target eosinophilic inflammation and identifying suboptimal compliance with maintenance asthma medications. The first NO analyzers used to measure FeNO were ozone chemiluminescent analyzers with a rapid enough response time to display real-time exhalation profiles of NO versus time termed 'online measurement'. A typical NO versus time profile has an initial washout phase where NO rises as the anatomical dead space is cleared followed by manual methods or software to select the steady state washout plateau which is taken as the FeNO measurement. Subsequently, the technical complexity, size and cost of chemiluminescent analyzers resulted in the development of alternative methods of NO detection predominantly based on electrochemistry. Electrochemical sensors have slower response times, and therefore are unable to be used for online measurement. Instead, a sample from a qualifying exhaled breath is taken into the sensor chamber, and a FeNO result is generated after some nominal delay.

The American Thoracic Society (ATS) and European Respiratory Society (ERS) guidelines from 2005 [1] were written principally for chemiluminescent analyzers, and included a recommendation that repeated, reproducible exhalations should be performed to obtain at least two NO plateau values that agree within 10% of each other. Exhaled NO is then calculated as the mean of two values. Additionally, the duration of exhalation was recommended to be at least 4 s for children >12 years and >6 s for children >12 years and adults. The guidelines also recognized that a 10 s exhalation could be necessary for a FeNO measure in some patients.

An ERS technical standard on exhaled biomarkers from 2016 states that the ATS/ERS guideline recommendation for two replicate measurements is still valid, although it was appreciated that if only one measurement could be performed owing to financial or other constraints that this could provide valuable data [2].

A novel electrochemical FeNO monitor, Fenom Pro^TM, Spirosure Inc., Pleasanton CA, was recently cleared in the USA by the Food and Drug Administration.Version 3 of Fenom Pro^TM has CE marking for commercialization in the EU, is pending approval in the USA, and was used in the studies in this report. Studies were performed for precision, agreement with the NIOX VERO^® (Circassia, Morrisville, NC, USA) and the impact of single versus replicate exhalations, as well as 6 versus 10 s exhalations, and are presented herein.

Study design

The primary objective was to evaluate the clinical precision of the Spirosure Fenom Pro^TM device in adult and pediatric subjects. The secondary objectives were to compare exhalations of 6 and 10 s for the Fenom Pro^TM device, to perform a method comparison of the Fenom Pro^TM and the NIOX VERO^® devices, and to compare the first measure versus the average of two FeNO measures for the Fenom Pro^TM device.

Study devices

Principal inclusion and exclusion criteria

Safety observations

• 1.  
  Subjects performed repeated exhalations to obtain two valid 10 s exhalations on the Fenom Pro^TM device.
• 2.  
  Subjects performed repeated exhalations to obtain two valid 6 s exhalations on the Fenom Pro^TM device.
• 3.  
  Subjects performed repeated exhalations to obtain a single valid exhalation on the NIOX VERO^® device.

Using simulated human breath humidified samples, a benchtop comparison was made between the Fenom Pro^TM to an ozone chemiluminescent analyzer, the Sievers NOA 2801 instrument (Zysense Inc., NC, USA), which was calibrated according to manufacturer's instructions. Six concentrations were measured across a range of 200 ppb.

Primary Objective

The study was performed at three clinical research sites in the USA. The study was approved by a central institutional review board (Aspire IRB, Santee, CA), and all subjects or caregivers for children under 12 years of age signed an informed consent. Children of sufficient capacity also signed an assent form approved by the IRB.

One hundred and twenty-six subjects were enrolled, with one screen failure occurring due to current enrollment in another investigational study. Of these, an eight year old Caucasian male was unable to provide a valid exhalation on any of the three FeNO devices and thus was excluded from the per protocol (PP) population (n = 125). Data from the intent-to-treat population (ITT; n = 126) is presented henceforth. Of the 126 enrolled, 124 (98.4%) completed all study procedures. No major protocol violations occurred during the study, and there were no SAEs.

Table 1 presents the demographic variables for the ITT population. The population included 43 subjects (34.1%) in the pre-specified 5–17 age group, with 16 subjects under age 12, and 83 (65.9%) in the ≥18 years age group. The study achieved the target minimum enrollment of 40 children 5–17 years of age, as well as 12 subjects under 12 years of age.

Distribution by sex was 39.7% male and 60.3% female. Caucasian race represented the majority of the population (73.0%) with 15.9% Black or African American, 4.8% American Indian or Alaskan Native, 4.0% Asian and 1.6% two or more races. A total of 15.1% of subjects self-identified as Hispanic or Latino.

Table 2 presents the clinical within-subject precision of FeNO as measured on the Fenom Pro^TM in the ITT population using a 10 s exhalation. Only 124 out of the 126 ITT subjects provided replicate measures. The table presents the N within each range of median FeNO concentrations, the within-subject mean of the SDs of the measurements, the within-subject mean of the calculated % CVs of the measurements and the associated two-sided 95% CIs for both the SD and %CV parameters. The population SDs vary by category of median FeNO concentration with higher SDs resulting from higher median concentration categories. However, the %CV results display an acceptable performance within each concentration category with point estimates ranging from 4.21%–9.25%, with lower %CVs generally corresponding to higher concentrations, as would be expected for a continuous scale quantitative variable. The 95% CIs by concentration category vary according to the N within each category, with wider CIs resulting from categories with lower N, as expected. Nevertheless, the upper bounds of the CIs remain below 12% in all concentration categories, indicating excellent clinical within-subject precision of FeNO measurements on the Fenom Pro^TM device.

Tables 3 and 4 present the clinical precision results in the pediatric and adult sub-cohorts of the ITT population using the 10 s exhalation. In the pediatric sub-cohort, several FeNO median concentration categories display N's ≤ 12 (10 to <20, 20 to <30, 30 to <40, 40 to <50, and ≥50 ppb). In these latter categories, the upper bound of the 95% CI of the %CV exceeds 12% with a maximum of 26.73% for the 10 to <20 ppb category. However, the point estimates for all FeNO categories are below 7% ranging from 4.44%–6.63%, except for the 10 to <20 category (15.74%).

For the adult sub-cohort, point estimates range from 2.93% in the next-to-highest FeNO concentration category (40 to <50 ppb) to 8.58% in the lowest category (0 to <10 ppb). In this sub-cohort, the upper bounds of the 95% CI for the %CV tend to display higher results with lower N with the highest upper bound of 31.97% occurring in the lowest FeNO category and at the lowest observed N of four subjects within the category. With the limitations of the number of subjects and focusing on the point estimates, these data demonstrate excellent clinical precision.

Tables 5–7 present the clinical precision results using Fenom Pro^TM for the combined ITT population (table 5) and the pediatric and adult sub-cohorts for a 6 s exhalation (tables 6 and 7. In general, the observed precision appears similar to the 10 s exhalation results with point estimates ranging from 4.29%–10.46%. Again, higher FeNO values tend to provide better precision estimates as do those FeNO categories with more subjects since both factors serve to narrow the confidence intervals and provide more accurate point estimates. Of note, the results for the pediatric sub-cohort indicate slightly better precision for a 6 versus a 10 s exhalation with point estimates ranging from 2.52%–8.14%.

Figure 1 presents a Deming regression comparing the first valid 6 s exhalation FeNO value (y-axis) to the first valid 10 s exhalation FeNO (x-axis) for the Fenom Pro^TM in the combined adult and pediatric ITT population.

Zoom In Zoom Out Reset image size

Table 8 presents the point estimates for the slope and y-intercept and their associated 95% CIs, as well as point estimates for the correlation coefficient (R), and the standard error of the residuals (Sy∣x) for the comparison of 6 and 10 s exhalations in the combined adult and pediatric ITT population. The results demonstrate a tight quantitative agreement between the two exhalation times, with slope point estimate = 1.025 (95% CI: 0.9590–1.091), y-intercept = −0.4189 (95% CI: −2.306 to 1.468 ppb), R = 0.978 and Sy∣x  =  6.3 ppb. Of note, the 95% CIs for the slope and y-intercept include the identity targets of 1.0 and 0.0, respectively, thus indicating the interchangeability of the two exhalation times.

Figure 2 presents the results of the Bland–Altman analysis of paired differences (y) versus mean FeNO concentrations in ppb (x) in the ITT population for 6 versus 10 s exhalations on Fenom Pro^TM. The points generally distribute evenly around the line of identity (0.0 ppb bias) but without any visual trend of either increasing or decreasing deviation with increasing measurement range. The mean bias is 0.4 ppb with 95% limits of −1.9 to 12.7 ppb. Since the 95% limits of the agreement include 0.0 ppb, the data indicate substantive agreement between the 6 and 10 s exhalations on the Fenom Pro^TM in the ITT population.

Zoom In Zoom Out Reset image size

There was also a high level of agreement for 6 versus 10 s exhalations, as shown in table 9 and comparable to the data in the combined ITT population.

Figure 3 presents the Deming regression comparing the first valid 10 s exhalation ppb values (y-axis) for the Fenom Pro^TM to the NIOX VERO^® ppb values (x-axis) in the ITT population. The plot presents the individual points, the fitted regression and the line of identity.

Zoom In Zoom Out Reset image size

Table 10 presents the point estimates for the slope and y-intercept and their associated 95% CIs, as well as point estimates for the correlation coefficient (R), and Sy∣x comparing the first valid 10 s exhalations on the Fenom Pro^TM to the NIOX VERO^®. The results demonstrate a tight correlation between the two devices (albeit with slightly higher values in the Fenom Pro^TM) with a slope point estimate = 1.149 (95% CI: 1.077–1.221), y-intercept = −1.211 (95% CI: −2.911 to 0.4895) ppb, R = 0.981 and Sy∣x = 5.7 ppb.

Figure 4 presents the results of the Bland–Altman analysis of paired differences (y) versus mean FeNO concentrations in ppb (x) in the ITT population comparing the first valid 10 s exhalations on the Fenom Pro^TM to the NIOX VERO^®.

Zoom In Zoom Out Reset image size

The points generally distribute evenly around the line of identity (0.0 ppb bias) with a slight trend toward increasing positive differences with increasing measurement range. The mean bias is 3.2 ppb with 95% limits of −9.6 to 16.0 ppb, with slightly higher values for the Fenom Pro^TM. Since the 95% limits of the agreement include 0.0 ppb, the data nevertheless indicate substantive agreement between the Fenom Pro^TM and NIOX VERO^® in the ITT population.

Figure 5 presents the comparison of the Fenom Pro^TM to the Sievers NOA 280 analyzer over the 200 ppb NO concentration range. The fitted line is Fenom Pro^TM = −1.147 024 + 1.032 8914 * Sievers 280i, demonstrating a tight agreement between the two analyzers.

Zoom In Zoom Out Reset image size

The first valid 10 s exhalation was compared to the average of the first two valid 10 s exhalations on the Fenom Pro^TM to evaluate agreement. The ATS 2005 guidelines recommend evaluating the average of two exhalations [1].

Figure 6 and table 11 present the Deming regression for one measure versus the average of two measures. The points generally distribute evenly around the line of identity (0.65 ppb bias) with a slightly observable trend for increased divergence with increasing measurement range. The mean bias is 0.65 ppb with 95% limits of −3.66 to 4.96 ppb. The results demonstrate a tight correlation between one and the average of two exhalations with a slope point estimate  =  1.026 (95% CI: 1.012–1.041), y-intercept = −0.2096 (95% CI: −0.7230 to 0.3038) ppb, R = 0.998 and Sy∣x = 1.8 ppb. The data indicate substantive agreement between the one and the average of two exhalations in the ITT population. Figure 7 presents a Bland–Altman plot with a mean difference of 0.65 ppb and 95% limits of agreement ranging from 3.19 − 4.49 ppb, and as this includes 0, this supports the substantial agreement for the comparison.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

One versus two exhalations on Fenom Pro^TM for the separate adult and pediatric ITT populations

Two of 126 subjects (1.6%) in the safety population experienced self-limited mild AEs, headache and lightheadedness. The AE for lightheadedness was judged related to the investigational device and appeared associated with repeated breathing into the machine. The subject recovered within 2 min with no adverse sequelae.

This study presents for the first time device performance for a recently cleared electrochemical office-based FeNO device, Fenom Pro^TM. The primary objective was to evaluate the clinical precision of within-subject FeNO measurements using the Fenom Pro^TM device for a mixed study population of pediatric subjects (ages 5–17 years) and adults. The secondary objectives were to evaluate the agreement between 6 and 10 s exhalations as the former would be easier for subjects to perform. In addition, one measure versus two measures were compared to see if one measure is adequate for routine practice. Finally, a method comparison was performed against the NIOX VERO^®, a commonly used marketed FeNO device.

Clinical precision for the combined, adult and pediatric ITT populations for 10 s exhalations was excellent with within-subject SDs ranging between 0.56–3.73, within-subject mean CVs ranging from 4.21%–9.25% with maximum upper 95% Cl of 11.92%. The Fenom Pro^TM and NIOX VERO^® are both electrochemical portable devices which report results in a short timeframe. Comparison of the clinical precision of the Fenom Pro^TM to publicly available precision data for the NIOX VERO^® (www.niox.com) indicates that the clinical precision is essentially the same for the combined and separate adult and pediatric ITT populations; the clinical precision was comparable.

For the Fenom Pro^TM device, our results also indicate comparable clinical precision for 6 s exhalations to 10 s exhalations for the combined and separate adult and pediatric ITT populations, and importantly, better precision for 6 versus 10 s exhalations in children. Secondly, the comparison between 6 and 10 s exhalations demonstrated very close agreement in the combined and separate adult and pediatric ITT populations. Thus, 6 s exhalations are suitable for wide application in adults and pediatric populations. This will convey the benefit of shorter procedures and perhaps superior comfort for patients (less effort). We observed a much higher success rate with 6 versus 10 s exhalations as (72.8%) achieved success in two attempts on the Fenom Pro^TM with a 10 s exhalation versus 93.6% with a 6 s exhalation (data not shown).

Lastly, we have convincingly demonstrated that one valid exhalation is in substantial agreement with the mean of two valid exhalations in both the combined and separate adult and pediatric ITT populations using Fenom Pro^TM. The ATS recommendations from 2005 recommend performing two valid exhalations that agree within 10%. However, the recent ERS technical standard proposes that one valid exhalation is adequate [2]. Our data support this proposal. The ability to perform a single exhalation will convey advantages, including a shorter overall FeNO test time, less procedures for patients and easier workflow in the clinic.

The method comparison of the Fenom Pro^TM to the NIOX VERO^® indicates a high level of agreement with a mean bias of 3.2 ppb (range −9.6–16.0) in the ITT population, and these confidence limits include zero, indicating substantive agreement between the two devices with slightly higher values on average for the Fenom Pro^TM.

On the issue of 6 versus 10 s exhalation, this may prove of particular importance for young children. Ito et al compared 6 and 10 s exhalation in children aged 4–15 [3]. Median FeNO-10 (29 ppb [IQR 15.2–42.0]) and FeNO-6 (27 ppb [IQR 16.0–43.5]) were not significantly different (P = 0.90), with excellent correlation between both values (r = 0.984, P < 0.001). The mean bias was −0.151 ppb in 46 asthmatic children (median age 7 years [range 4–15]) with no experience of FeNO measurement. While all children aged 8 years and more (n = 21) completed FeNO measurement for both durations, for children <8 years (n = 25) the success rates were 60.0% (10 s) and 92.0%, (6 s), respectively. Rickard et al reported very similar findings in children aged 7–10 years old with a median bias of 0.5 ppb [4]. Thus, our demonstration of the excellent precision and equivalence of the 6 to the 10 s exhalation agrees with these reports [3] and opens the way to application of great relevance to very young children. Ten seconds of exhalation at 50 ml s⁻¹ leads to a rapid deflation of the lung in very young children, which is the most plausible explanation of the lower success rate compared to 6 s exhalations.

There have been multiple [5–18] reports comparing different FeNO monitors, including diverse sensor technologies. Early studies focused on comparison of electrochemical sensors to ozone chemiluminescent analyzers, which are regarded as the 'gold standard'. Alving et al compared the NIOX MINO® (a handheld electrochemical device) to the NIOX^®(a chemiluminescent device), and reported that the mean bias was −1.2 ppb with higher readings for the NIOX MINO^® and 95% limits of agreement ranging from −9.8 to 8.0 ppb [19]. Boot et al compared the NIOX MINO^® to the Ecomedics^® ozone chemiluminescent device, and reported a high correlation (r = 0.975, p < 0.0001) but the mean bias was −10% with the NIOX MINO^® demonstrating lower values but the 95% confidence limits were broad ranging between −36% to +28% [5]. Inoue compared the NIOX VERO^® to NObreath^®, two handheld electrochemical analyzers, reporting a high correlation (r = 0.92) but differing agreement for low FeNO and high values with measurements on the NObreath^® higher for levels of FeNO > 58 ppb but lower for FeNO, 58 ppb compared to the NIOX VERO^® [9]. Molino et al compared the NIOX VERO^® (Circassia, Morrisville, NC, USA), the Vivatmo^® PRO (Bosch, DE) and the HypAir FeNO^® (Medisoft, Dinant, Belgium) [15]. The mean FeNO values (95% CL) were 24.0 (18.6–29.4) ppb for the NIOX VERO^®, 19.6 (13.6–25.7) ppb for the Vivatmo PRO and 20.4 (15.7–25.1) ppb with the HypAir FeNO^®. The mean difference between pairs of analyzers varied between −0.7 and 4.3 ppb with the upper 95% CL ranging between 15.0 and 25.7 ppb, with the NIOX VERO^® higher than the other two analyzers. In summary, when comparing devices, in general, the levels of correlation are high, but there may be significant differences between the absolute values obtained on one analyzer versus another. The reasons for the between-analyzer differences are probably multifactorial but could include divergent sensor technologies and calibration reliability, sensor drift and different precision in controlling an exhalation flow rate of 50 ml s⁻¹, as FeNO is markedly flow dependent [20]. This means that analyzers cannot be assumed to be interchangeable, although the between-analyzer differences are small. Certainly, for clinical research in a multisite setting, the same FeNO monitors would be a sensible requirement.

The limitations of the study are as follows. We only compared the Fenom Pro^TM to the NIOX VERO^® as this is the most prevalent FeNO meter in use in the USA, but comparison to other FeNO devices would add more information for users outside the US. Also, we did not include a comparison to ozone chemiluminescent analyzers as these are not widely used in the physician offices where the study is performed, but we have included a benchtop comparison between the Fenom Pro^TM and Sievers 280i devices. While the sample size was adequate for the purpose of this study, we had smaller numbers of subjects with FeNO values in the <10 ppb and 40–50 ppb and >50 ppb ranges. Also, the number of pediatric subjects under 12 and aged 5 and 6 was low. Finally, this data was collected in stable adult and pediatric subjects. However, we have also evaluated the Fenom Pro^TM analyzer in unstable pediatric and adult subjects before and after corticosteroid therapy, representing a more diseased population and demonstrating a marked decline in FeNO after this intervention (data on file).

In summary, FeNO measurements on the Fenom Pro^TM monitor demonstrated excellent clinical precision for both adults and children, whether for a 10 s exhalation or a 6 s exhalation, and the 6 s and 10 s exhalations show close agreement. Furthermore, the first valid exhalation on the Fenom Pro^TM demonstrates good agreement with the average of the first two exhalations across adult and pediatric subjects. Lastly, the Fenom Pro^TM FeNO values are in good agreement with the NIOX VERO^® device.

We wish to acknowledge the following individuals:

Clinical investigators: Alexander Greiner MD, Asthma and Allergy Medical Group and Research Center, San Diego California, Frank Hampel MD, Central Texas Health Research, Texas, and David Bernstein MD, Bernstein Clinical Research Center , Cincinnati, OH.

Vision Clinical Research: San Diego California: Michael Lichtman, Guy Gammon, MD.