Evaluation of the standard test methods for optical coherence tomograph for the posterior segment of the human eye

Optical Coherence Tomography (OCT) has revolutionized retinal imaging by offering non-invasive high-resolution three-dimensional visualization capabilities. OCT has become the standard of care in routine ophthalmological practice, especially for the posterior segment. Given its widespread clinical applications, establishing standardized test devices and methods for key OCT parameters is imperative to ensure both optimal imaging performance and diagnostic accuracy and treatment effectiveness. As a widely applied standard, ISO 16971:2015 published by the International Organization for Standardization specifies the minimum requirements, test device, and methods for OCT for the posterior segment of the human eye. Notably, these standards lacked experimental validation. In the present study, we implement the test device according to ISO 16971:2015, and assess a commercially available ophthalmic OCT instrument with the suggested test device and methods. Results show that the test device and methods could facilitate a rudimentary evaluation of OCT key parameters. Nevertheless, refinements of the test device and methods are requisite to enhance measurement accuracy, reliability, traceability, and practicability, catering to the diverse needs of manufacturers, end-users, and regulatory entities.


Introduction
In 1991, optical coherence tomography (OCT) was first demonstrated by David Huang et al as a novel highresolution noninvasive cross-sectional imaging technique using images of the retina and coronary artery [1].Since its invention, OCT has been broadly applied in ophthalmology, cardiology, gastroenterology, and dermatology [2][3][4].In ophthalmology, OCT has become the standard of care, especially for diagnosing and monitoring ophthalmic diseases in the posterior segment, such as age-related macular degeneration, diabetic retinopathy, glaucoma, retinal detachment, and so on [5][6][7][8].Precise physiology parameter measurements provided by ophthalmic OCT are important for ophthalmic disease diagnosis and treatment, for example, the thicknesses of the retinal nerve fiber layer and the macula are pivotal in the diagnosis and management of optic nerve head and macula anomalies, respectively.
For the development and manufacture of OCT instruments, the establishment of standardized test devices and methods for critical specifications is paramount to ensure the accuracy, uniformity, and reproducibility of OCT measurements.In addition, clinical OCT instruments are subject to continuous wear, degradation, and environmental variations, potentially compromising measurement accuracy over time.Consequently, proper maintenance, routine inspection, and periodic calibration of the OCT instruments are essential to uphold the precision of OCT measurements and the overall quality of medical services.This upkeep relies on reasonable and standardized test devices and methods [9].Furthermore, from a regulatory perspective, reasonable, applicable, and traceable standards are required so that the quality of OCT instruments on the market can be ensured and monitored [10].
Over the years, many efforts have been made to develop appropriate phantoms to characterize the performance of OCT.Various sub-resolution particles, such as gold nanoshells, iron oxide particles, and polystyrene microspheres have been uniformly dispersed in a transparent matrix with a different refractive index, such as epoxy resin, polyurethane, and polydimethylsiloxane (PDMS), to examine the point spread function (PSF) of an OCT system [11][12][13][14][15].Anthony Fouad et al investigated the efficacy of different types of subresolution PSF phantoms and proved the practical utility of PSF phantoms for quantitative inter-comparison of OCT system resolution and signal uniformity [16].Studies also proved the feasibility of integrating the subresolution PSF phantoms with model eyes to facilitate the assessment of ophthalmic OCT system performance [17,18].Besides sub-resolution particle phantoms, Tomlins et al demonstrated that a PSF phantom created with femtosecond laser inscription could evaluate OCT resolution performance [19].Additionally, threedimensional (3D) resolution targets were also designed and fabricated by 3D printing, soft lithography, or femtosecond laser inscription to evaluate the resolution of OCT systems [20][21][22].Besides, thin layer and thin bar chart phantoms were proven as efficacious tools for assessing the axial resolution of OCT systems [23][24][25].Beyond resolution, sensitivity is also a critical parameter, as it determines the minimum sample reflectivity that can be detected by an OCT system.Anant Agrawal et al undertook a comparative study assessing the efficacy of the conventional specular surface method, laser-inscribed phantom, and microsphere suspension phantoms in evaluating the OCT system sensitivity [26,27].Other than phantoms designed to test OCT parameters, phantoms emulating both healthy and pathological retinal structures have been introduced to gauge OCT image fidelity [28][29][30].
As a response to the pressing standardization request from the ophthalmic OCT industry and consumer stakeholders, Subcommittee 7 of the Technical Committee 172 under the International Organization for Standardization (ISO) published a standard (ISO 16971: 2015) for OCT for the posterior segment of the human eye [31].In this standard, six critical ophthalmic OCT parameters, including the angular field of view (FOV), depth scaling, lateral and axial resolution, sensitivity, and the co-alignment of fundus image with OCT scan, were defined.For the accurate quantification of these parameters, the standard suggests a specific test device and the accompanying methods.The precision and efficiency of the measurements enabled by this standard have profound implications for both the industry and the medical area.However, it is noteworthy that the proposed test device and methods within the standard were not supported with experiment results.In this work, we implemented the test device and measured all relevant parameters with a clinical ophthalmic OCT instrument, in strict accordance with ISO 16971:2015.Result images were acquired and analyzed to demonstrate the feasibility of the standard.

OCT data collection
In this investigation, we employed a commercial ophthalmic swept-source OCT instrument (Model VG200D, SVision Imaging, Ltd, Luoyang, China) characterized by a central wavelength of 1050 nm and a bandwidth spanning 100 nm.It provides OCT images with an axial resolution of 5 μm (in tissue), a lateral resolution of 20 μm, and an A-line rate of 200 kHz.The imaging depth is approximately 6.3 mm in tissue with a digital resolution of 2.0 μm.During the imaging process, spectral interferograms at evenly spaced wavenumbers were obtained.Subsequent data processing steps included background subtraction, spectral reshaping (windowing), dispersion compensation, and Fourier transformation.This series of processes yielded depth-reflectivity profiles in the complex domain, denoted as i x .[ ] ̅ ), was then used for the following OCT performance assessment.

Test device
As shown in figure 1, a test device was designed and fabricated based on the description in ISO 16971:2015.A + 60D K9 plano-convex lens with a transmittance of T l was used to simulate the optical power of a human eye, providing a focal length of 17 mm.A piece of millimeter-thick K9 glass with known and traceable refractive index n g , thickness L g , and reflectivity R g was mounted at the focal plane of the lens.In this experiment, M-scans (multiple A-lines at the same place of the sample) of the glass were taken, so that the thickness of the glass measured with the OCT instrument could be obtained.Meanwhile, with the known set tissue refractive index n 0 of an OCT instrument, the measured thickness of the glass L m could be calculated as and the depth scaling of the OCT instrument could be validated.
A modified USAF 1951 resolution target was created on the front surface of the K9 glass using a photomask, with which the lateral resolution and angular FOV of an ophthalmic OCT instrument could be measured, as shown in figures 1(b), (d).Instead of using all the components in the USAF 1951 resolution target, elements in group 2 to group 7 were chosen in this work, in which the bar width ranges from 125 μm to 2.19 μm.The six groups of markers were arranged at the center of the modified resolution target, and each group includes six elements.In each element, there are three pairs of horizontal and vertical bars.The width of the bars was traceably calibrated, as shown in table 1, to provide a reliable measurement of the lateral resolution.The outer square rim of the resolution bar region indicates a 4 mm × 4 mm FOV.Four scales extended in four directions starting from the outer square rim of the bar region.Each scale has a full length of 6 mm and an increment of 100 μm, providing coverage of 4 mm × 4 mm to 16 mm × 16 mm FOV.With the maximum dimension of the OCT image field (i.e.2r) measured using the scales and the known distance L between the exit pupil and the target plane, the angular FOV of the ophthalmic OCT instrument was calculated as To prevent detector saturation caused by the specular reflection from the glass, a neutral density filter with a known transmittance rate of T f was mounted between the lens and the glass.
The full width at half maximum (FWHM) of the intensity of a reflection I r from the non-marker region of the glass front surface was calculated to assess the axial resolution of the OCT system.The sensitivity of the tested OCT was calculated as [27]: in which I r is the intensity of the reflection peak, I bg is the averaged intensity of the background noise at the same depth x r of the reflection peak with the sample arm being blocked, T f and T l are squared to account for the double pass of light through the neutral density filter and the lens.In ISO 16971:2015, the requirement for the co-alignment of the fundus image and OCT scan is ±100 μm.According to this requirement, a 200 μm diameter filament can be employed as a testing tool, instead of the 100 μm filament described in the standard.In this  research, to examine the co-alignment of the fundus image and OCT scan, a piece of bare single-mode fiber (SMF 28) with an outer diameter of 125 μm was mounted above the glass.

Test experiment
A fixture with two degrees of freedom was designed to hold the test device in a proper position relative to the OCT.The tested commercial OCT was adjusted so that the front surface of the glass in the test device was at its focal point.OCT images were collected as described in 2.1 and analyzed for all parameters as shown in table 2.

Angular FOV and lateral resolution
Figure 2 shows the 3D view of the modified resolution target with the FOV scales and its en-face image.It is noticed that the flat glass with the modified resolution target and scales appears as a curved panel with the center being higher than the periphery in the 3D view.Because the ophthalmic OCT instrument is designed for imaging the human eye fundus and its focal plane is a curved surface similar to the shape of the human eye fundus, the OCT image of a flat surface will deform into a curved surface.In our experiment, when the center of the target was in focus, the scales were out of focus and the periphery of the scales fell out of the OCT imaging depth range.As a result, the angular FOV measurement with this test device failed.Figure 3 shows the lateral resolution measurement results with this test device.Figures 3(b) and (c) display the enlarged-view images of markers in group five and group six.With qualitative evaluation, the bars in element six group five are the minimal bars that can be resolved, and the bars in element one group six are the largest bars that cannot be differentiated.This result corresponds to a lateral resolution between 7.81 μm and 8.77 μm.Figures 3(d)-(f) present the vertically averaged cross-sectional intensity curves through group five element six to group six element two as marked by the yellow dash lines in figures 3(b) and (c).The dropped ratio between the intensity of the peaks and the dips was calculated with the averaged intensity curves.In the Rayleigh criterion, if the dropped ratio is smaller than 26.5%, the bars cannot be differentiated.Therefore, the lateral resolution of this tested OCT is between elements one and two in group six based on the Rayleigh criterion, which corresponds to 6.96 μm and 7.81 μm.Although this test device could provide a quick and qualitative lateral resolution measurement as a range by observing the smallest bars that can be resolved and the largest bars that cannot be resolved with the en-face OCT image, the measured result may differ from person to person.Different from measuring the resolution of a microscope with the resolution target, OCT images are heavily impacted by the speckle noise due to the essence of coherent imaging, which further reduces the measurement accuracy.On the other hand, the combination of this test device with the Rayleigh criterion provides a well-controlled and more accurate measurement.While the measured lateral resolution based on observing and Rayleigh criterion could be different.Consequently, a more detailed lateral resolution test method is needed.
The knife-edge method was also frequently applied in the microscopic imaging field to measure the imaging resolution [32][33][34][35].In OCT imaging, because of the speckle noise in the images, it is difficult to obtain a noiseless intensity curve across the knife edge.Therefore, the commonly used setup is measuring the sample arm imaging beam width by translating the knife edge across the beam at different depths and monitoring the power of the light behind it with a power meter [36,37].However, it is difficult to integrate this test method with a model eye as a test device.Whereas, the invention of the PSF phantom solved this problem.By doping the subresolution particles into a transparent polymer with a different refractive index, the phantom could provide an accurate measurement of the lateral and axial resolution through an OCT B-scan image [11][12][13][14][15][16].Furthermore, the PSF phantom could also be used together with a model eye to facilitate an easy test procedure [17,18].
Moreover, the OCT image of the flat resolution target deforms into a curved surface ranging over a large depth, which increases the difficulty in measuring the lateral resolution and the angular FOV.Most of the clinical ophthalmic OCT instruments utilize a Gaussian-like imaging beam to probe the biological tissue.The width of a Gaussian beam diverges quickly outside of the beam waist region, and so does the lateral resolution.Therefore, the test device may need to be placed at different focal depths to obtain the in-focus images of all the elements on the resolution target for an accurate lateral resolution and angular FOV measurement.A potential method to address this issue is using a curved resolution target in the test device.However, considering the projection ratio between the photomask and the curved surface, it is difficult to fabricate the curved resolution target precisely and traceably measure its dimensions.To solve this problem, an improvement to the test device has been introduced by changing the single-focusing lens to a doublet [18].The doublet comprises two convex lenses, and the focal plane of the first lens is the back surface of the second lens.The scale used to measure the angular FOV of an OCT can be laser inscribed or printed on the back surface of the second lens.In this way, the scale markers will appear at the same depth in an OCT image volume and the angular FOV can be measured with one volume scan.The other parameters can be measured through a small flat window designed in the center of the second lens back surface.
3.2.Axial resolution, depth scale, and sensitivity FWHM of the reflection peak from the front surface of the glass shows that the axial resolution of the OCT device is 3.7 μm in tissue.The distance between the reflection peaks of the front and back glass surfaces measured with the OCT is L m = 1770 μm (in tissue).Given that the group refractive index n g of K9 at 1050 nm is 1.5207 and the designated tissue refractive index of the OCT device n 0 is 1.35.Therefore, L m is converted to the physical thickness of the glass L g = 1571 μm.Compared with the glass plate's traceable physical thickness of 1577 μm, the measurement error is -6 um or −0.38%, which meets the required accuracy within ±3%.In this test device, T f is 0.1%, T l is 92.16%, and R g is 4%.With the measured reflection peak intensity from the front surface of the glass and the averaged intensity of the noise floor, the calculated sensitivity of this tested OCT device is approximately 105.4 dB.
During the measurement, adjustment was made so that the front surface of the glass was in focus.Despite the presence of the ND filter, saturation was still observed.It is hypothesized that the reflections from the flat surface of the plano-convex lens and the front and back surfaces of the ND filter contributed to the detected light and caused saturation.To circumvent this saturation, it is recommended that the ND filter be mounted at a slight tilt, thereby minimizing back reflections.
For assessing OCT sensitivity, this test device used an attenuated specular reflector to measure the minimum reflectivity that the system can detect, which is commonly used by many research groups.However, there exist varying conventions (average or variance) for evaluating the background noise, and the resultant sensitivity value can differ upon the chosen calculation method [27,[38][39][40][41]. Furthermore, there are attempts using a fused silica plate with femtosecond-laser inscribed lines of varying widths (ALS-OP01, Arden Photonics, West Midlands, UK) for its convenience during measurement or phantoms with different microsphere suspension concentrations due to the tissue-similar scattering properties [19,42].Whereas, both these two methods are complicated in their principles.Compared to these two phantoms, measuring sensitivity with an attenuated specular reflector is straightforward, repeatable, and traceably characterizable, which is critical for a widely applicable testing method.It should be admitted that, with technological progress and fabrication improvement, the other two methods might receive more recognition by OCT stakeholders and be included in international standards in the future.displacement, either two truncated lines or no line at all will appear in the OCT B-scan above the glass image, as illustrated in figures 4(c) and (d) respectively.This test device effectively facilitates the assessment of the coalignment between the fundus image and the OCT scan.

Conclusion
In this work, we constructed an ophthalmic OCT test device according to ISO 16971:2015 and tested it on a commercial ophthalmic OCT instrument.As a published consensus of all the stakeholders, ISO 16971:2015 guides the registration and evaluation of all OCT for the posterior segment of the human eye in the nations that are attending ISO.Consequently, this suggested test method has a substantial impact.However, the suggested test method is not supported with solid experimental results.We hope this work could serve as a piece of objective evidence for the discussion of the revision of this standard in the ISO committee and for the relative parties who need an effective test method for OCT for the posterior segment of the human eye.
The test device enables the evaluation of the performance of an OCT instrument.Nonetheless, it is noteworthy that the ISO standard lacks comprehensive descriptions for lateral resolution and sensitivity test methods, which may pose challenges in achieving consistent and reliable measurements.Moreover, the modified resolution target's distorted images within an OCT volume scan compromise the accuracy of angular FOV measurements.ND filter should be tilted to avoid signal saturation.
In summary, although the current ISO standard provides the test device and test methods for a rudimentary measurement of the critical parameters of an ophthalmic OCT instrument, there remains a pronounced demand for a more practical and reliable test device with higher integration, a wider application scope, and a complete traceability chain.
The intensity signal in power values, I[x n ] (i.e.squaring of the amplitude of the depth-reflectivity profiles, i x n

Figure 1 .
Figure 1.Schematic drawings (a), (b) and the corresponding pictures (c), (d) of the test device and modified resolution target, respectively.

Figure 2 .
Figure 2. The 3D volume scan of the glass with the modified resolution target and FOV scales (a), and its en-face view image (b).

Table 2 .
Summary of the test methods of six critical ophthalmic OCT parameters.minimum bar width that can be resolved Axial resolution Glass front surface reflection FWHM of the reflection peak intensity curve Depth scale Glass front and back surface reflections = Bare single-mode fiber The appearance of fiber in the B-scan image

Figure 3 .
Figure 3. OCT images of the modified resolution target on the front surface of the glass (a), and the enlarged images of the markers in group 5 (b) and group 6 (c).(d)-(f) show the averaged cross-sectional intensity of the bars in group 5 element 6, group 6 elements 1-2 indicated by the yellow dash lines in (b) and (c).

Figure 4
presents the results of the test assessing the co-alignment between the confocal guiding image and the OCT scan.With a piece of bare single-mode fiber as the thin filament, two parallel lines appear in the B-scan when the fundus image and OCT scan are aligned, as shown in figure 4(b).In contrast, when there is a rotation or

Figure 4 .
Figure 4. Results of the co-alignment of fundus image with OCT scan test, (a) confocal guiding image with a green arrow indicating the intended OCT scan location, (b) aligned B-scan image, (c) and (d) misaligned B-scan images.

Table 1 .
Width of the resolution target bars (unit: μm).