Objective comparison of lesion detectability in low and medium-energy collimator iodine-123 mIBG images using a channelized Hotelling observer

Each collimator's septal penetration fraction and system sensitivity were measured for ¹²³I imaging in accordance with the NEMA standards (NEMA 2007).

Images were created using an anthropomorphic phantom to simulate the clinical challenge of lesion detection for each collimator/camera combination.

The Liqui-Phil^™ anthropomorphic abdomen-shaped phantom was filled with 9585 ml inactive water. The dimensions of this phantom, indicated in figure 2, correspond to an older child or adult. The phantom contained an inactive 2.5 cm diameter polyoxymethylene cylinder to simulate the attenuation of the spine. Lesion-mimicking-inserts filled with ¹²³I were positioned in the phantom at the positions shown in figure 2. These were a 2 cm diameter sphere adjacent to the anterior surface of the spine, a 3 cm diameter sphere near the centre of the liver and (1 mm internal diameter) Polyethylene tubing wrapped twice around the spinal cylinder to represent a single mIBG avid vertebra. Three separate anterior and posterior pairs of planar scans were acquired with the abdomen-shaped phantom containing only one lesion at a time. In addition the Liqui-Phil^™ liver insert filled with ¹²³I was imaged separately within the abdomen-shaped phantom. Finally ¹²³I was added to the abdomen-shaped phantom itself and it was imaged independently to produce an abdominal background image. The activity concentrations used for all inserts and the background are shown in figure 2. The phantom was positioned on the standard imaging couch. Planar scans were acquired with each collimator on each gamma camera with a 20% wide energy window centred on 159 keV, a 256  ×  256 matrix and uniformity correction. The uniformity correction on the Skylight was based on intrinsic ¹²³I maps, to correct for photon detector tube edge enhancement artefacts. No such artefacts were observed in these phantom images. This was not necessary for the other 2 systems and an extrinsic Cobalt-57 uniformity map was used. The pixel dimensions were 4.6  ±  0.2 mm² as the detector field-of-view varies for each camera. The Siemens Intevo has the largest 614 mm FOV, the Philips Skylight has a 597 mm FOV and the GE Discovery has the smallest 565 mm FOV. The collimator faces were positioned as close as possible to the phantom posterior and anterior surfaces. The objects were scanned for as long as possible depending on each camera's availability. The frame duration used for each collimator image set was adjusted to account for the physical decay of the ¹²³I between each image set, so that the image count statistics remained comparable. Therefore these high-count images reflect the relative variation in sensitivity, spatial resolution and septal penetration between the collimators. A minimum of 3k counts was collected for the inserts and 347k counts for the abdomen. For each image set acquired the abdomen-shaped phantom position was accurately reproduced using markings on the camera couch.

Zoom In Zoom Out Reset image size

The acquired images were combined using an interactive data language (IDL) (release 8.2.3, ITT Visual Information Solutions) user script. This program first reduced the liver and abdomen image counts to values representing clinical image frame times between 400 and 600 s. The count in each pixel was then randomly selected from a Poisson distribution, with a mean value set to the rescaled count. A different random seed was used to create 200 different statistical realisations of the liver and abdomen. These images were then summed to produce 200 background images per collimator of the liver and physiological abdominal background images with liver-to-background ratios (LBR) ranging from 3.6 to 5.6.

For each collimator 200 images containing a low-contrast single-lesion were created by adding lesion-insert-images to the background images. The lesion-insert-image counts were rescaled prior to Poisson resampling to simulate varied tumour-to-background ratios (TBR). The tumour images were also shifted axially to simulate a variety of lesion locations. The resampled-shifted-lesion-insert images were then summed to the normal scans. In this way 17 images with the 3 cm sphere in the liver, 79 with the 2 cm sphere next to the spine and 104 images with a vertebra were created, all with different TBRs. The resulting liver-lesion TBRs ranged from 2.8 to 7.2, the spine-lesion TBRs were 29 to 255 and the vertebra TBRs were 179–1785. The vertebra required a high TBR to be visible. The range of simulated activity concentrations are shown in table 2. These lesion-to-background activity concentration ratios and positions were chosen to provide a broad range of challenging lesion detection tasks, from clearly visible to barely perceivable lesions.

A total of 3200 posterior images were used for CHO scoring. The resultant average signal-to-noise ratios (SNR) in 3 cm circular ROIs in the hottest part of the liver and coldest part of the background were compared to those measured in 6 recent patient scans. Three of these 10 min static patient scans were acquired with MEGP collimators and 3 were acquired with LEHR collimators. The SNR was calculated as the mean divided by the standard deviation in the mean ROI counts.

A mathematical model, the channelized Hotelling observer (CHO) was used to score the images for the possibility of an abnormality. The CHO model was written in IDL following the method published by Shidahara et al (2006). Test statistics were produced from the Fourier transformed images, F(ρ), filtered into different frequency channels. These channels have a psychophysiological basis in reflecting the frequency selective channels of the human visual system. Radially symmetric channels were applied in the frequency domain, described by;

$$\begin{eqnarray}&&{{u}_{\text{c}}}\left(\rho \right)=\left\{\begin{array}{c} 1\parallel \rho \parallel \in \left[{{\rho}_{0}}{{2}^{c-1}},{{\rho}_{0}}{{2}^{c}}\right] \\ 0~\,\text{Otherwise} \end{array},\right. \end{eqnarray} \tag{ 1 }$$

where $c\in \left\{1,2,3,4,5\right\}$ ${{\rho}_{0}}$ pixel⁻¹ is the low end cut-off frequency. The average value in each channel was calculated to create a matrix of values, r_i, for each of the 5 channels, i, r  =  [r₁, r₂, r₃, r₄, r₅] for each image f. The difference of the means of the channel matrices per camera-collimator combination, for the lesion present ${\boldsymbol{\overline{r}}_{a}}$ and lesion absent ${\boldsymbol{\overline{r}}_{n}}$ channel matrices were used to define a matching filter ${\boldsymbol{\overline{r}}_{a}}-{\boldsymbol{\overline{r}}_{n}}$.

Pre-whitening was used to de-correlate the noise in the channels by multiplication with ${{\mathbf{K}}^{-1}}$, which is the inverse of the average of the 2 noise covariance matrices for the entire abnormal and normal ensemble. Then a test statistic λ_f for each image was calculated as;

$$\begin{eqnarray}&&{{\lambda}_{f}}={{\left[{\boldsymbol{\overline{r}}_{a}}-{\boldsymbol{\overline{r}}_{n}}\right]}^{T}}{{\mathbf{K}}^{-1}}\mathbf{r}.\end{eqnarray} \tag{ 2 }$$

In effect CHO provides a score for the possibility of a lesion presence, as the weighted sum of the counts in the frequency channels of the tested image.

In order to validate the CHO model, 4 experienced human observers scored 240 of the posterior phantom images for the presence of a lesion, using the scoring system in table 3. The observers were 2 medical physicists and 2 radiologists all with over 10 years' experience at interpreting nuclear medicine scans. The images consisted of 60 of the background and 60 images containing lesion inserts generated for each of the Philips LEHR and MEGP collimators. These were viewed on the Hermes work station (Hermes Medical Solutions) that all the observers were familiar with. The images from the different collimators were randomly interspersed so the observers were not informed of the collimator used. The first 20 images from each collimator were used as training sets as the observer learnt the scoring system and adjusted their own criteria to apply it. The following 100 images were used to form ROC curves. For each score for each of the background and lesion present images, the fraction of the scores equal to and above a threshold score was calculated. The threshold that was used to assign an image as positive or negative for the presence of a lesion, was incremented from the lowest to the highest score (1–5). The true positive fraction (TPF) was calculated from the lesion present images and the false positive fraction (FPF) was calculated from the background images. The average TPF and FPF for each human score threshold was used to create an average human ROC curve against which the CHO curves were compared.

CHO was used to score the same 100 (non-training) images for each collimator. CHO scores were binned into 12 score thresholds equally covering the range of scores generated by CHO for each data set. This gave comparable discrete FPF sampling frequency for the CHO and human ROC curves for validation. The area under these ROC curves, Az, was then calculated using a trapezoidal method. Standard deviations on the data points were calculated according to Metz (Metz 1978). These standard deviations were propagated to estimate errors on the area under the ROC curves. The CHO cut-off frequency was adjusted to achieve the best agreement between the CHO and average human observer Az. The validated cut-off frequency was used for the further comparison of collimators.

The full set of 3200 posterior anthropomorphic phantom images was scored using the validated CHO model with the optimised cut-off frequency. The anterior images were discarded as neither observer was able to identify the lesions in the images leading to ineffectual ROC curves with an Az of 0.5.

The 400 CHO scores for each collimator were used to generate ROC curves for near continuous thresholds using ROCkit 0.9B (Metz 1998). This software also fitted a binormal distribution to the ROC curve, calculated Az, compared the correlated ROC curves and provided a two-tailed p-value to quantify the statistical significance of the difference between them. It had not been possible to apply this software to the human scores due to limited number of data points associated with the 5 human scores.

Table 1 shows the NEMA test results alongside the other collimator properties. No septal penetration was found for any of the medium-energy collimators. The long thick septa of these collimators appeared to stop the high energy photons. However the ME collimators were the least sensitive for each camera. The LEGP collimators were the most sensitive and gave double the sensitivity of the MEGP collimators but with over 5% septal penetration. The most septal penetration (16.3%) occurred for the Siemens LEHR collimator which has the shortest thin septa of all the collimators tested. Star artefacts were evident in the images from this collimator. The septal penetration of the Siemens' LEHR collimator was 15 times higher than that of the Philips, due to the shorter Siemens' collimator holes. The GE collimators provided the lowest septal penetration of the three manufacturers.

The sensitivity between camera collimators could not be compared because the thicker 5/8'' crystal of the Skylight (Philips) will bias these sensitivity results.

Images before and after Poisson resampling are shown in figure 3. These are inset on a plot of a single pixel wide profile across the vertebra insert.

Zoom In Zoom Out Reset image size

The resultant simulated total image counts ranged from 245k to 378k counts, which are typical of those acquired from 10 min abdominal mIBG planar scans. Examples of the generated images are shown in figure 4. The average pixel count in the background of the 6 patient scans was 20  ±  5 compared to 15  ±  5 in the background of the phantom scans. The average liver counts were 45  ±  7 and 30  ±  8 in the patient and phantom images respectively. The average liver-to-background ratio (LBR) counts were 2.26  ±  0.89 and 1.96  ±  1.59 in the patient and phantom images respectively. The SNR in the patient images background was 4.16  ±  0.59 and the liver was 6.49  ±  1.38. In the phantom images the SNRs were 3.47  ±  0.36 in the background and 4.73  ±  0.44 in the liver. All errors on these values are the standard deviations in the means. The count densities and SNRs were therefore lower in the phantom than the patient images, but a similar order of magnitude.

Zoom In Zoom Out Reset image size

The least difference between the human observer and CHO ROC Az scores was observed at a low end cut-off frequency of 0.008 pixel⁻¹. This was the lowest achievable frequency at 2 per 256 pixels. This value of ${{\rho}_{0}}$ was used for all subsequent CHO experiments. The validation ROC curves are shown in figure 5, the areas under these ROC curves are shown in table 4. The values are within the standard errors of one another.

Zoom In Zoom Out Reset image size

The ROC curves generated using CHO from 200 normal and 200 abnormal images for each collimator are shown in figure 6. The LEHR ROC curves consistently demonstrated lower TPF values than for the GP curves indicating poorer lesion detectability with LEHR collimators than for the other collimators.

Zoom In Zoom Out Reset image size

The areas under these curves are given in table 5. On average the LEHR collimators resulted in smaller Azs of 0.891  ±  0.004 than 0.933  ±  0.004 for the MEGP collimators. This difference was statistically significant (p  <  0.05) see table 6. There was no statistically significant difference between the Philips LEGP and LEHR collimators. However there was a statistical difference between the Az for the equivalent Siemens collimators. In contrast there was a statistically significant difference between the Philips MEGP and LEGP collimators but not for the corresponding Siemens collimators.