ICoRD: iterative correlation-based ROI detection method for the extraction of neural signals in calcium imaging

A novel ICoRD method that iteratively detects ROIs to obtain signals closer to the ground-truth calcium signal is proposed (figure 1(a)). For the description of ICoRD, we first define a few terminologies; ground-truth calcium signal, ground-truth correlation coefficient, reference signal, and correlation coefficient with final estimate. Ground-truth calcium signal defines the true activation calcium signal without noise. The correlation coefficient between the signal extracted by estimation method and the ground-truth calcium signal of the target neuron is defined as the ground-truth correlation coefficient. The reference signal indicates a seed signal that is the basis for calculating the correlation coefficient in the ICoRD algorithm. The correlation coefficient with final estimate is defined as the correlation coefficient between the reference signal and that extracted by the ICoRD algorithm when the ground-truth calcium signal is unknown.

Zoom In Zoom Out Reset image size

To accurately extract signals from neurons with low SNR, ICoRD requires a representative pixel and an observation range for the target neuron in the initial step (figure 1(a)). For this initial step of ICoRD, both automatic or manual initialization can be selected depending on the SNR. Manual initialization is recommended when the SNR of the target neuron is low or too hard to detect automatically. For automatic initialization, the cell boundary is detected based on the local maximum of the summary image (mean or correlation image) (Reynolds et al 2017). The mean image represents the average time-axis signal for each pixel. The correlation image was computed by averaging the correlation coefficient of each pixel with its eight immediately adjacent pixels. In automatic initialization, the central pixel of the ROI detected as a local peak is selected as the representative pixel, and the area of a predetermined size around the representative pixel is selected as the observation range. ICoRD also provides manual initialization to extract signals from neurons with low SNR that are difficult to detect with existing automatic detection techniques. Users can directly select the location and the observation range of neurons.

The first reference signal for the iterative update was defined as the average signal of nine pixels adjacent to a representative pixel in the neuron of interest. Within the observation range (red dotted box in figure 1(a)), ROI pixels were selected to maximize the correlation coefficient between the average signal of the candidate pixels and the reference signal. When certain pixels had to be excluded for various reasons, such as overlapping with neighboring neurons, the algorithm was executed without the rejection range selected by the user (figure S1).

Figures 1(b) and (c) show the correlation-based ROI detection method indicated in figure 1(a). This method effectively extracts the calcium signal of a target neuron, even in extremely noisy environments. The ROI detection technique proposed in this study aims to detect a neuron signal closer to the ground-truth calcium signal. ICoRD was assumed to acquire a reference signal close to the ground-truth calcium signal upon iteration, and an ROI detection method that maximized the correlation coefficient with the ground-truth calcium signal was first derived.

To detect the ROI that maximizes the correlation coefficient with a reference signal, we calculated the correlation coefficients twice in total. For the first calculation, ICoRD calculated the correlation coefficients between the reference signal and the signals of individual pixels to sort the pixels. All candidate pixels were sorted in descending order of the correlation coefficient calculated based on the reference signal (the number of pixels $i$ ranged from one to the maximum number of pixels (${\text{max}}$) within the observation range in figure 1(b)). The second correlation coefficient calculation was performed to calculate the correlation coefficient between the reference signal and the average signal of the different number of pixels ($i = 1\sim {\text{max }}){\text{ }}$ in figure 1(c). The temporal signal ($y$) of each pixel ($i$) is composed of a signal ($x$) and noise ($n$). Pixel order ($i$) is sorted in descending order based on the Pearson correlation coefficient of each pixel's signal (${y_i}$) and ground-truth signal (${x_1}$). The number of pixels that maximizes the correlation coefficient with the ideal reference signal (${x_1}$) was defined as the optimal number of pixels ($N$), and a pixel group with an optimal number of pixels was detected as an ROI. In the process of obtaining this second correlation coefficient, we observed a trade-off between signal power increase and noise power reduction. When the number of pixels was significantly large, pixels with weak signals were averaged, which reduced the correlation coefficient with the reference signal (figure 1(c)). However, when the number of pixels was significantly small, white noise was pronounced, resulting in a low correlation coefficient (figure 1(c)). A set of pixels with the optimal number of pixels was chosen as an ROI to achieve the highest correlation. The ground-truth calcium signal was then estimated by iteratively setting a new reference signal as the average signal of the ROI or the deconvolution signal of the average signal (figure 1(a)). This process was repeated until the reference signal was close to the ground-truth calcium signal. The detailed iteration algorithm is shown in figure S2.

Thus far, we have described how to determine the optimal number of ROI pixels that maximizes the correlation coefficient between the average signal of the ROI and the estimated ground-truth calcium signal. To estimate the ground-truth calcium signal of the target neuron in vivo, a CD method was applied to the average signal of the detected ROI (Pnevmatikakis et al 2016). Because this CD method has a denoising effect, its application helped estimate the ground-truth calcium signal. ICoRD iteratively detected the ROI and updated the average signal or CD signal to the reference signal, such that the reference properly approximated the ground-truth signal. The criterion for selecting the reference signal was the information difference between the average calcium signal of the previous and current iterations. The information difference was one minus the correlation coefficient between the average signal of the ROI detected in the current (${\text{Rawc}}$) and previous iterations (${\text{Rawp}}$) (equation (1)).

$$\begin{align} & {\text{Information }}{} {\text{difference }}\left( {{\text{Rawp}},{\text{ }}{} {\text{Rawc}}} \right) \nonumber \\ & = 1 - {\text{ }}\frac{{\sum\nolimits_i^n {\left( {{\text{Raw}}{{\text{p}}_i} - \overline {{\text{Rawp}}} } \right)} \left( {{\text{Raw}}{{\text{c}}_i} - \overline {{\text{Rawc}}} } \right)}}{\sqrt {\sum\nolimits_i^n {{{\left( {{\text{Raw}}{{\text{p}}_i} - \overline {{\text{Rawp}}} } \right)}^2}} } {\text{ }}\sqrt{\sum\nolimits_n^i {{{\left( {{\text{Raw}}{{\text{c}}_i} - \overline {{\text{Rawc}}} } \right)}^2}} }}.\end{align} \tag{ 1 }$$

The information difference in each iteration was used to select a reference signal to avoid significant bias in the initial CD result. A detailed algorithm flowchart of ICoRD is presented in figure S2. ICoRD eventually provided the CD result reconstructed from the average signal of the ROI with the maximum correlation coefficient using the estimated ground-truth calcium signal from the last iteration.

Having established a new signal-extraction algorithm, its performance at various noise levels was investigated. For a fair measurement of the signal extraction performance of ICoRD, a ground-truth calcium signal and various modeled noise levels were required. Therefore, a realistic model that simulates calcium imaging data with a designated noise level was developed. All simulations and analyses were performed using MATLAB R2020b (MathWorks, Natick, MA, USA). The simulated calcium imaging data were generated by modeling and multiplying the spatial and temporal components of calcium dynamics. The spatial component was modeled as a Gaussian shape of light propagation owing to the calcium indicator at the central location (75, 75) in 150 horizontal and 150 vertical spaces (figure 2(a)). The temporal component for single activation was modeled as a 10 Hz sampling rate based on the rising tau (${\tau _{\text{r}}} = {\text{ }}$ 550 ± 52 ms) and decaying tau (${\tau _{\text{d}}} = {\text{ }}$ 179 ± 23 ms) values of pGP-CMV-GCaMP6s (GCaMP6s) (figure 2(b), equation (2)) (Chen et al 2013):

$$\begin{equation}h\left( t \right) = \left\{ {\begin{array}{*{20}{c}} {{e^{ - t/{\tau _{\text{d}}}}} - {e^{ - t/{\tau _{\text{r}}}}},{ }t > 0} \\ {0,{ }t \leqslant 0} \end{array}.} \right.\end{equation} \tag{ 2 }$$

Zoom In Zoom Out Reset image size

The resulting signal and noise of the central pixel's simulated calcium trace from the neuron with an SNR of −30 dB are illustrated in figure 2(c). The ratio of the neuron signal power to the white Gaussian noise power was calculated as the SNR to generate a simulated calcium movie with various SNRs (equation (3)):

$$\begin{equation}{\text{SN}}{{\text{R}}_{{\text{dB}}}} = 10{\log _{10}}\left( {\frac{{{P_{\text{S}}}}}{{{P_{\text{N}}}}}} \right),\end{equation} \tag{ 3 }$$

where ${P_{\text{S}}}$ denotes the signal power, and ${P_{\text{N}}}$ is the noise power. The modeled temporal dynamics were generated as simulation data for 50 s based on a 10 Hz sampling rate by convoluting the calcium events at various intervals. If selective ROI detection using ICoRD increases the sensitivity of event detection, the events could be resolved in severely noisy images, and closely interleaved events could be resolved better than with the conventional CNMF method. To build a testing platform, two or more events occurring within 5 s with various intervals and noise levels between −30 and 0 dB SNRs were modeled. Figure 2(d) shows the mean and correlation images of the simulated calcium movie with an SNR of −30 dB. In general, the mean image provides an estimate of the location and shape of the neuron because the average pixel value of the neuron signal tends to be higher than the background signal. In the correlation image, more frequent activation of the neurons or the stronger signal intensity versus noise, contributed to enhancing the clarity of the appearance of the neuron.

All procedures involving mice were approved by the Korea Brain Research Institute, Institutional Animal Care and Use Committee (approval number: IACUC-19-00041). Adult (older than P60) transgenic mice expressing GCaMP6s (C57BL/6J-Tg [Thy1-GCaMP6s] GP4.12Dkim/J, The Jackson Laboratory) underwent surgery to implant a cranial window and headplate. A 3 mm craniotomy was performed over the posterior parietal cortex (PPC) area centered 2 mm posterior and 1.7 mm lateral to the bregma. The window covering the craniotomy area was constructed by bonding 3- and 5 mm diameter No. 0 coverslips (Warner Instruments). The titanium headplate was attached to the skull over the window using opaque dental cement (Super-Bond, Sun Medical) and covered all exposed tissue.

Images were acquired with a Nikon 16× (0.8 NA with 3.0 mm working distance) objective lens attached to a microscope (HyperScope, Scientifica) running ScanImage (2019a). A Ti-Sapphire LASER (Chameleon Vision II, Coherent) operated at 910 nm was used as the light source for the imaging microscope. The head-fixed mouse was awake during the imaging procedure of the PPC and kept warm with a custom-made body restrainer. The imaging area was approximately 200 μm deep and centered at the PPC; it was adjusted to avoid thick blood vessels appearing in the ROI. The resolution of the images was 512 × 512 pixels, and the FOV was a 700 μm square. The frame rate of the microscope was set to 30 Hz.

The performance of the proposed ICoRD algorithm was compared with CNMF using the simulated calcium imaging data. The initial parameters used in CNMF are described in the Supporting Information (table S1). A comparison was conducted between the extracted traces of the ROIs set by the two methods from the simulated calcium images with an SNR of −30 to 0 dB (figure 3). In the CNMF method, if the selected ROI candidate was determined as unsuitable, it was automatically removed. Using the calcium images with severe noise (−30 dB), CNMF automatically determined and rejected all the pixels in the image with an SNR of −30 dB as noise. Therefore, we restored the ROIs by recovering the candidate ROIs that were originally rejected by CNMF for comparison with ICoRD. Conversely, ICoRD detected most pixels inside a neuron, although a substantial number of background pixels were included within the ROIs (figure 3(a)). Furthermore, the extracted signal from the ICoRD-selected ROI correlated significantly better with the ground-truth than the detected ROI from CNMF (figure 3(b); the correlation coefficients with the ground-truth from CNMF and ICoRD were 0.8267 and 0.9834, respectively). Subsequently, the performance of ICoRD at various noise levels was tested. At all noise levels, signals extracted from ROIs detected by ICoRD were closer to the ground-truth calcium signal than those extracted by CNMF (figures 3(c) and (d)), demonstrating that ICoRD-based ROI detection is more robust and suitable for noisy signals, as in many in vivo calcium images. As the neuron's SNR decreased, ICoRD lowered the noise level by detecting not only the pixel corresponding to the neuron, but also the surrounding pixels to maximize the correlation coefficient with the reference signal. In the case of CNMF, all ROIs in the data with SNRs of −30 and −25 dB were automatically determined as noise and rejected; therefore, the rejected ROIs were restored. CNMF detected a smaller number of pixels as the SNR decreased. Figure 3(d) shows the correlation coefficient of the ground-truth signal and the signal extracted by the two methods from 20 datasets (140 total) with white Gaussian noise added to each SNR (−30 to 0 dB). At all noise levels, the ICoRD result had a higher correlation coefficient with the ground-truth signal than the CNMF result. As shown in figure 3(d), the difference in performance between the two methods was particularly large in the SNR range of −30 to −20 dB. Because the proposed method is a correlation-based ROI detection method that aims at true calcium signal extraction, it shows excellent performance at severe noise levels (SNRs of −30 to −20 dB).

Zoom In Zoom Out Reset image size

ICoRD exhibits significantly better performance in detecting some neurons with a low SNR that other techniques were not able to detect. However, to compare the universal practicality of the two techniques in more detail, the processing time of ICoRD and CNMF were measured on simulated calcium imaging data with different number of cells and frames (table S2). ICoRD applied parallel computing toolbox in MATLAB to decrease the processing time that scale with the number of neurons. ICoRD had shorter processing time than CNMF for longer frame data. However, since the processing time of ICoRD increases with the number of neurons, it can be inefficient when detecting thousands of neurons. CNMF had advantageous when detecting a larger number of neurons, except for neurons with low SNR, but it takes longer to analyze data with many frames.

Figure 4 shows the trend of the estimated and ground-truth correlation coefficients and the information difference during the iteration of the proposed method in simulated calcium imaging data with an SNR of −30 dB. The correlation coefficient with final estimate at each iteration represents the correlation coefficient between the CD result of the last iteration and the CD result of each iteration. The ground-truth correlation coefficient of each iteration is the correlation coefficient between the ground-truth signal and the CD result of each iteration. ICoRD uses the average signal (Avg) of the ROI or CD result as a reference signal for the next iteration. These two reference sources determine the calcium signal extraction performance of ICoRD. Figure 4 shows the estimation performance of the ground-truth calcium signal based on reference signal selection. The Avg-based update has a white background, and the CD-based update of the reference has a pink background. Figure 4(a) shows the trend of the estimated and ground-truth correlation coefficients when the reference signal was selected using ICoRD. The trends of the estimated and ground-truth correlation coefficients were similar, and the ground-truth correlation coefficient generally increased with iterations (figure 4(a)). This trend of the ground-truth correlation coefficient reveals that ICoRD has been effectively designed to estimate the ground-truth calcium signal of the target neuron during the iterations. The maximum ground-truth correlation coefficient was 0.9889 at the 17th iteration. The bottom of figure 4(a) shows the vibration trend of the information difference, which is the criterion for selecting the Avg or CD as the reference signal (equation (1), figure S2). The information difference is used for avoiding significant bias in the calcium model estimated by only CD during the initial iteration. If the information difference is less than a certain threshold (for example, 0.02), the algorithm determines that there is no more information to be obtained from the previous reference source and updates it with another data source (Raw → CD, CD → Raw). In the process of information difference oscillation, ICoRD estimates closer to the ground-truth calcium signal. An Avg-based update could estimate the ground-truth signal with a moderately high correlation coefficient (0.9597 at the 17th iteration); however, the estimation performance was relatively low compared to ICoRD owing to the lack of the noise removal effect in CD (figure 4(b)). In addition, in the case of Avg-based update, the improvement of the ground-truth correlation coefficient by iteration was hardly observed after 6th iteration. Although CD has a noise-removing effect, there is a risk of continuously estimating the calcium model at the initial iterations in CD-based updates (figure 4(c)). In the CD-based update, the speed of finding the ground-truth calcium signal as increases of the iteration was relatively slow, and the maximum ground-truth correlation coefficient was 0.8087 in the 14th iteration, which was not sufficient to analyze neural activity. The information difference of the CD-based updates was saturated to zero overall with iterations. This saturated information difference indicates that the calcium model incorrectly estimated in the initial iteration has been maintained continuously from the initial iteration. The results in figure 4 show that appropriate conversion of the reference source allows ICoRD to estimate the ground-truth signal with a high correlation coefficient.

Zoom In Zoom Out Reset image size

It was verified that the proposed method has a powerful advantage in terms of the SNR in the simulation data. In this section, the practicality of the proposed method is examined using two-photon calcium imaging data measured in a mouse brain.

The proposed ICoRD algorithm was applied to raw in vivo calcium imaging data using the same algorithmic process as the simulation-based validation. The spatial and temporal results of the selected neurons using ICoRD and CNMF in vivo calcium imaging data were compared (figure 5). To compare ICoRD with CNMF, a neuron with high and frequent calcium activity (neuron 1) and a neuron with relatively sparse activity (neuron 2) were selected among the neurons clearly detected by CNMF (figure 5(a)). A neuron with a low SNR and sparse activation (neuron 3) was detected by ICoRD but not CNMF. Figure 5(b) shows the zoomed images (mean, correlation) around the three target neurons in the in vivo calcium imaging movie and the ROIs detected by ICoRD and CNMF. The location of the target neuron is marked in the mean and correlation images by a red dotted line. The SNR of neuron 1 was sufficiently high to be identified in both the mean and correlation images. The ROI of neuron 1 detected by ICoRD and CNMF were similar. The results of the two methods for high-SNR neurons in vivo were similar to the detection trends observed in the simulation environment. Neuron 2 only exhibited the blurry shape of the target neuron in the mean image. ICoRD detected clustered pixels and several surrounding pixels outside the target neuron, whereas CNMF detected only the clustered pixels of neuron 2. Neuron 3 only showed a blurry neuron morphology in the mean image and its ROI was only detected by ICoRD. ICoRD extracted signals from neuron 3 despite this neuron being sparsely activated at high noise levels. ICoRD detected the ROI of neuron 3 with clusters and surrounding pixels, similar to the ROI of neuron 2. These ROI detection results with the pixels surrounding neurons 2 and 3 were similar to the ROI detection results in the simulation verification with low SNRs. Because ICoRD detects the ROI that maximizes the correlation coefficient with the estimated reference by detecting pixels that have a high correlation with the estimated reference signal, it detects surrounding pixels in the case of very low SNR neurons and takes advantage of the noise reduction effect.

Zoom In Zoom Out Reset image size

Figure 5(c) shows the raw signal of the selected pixel as a 'Raw' (grey line) of the three neurons in the ICoRD algorithm and each CD result of the ROI detected by ICoRD (orange line) and CNMF (blue line). We present the first reference source (Raw) and the result of the last iteration of ICoRD to explain how the estimated reference changes with iteration. Each signal was normalized by subtracting the mean and dividing it by the standard deviation. A CD-based SNR estimation method was used to evaluate the neurons in an in vivo environment (figure S3). The estimated SNR of individual neurons in the Raw signal was −1.05 dB in neuron 1, −17.86 dB in neuron 2, and −21.87 dB in neuron 3. The signals extracted by ICoRD and CNMF in neuron 1, with a high SNR and frequent calcium activity, exhibited similar trends (correlation coefficient: 0.9608). However, for neuron 2, with a relatively infrequent and low-amplitude calcium activation signal, the signal trends detected by the two methods were relatively different (correlation coefficient: 0.7241). Furthermore, only ICoRD, not CNMF, detected neuron 3 as an ROI. This result is similar to that of CNMF, indicating low signal extraction performance for an SNR below −20 dB in a simulation environment. Although the calcium signal of neuron 3 was highly sparse at high noise levels, ICoRD effectively extracted the calcium trace after iteration. Although ICoRD and CNMF apply the same CD method, the signal obtained from ICoRD appeared to have a higher SNR. This is because ICoRD provides a better input signal for the CD method by detecting the ROI that maximizes the correlation coefficient with the predicted ground-truth signal of the target neuron. ICoRD was confirmed to effectively denoise signals from neurons with a low SNR in in vivo calcium imaging data.

In figure 5, the ROI detection and signal extraction trend of ICoRD and CNMF according to the relative SNR difference for each neuron was confirmed. However, it was difficult to analyze the resulting trend of both methods according to the exact SNR change in the in-vivo signal. To further analyze the detection results according to the SNR in the in-vivo data, the detection results were additionally analyzed by adding different powers of white Gaussian noise to neuron 1 with high SNR in figure 6. Firstly, in the in-vivo calcium imaging data of 512 × 512 pixels, the area near neuron 1 (150 × 150 pixels) was cropped. Assuming that the cropped raw data power is the signal power, white Gaussian noise is added to have an SNR of 0 to −15 dB. As noise was added from 0 to −15 dB, the shape of neurons in the mean image and correlation image became blurred (figure 6(a)). Figure 6(b) indicates the detected ROI using ICoRD and CNMF in neuron 1 with different SNR. As shown in the simulation test results trend in figure 3, as the SNR decreased, ICoRD detected more pixels as ROIs and CNMF detected fewer pixels as ROIs. CNMF was not able to detect ROI or ROI candidates in neuron 1 with the SNR of −15 dB. Figure 6(c) shows the mean and standard deviation of the correlation coefficients between the individual signal extracted by the ICoRD or CNMF and the average of the two calcium signals extracted by ICoRD and CNMF from the raw data (n = 5 for each SNR). The average of the two calcium signals extracted by ICoRD and CNMF from raw data was defined as the ground-truth to evaluate the signal extraction performance with different SNR. When a total of five simulation tests were performed for each individual SNR, ICoRD always had a high correlation coefficient compared to CNMF. In the case of −10 dB SNR, ICoRD had a higher correlation coefficient than CNMF despite detecting extrasomatic pixels as ROI. In addition, CNMF did not detect neurons in five test data with an SNR of −15 dB. Figure 6(d) show the extracted calcium trace in neuron 1 with SNR of −10 and −15 dB. Both techniques succeeded in extracting calcium signals, but CNMF had a lower correlation value (0.9436) than ICoRD (0.9741), and a higher baseline noise level was observed in the CNMF result. In the case of −15 dB SNR, only ICoRD could detect neuron 1 and still exhibit a high correlation coefficient (0.9584) with the ground-truth signal.

Zoom In Zoom Out Reset image size