Label-free identification of cell death mechanism using scattering-based microscopy and deep learning

The detection of cell death and identification of its mechanism underpins many of the biological and medical sciences. A scattering microscopy based method is presented here for quantifying cell motility and identifying cell death in breast cancer cells using a label-free approach. We identify apoptotic and necrotic pathways by analyzing the temporal changes in morphological features of the cells. Moreover, a neural network was trained to identify the cellular morphological changes and classify cell death mechanisms automatically, with an accuracy of over 95%. A pre-trained network was tested on images of cancer cells treated with a different chemotherapeutic drug, which was not used for training, and it correctly identified cell death mechanism with ∼100% accuracy. This automated method will allow for quantification during the incubation steps without the need for additional steps, typically associated with conventional technique like fluorescence microscopy, western blot and ELISA. As a result, this technique will be faster and cost effective.


Introduction
Cell death plays a key role in many physiological processes including maintaining homeostasis and disease progression. Dysregulation of the cell death process can lead to many life-threatening diseases, such as neurodegenerative diseases, autoimmune disorders, and cancers, amongst others. One such example is the development of drug resistance in cancer cells, which results in the failure of conventional chemotherapeutic treatments [1][2][3][4]. Therefore, anti-cancer drug discovery process for overcoming such multi-drug resistance relies heavily on identification of cell death mechanism induced by new drugs [5][6][7][8]. There are mainly two mechanisms of cell death: apoptotic and necrotic pathways. Apoptotic pathway is a programmed cell death pathway which occurs in a regulated fashion in the human body to maintain homeostasis [9][10][11][12]. Some of the hallmarks of apoptosis include cell shrinkage, membrane blebbing, DNA degradation, and the formation of apoptotic bodies [13][14][15][16][17][18][19]. The necrotic pathway, which is unregulated cell death, results from internal or external stresses such as injuries, toxins, and radiation exposure [20,21]. Necrosis results in the rupture of the plasma membrane and lysis of the cell [11,[22][23][24].
The current gold standard techniques for studying cell death mechanisms are mainly based on Biochemical assay [25], such as standard ELISA [26], Western blot [27,28], and TUNNEL [29,30], amongst others. Although these techniques have high sensitivity and accuracy, they are cumbersome, time consuming, and costly. For example, a typical western blot assay requires multiple steps including protein extraction, gel electrophoresis to separate proteins, protein transfer, blocking, primary and secondary antibody incubation, protein detection and visualization. All these complex steps can take up to a few days to get results. These assays typically require special expertise as well. The aforementioned drawbacks also hold true for other commonly used techniques such as electron microscopy [31,32], fluorescence microscopy, and flow cytometry [33][34][35][36][37][38][39]. While electron microscopy enables very high-resolution imaging, it cannot be used on live cells and requires extensive sample preparation [40]. The fluorescence-based microscopy techniques require labeling the cells with different fluorophores such as Calcein, Propidium Iodide, fluorescently labeled Annexin V [41][42][43] and others. Label-free techniques such as quantitative phase imaging (QPI), phase contrast microscopy [44][45][46][47], Raman microspectroscopy [48][49][50][51][52], and digital holography [53,54] have also been explored as alternative modalities to visualize (and quantify) some of the morphological hallmarks of cell death, such as cell shape factor, volume, blebbing, shrinkage, and membrane rupture, amongst others. In Phase contrast microscopy [55], the change in phase of light passing through and interacting with sample is visualized as a change in brightness in the image and is not quantitative. However, QPI [47] provides quantitative spatial mapping of the phase change due to the sample. The main advantage of the label-free imaging techniques is the ability to distinguish the different cell death types in real time without the need for external labeling.
Here, we explore the feasibility of using scattering-based microscopy (dark field), which is another label-free imaging modality, for identifying the cell death mechanism. Dark field microscopy is a nondestructive imaging method based on light scattering from the sample of interest. Previously, dark field microscopy had been used for different intracellular imaging applications, e.g. imaging nanoparticle uptake [56,57]. Compared to brightfield, this modality has a low background, which results in high signal to noise ratio (SNR) and enables sensitive detection at the level of individual macromolecules and nanoparticles [58][59][60][61]. Furthermore, dark field microscopy can offer detailed information regarding the sample's structural and morphological characteristics, such as surface texture and contour, which is useful for studying cells and tissues. Another advantage of utilizing the scattering information, compared to the phase, is its ability to translate this technology for in vivo and ex vivo applications. Scattering signatures from tissues have previously been used for studying different properties, such as identifying cancerous regions [62][63][64], including its microscopic morphology [65]. Some of the commonly used scattering based imaging techniques include dark field microscopy, diffuse reflectance and sub-diffuse spatial frequency domain imaging amongst others [66][67][68][69][70][71][72][73]. While dark field microscopy enables detection of nanostructures down to 10 nm, other techniques that can offer similar sensitivity include interferometric scattering (iSCAT) microscopy, coherent brightfield (COBRI) microscopy, spatial light interference microscopy (SLIM), and interferometric reflectance imaging sensor (IRIS), amongst others [74]. For example COBRI microscopy has been used to track single nanoparticles with size down to 10 nm as well as image chromatin organization of unlabeled cells, iSCAT was used for label-free imaging of biomolecules such as virus (20-200 nm) [75][76][77] and IRIS can detect particles with diameter down to 50 nm such as exosomes [78]. However, these techniques require complicated and expensive optical setup, and sensitivity can be limited by speckle noise. Comparatively, dark field microscopy requires simple low-cost optical components and does not require intensive computation.
Our proof-of-concept experiments using dark field microscopy were performed using breast cancer cells (BT-20). Cell death was induced by known antagonists such as Raptinal which induces apoptosis, and highly concentrated hydrogen peroxide which induces necrosis [79,80]. The changes in morphological features of the cells as they interact with the drugs, such as swelling, blebbing and membrane rupture were recorded in real time using a dark field microscope based on forward scattering of light. Phase contrast images were also recorded simultaneously to validate the results. A neural network was trained to quantify cell mortality and identify the cell death mechanism (apoptosis vs. necrosis) automatically with an accuracy of more than ∼95%. The pre-trained neural network was then tested on the same cell lines, but with a chemotherapeutic drug Doxorubicin that induces apoptosis [81], which was not used for training. The network was correctly able to classify the cell death pathway with ∼100% accuracy.

Imaging BT-20 cells
The experiments were performed using BT-20 cancer cells cultured in 10% serum-supplemented DMEM cell medium on a cover glass bottom Petri dish. Cells were incubated with different concentrations of drugs such as Doxorubicin (DOX) (10 µM, 20 µM, 40 µM, 50 µM, 100 µM), Raptinal (RAP) (10 µM, 12 µM,15 µM, 30 µM, 50 µM, 100 µM) and hydrogen peroxide (H 2 O 2 ) (5 mM, 10 mM, 25 mM) to induce cell death. Different concentrations of the drug were used to determine the dose-response relationship and evaluate the drug's effects at varying levels of exposure. A multi-modal microscope, with dark field, phase contrast, and fluorescence modalities (LEICA DMi8) and Leica DFC3000 G CCD camera was used to image the cells. The drug was added, and the imaging of the cells commenced from the moment of drug addiction up to three hours at regular intervals of 3 min with no extra incubation time. The imaging was conducted in a normal laboratory environment that was identical for all cells to minimize the influence of external factors. Control experiments were performed by imaging the cells following incubation with just DMEM (without adding any drug). All the experiments were performed in triplicates. This approach helps to ensure that the results are reliable and reproducible. A white light illumination and a 10x objective with numerical aperture (N.A.) of 0.25 was used for imaging purpose, as it would enable imaging of >100 cells at a time. The schematic view of the optical setup is shown in figure 1(a). Phase contrast microscopy was used to validate the result of dark field microscopy. Validation of necrotic cell death was carried out by staining the cells with 10 µM propidium iodide (PI) and imaging them with fluorescence microscopy. PI is excited at 589 nm and the emission is collected at wavelength >600 nm. Furthermore, to verify the effects of each drug on the cells, we performed western blot analysis.

Cell lysis and western blot analysis
Raptinal (3 µM), hydrogen peroxide (300 µM), and Doxorubicin (1 µM) were added to the BT-20 cells and grown for 24 h. To obtain subcellular fraction, cells were washed with ice-cold PBS and subjected to ice-cold cytosolic lysis solution (10 mM Hepes (pH 7.9), 1.5 mM MgCl 2 , 10 mM KCl, 20 mM HEPES, 1 mM dithiothreitol (DTT), 1 mM phenylmethylsulfonyl fluoride (PMSF), 50 mM NaF, 1 mM Na orthovanadate, 10 mM Na pyrophosphate, protease inhibitor cocktail). The cells were then scraped and incubated for 15 min on a shaker at 4 • C. The mixture was then incubated for 3 min with 1% NP-40 before being centrifuged at 14 000 rpm at 4 • C. Without disturbing the sedimented cellular pellet, the cytosolic lysate was collected. Protein concentration was determined using the bicinchoninic acid (BCA) quantification method. After that, cytosolic proteins were separated using 10% acrylamide SDS-PAGE. These proteins were transferred to a PVDF membrane and incubated overnight at 4 • C with primary antibodies against caspases 3, 7, Bak, Bcl-2, and B-actin. The next day, HRP-anti-rabbit or antimouse secondary antibodies (1:4000) were added for 1.5 h, and protein bands were identified using Syngene's G:Box (Frederick, Maryland) and quantified using Image J software (NIH, Bethesda, MD, USA). The data is shown as a cytosolic protein/-actin ratio.

Training a neural network for cell death detection and classification
An algorithm was developed to classify the cell death mechanism using a neural network which identifies the morphological changes of cells observed with scattering-based microscopy due to the addition of different drugs such as DOX, H 2 O 2 and RAP, and buffer (control). The first step of this process involved drift correction of the time series images of cells, which was performed using a macro created by Nicholas Schneider for ImageJ software. Histogram equalization with respect to balanced image intensity was then performed on all the cells, in order to adjust the contrast of the images. Hough transformation was implemented to identify the center of each cell and other feature parameters. The accuracy of Hough transformation was improved by sharpening the images using unsharp masking, with a radius of 10 and an amount of 20, in MATLAB. Their parameters were manually determined by comparing the different images with a variety of 'radius' and 'amount'. The polarity for image binarization was set to the bright foreground and the threshold was set to adaptive. Any unconnected holes were closed by using the bwareaopen function. To extract regions of cells from the image, the sensitivity was set to 1 and the radius of circles to be detected was set between 10 and 15. These values correspond to the range of radius of cells and were obtained manually. A single field of view included >100 individual cells per frame. Therefore, the images were segmented so that each sub-image contained individual cells or small cluster (<5) of cells. Different augmentation techniques such as histogram equalization, normalization, random brightness change (100-200 pixel value), scaling (0.5-2.0 times), reflection and rotation (0 • -270 • , at 90 • interval) were performed to prevent overfitting and increase the accuracy. After image processing, features were extracted for each frame by using a 101-layer deep ResNet-101 neural network which has 100 convolution layers and one fully connected layer [82]. The feature vector for each frame of video after passing through the RESNET-101 network was a 2048dimension vector. The number of frames for each video ranged between 32 to 120 and every frame in the video was analyzed. The database of ImageNet was used to pretrain this network. A long short-term memory (LSTM) network was run on the extracted feature vectors to learn the temporal behavior of these extracted features.
We used SoftMax and a fully connected layer to classify the output. For training our network, we set the batch size to 32 and initial learn rate to 10 −4 . Adam optimization and mean squared error loss function were used to fit the model. The training was carried out on an i7-9700K computer with RTX 2080 Super graphics. The time for feature extraction was around 40 min and the time for training was 20 min, totaling a total time of 60 min for training 3791 videos. Machine learning algorithms can be done to extract specific cellular features such as cell volume, area, brightness, circularity, etc. This kind of specific feature extraction and training of LSTM network must be done for every specific cell line.

Detection of cell death process using microscopy
The cells (BT-20) were illuminated using an oblique beam of light and the scattered light from the different organelles were collected using the objective and imaged using a camera (Leica DFC3000 G, Sony(R) ICX455 interline CCD sensor), as shown in figure 1(a). The illumination angle cone is adjusted to exclude photons that are directly transmitted through the cytosol (and substrate) without interacting with the cells. The resulting images have a low background and therefore high SNR that enables clear visualization of the cell membrane as well as some of the larger organelles. This enables efficient tracking of the morphological changes of the cells shown while undergoing cell death. Figure 1(b) shows morphological changes of cells during apoptotic and necrotic cell death pathways. The morphological changes of BT-20 cells caused by the addition of different antagonists (Raptinal, Hydrogen peroxide, and Doxorubicin), as observed using the scattering signature, are shown in figure 2. In this study, we used commercially available drugs that are well characterized and have been shown to induce cell death through a specific mechanism. Doxorubicin [81,[83][84][85] and Raptinal [86] induce apoptosis [87] which results in the reduction of cell volume and membrane blebbing, among others, which can be clearly observed in figures 2(d)-(f). The concentrations of these drugs for BT-20 cells were previously optimized and their efficacy were monitored using holographic microscopy and western blot [53]. Whereas, hydrogen peroxide at high concentration induces necrosis [88,89] which is characterized by cell swelling and loss of membrane integrity which results in the leakage of cytosolic content. Figures 2(j)-(l) depict the changes in the intracellular scattering of cells undergoing necrosis. In this case, we observe an increase in cell volume as well as slight blurring of the images resulting from defocusing due to spillage of cytosolic content. Control experiments to check the effect of light toxicity were performed by imaging cells treated with buffer. We do not observe any changes in cell morphology during the total imaging period as shown in figures 2(a)-(c). Videos corresponding to each cell death are presented in supplementary section (dacf324supp2-9). In this study we used a low N.A. objective (10× magnification with a N.A. of 0.2), as it enables simultaneous imaging of a large number of cells (∼a few hundred). Based on the Rayleigh's criterion, (Resolution = 0.61 × wavelength/N.A.), the limit of resolution for this system will be ∼1 µm. At this resolution, we were able to observe most of the morphological hallmarks of apoptosis and necrosis (∼large blebs, changes in cell volume etc). Additionally, this resolution can further be enhanced using computational techniques, such as deconvolution and unsharp masking, amongst others, as shown in (supplementary figure S1).
The observed cellular changes based on its scattering signatures were validated using phase contrast microscopy as shown in figures 2(m)-(x). Phase images were sequentially acquired along with the dark field images at the same time intervals. Phase contrast microscopy produces high contrast images due to minute optical phase shifts, resulting from the waves passing through different intracellular organelles. The blebs and the cell shrinkage are very prominent in the phase contrast images of cells undergoing apoptosis. Similarly, for necrotic cells, the contrast between the cells and background has significantly reduced due to the leaking of cytosolic content after membrane rupture, which reduces the refractive index mismatch between the cells and the background. The control cells showed no changes even in phase contrast microscopy. The scattering-based images showed excellent agreement with the phase contrast. Although both phase contrast and dark field microscopy can be used for intracellular applications, we focus on the scattering-based imaging due to its ability to translate this technology for in vivo and ex vivo applications. The induction of necrosis using hydrogen peroxide was also validated using fluorescence microscopy by staining the cells with PI. Furthermore, to validate the cell death pathway induced by each of the drugs used here (antagonists), we performed western blot analysis.

Validation of cell death mechanism using western blot
Western Blot is a widely utilized immunoassay, that is used to detect and analyze specific proteins within a biological sample. It involves multiple steps, such as protein separation in the sample through gel electrophoresis, transfer of the separated proteins onto a membrane, blocking of non-specific binding sites, and incubation with antibodies that bind to the target protein. The bound antibodies are then visualized using various detection methods, providing valuable information about protein expression and interactions. We used this assay to observe the expression and activation of various proteins within the cells that are involved in cell death pathways, such as caspase 3 and 7, cleaved caspases, Bak and Bcl-2.
Caspases are crucial for intrinsic apoptosis. Intrinsic apoptosis is characterized by irreversible mitochondrial outer membrane permeabilization, which is promoted by pro-apoptotic proteins such as Bcl-2 associated X, apoptosis regulator (Bax), and/or Bcl-2 antagonist/killer 1 (Bak) and inhibited by anti-apoptotic proteins such as Bcl-2 [90,91]. Mitochondrial permeabilization results in the release of cytochrome c from mitochondria into the cytosol, which subsequently begins apoptosis by formation of an apoptosome via binding to apoptotic peptidase activating factor 1 (APAF1) and procaspase 9 [92]. This results in the proteolytic activation of executioner caspases, caspases 3 and 7 followed by induction of apoptosis [93]. Caspase 3 is also recognized as the most essential and dominant protease necessary for the proper execution of apoptosis. Similarly, caspase 7 plays an important function in the separation of apoptotic bodies. Several essential proteins involved in cellular functioning and survival are then cleaved by these caspases, resulting in apoptotic cell death [94]. In this investigation, we employed H 2 O 2 as a positive control for necrosis and Raptinal and Doxorubicin as a positive control for apoptosis. We performed western blotting to demonstrate the type of cell death that the compounds induce. Our findings ( figure 3) show that, in comparison to untreated samples, BT-20 cells treated with Raptinal (as low as 3 µM) and Doxorubicin (as low as 1 µM) promote cleavage of caspase-3 and caspase-7, with Doxorubicin producing the greatest level of caspase cleavage. Interestingly, 300 µM of hydrogen peroxide (H 2 O 2 ) also produced cleavage of caspase 3 and caspase 7. But necrotic cell death induced by H 2 O 2 is supported by the overexpression of Bcl-2, an antiapoptotic protein, indicating that H 2 O 2 produces necrosis. Raptinal and Doxorubicin, on the other hand, downregulate the Bcl-2 protein, indicating that they cause apoptotic cell death. Likewise, Doxorubicin and Raptinal upregulated the expression of Bak, a pro-apoptotic protein compared to control and H 2 O 2 . Overall, this suggests that Doxorubicin and Raptinal induce apoptotic cell death whereas H 2 O 2 induces necrotic cell death.

Deep learning for automated classification of cell death mechanism using dark field images
A neural network was trained to identify cell death pathway automatically based on the morphological changes (e.g. shrinkage, blebbing, membrane rupture, etc) observed in the dark field microscopy images. The algorithm including all the image processing steps and neural network architecture is shown in figure 4. Since each individual image consisted of a large number of cells, the first step involved automated segmentation to extract images of cells. Most of the extracted cells were either individual cells or small clusters of cells (<5). The segmentation algorithm enabled us to detect ∼80% of the cells, which was enough to train the network. The algorithm omitted large clumps of cells (>5) as well as cells that were already fragmented. The output of this step was the time lapse image sequence (video) of individual (or small cluster of) cells. The segmentation step using Hough transform yielded 889 control videos, 1142 H 2 O 2 videos (necrosis), 1760 RAP videos (apoptosis), and 366 DOX videos (used to test neural network). 70% of these videos in each category (control, apoptosis, and necrosis) were randomly selected to train the network. The rest 30% of the videos (1138 videos) were used for testing the accuracy of this neural network. It is challenging to identify the exact features used by the deep neural network to classify the images, because the features become very abstract. However, based on figures S2-S4, we conclude that the network is learning mostly the changes in morphological features and some of the intensity features as well.
As shown in figure 5 (Confusion Matrix and AUC curve), the overall accuracy of the neural network for classifying cell death is >95%. The accuracy for classifying the apoptotic (induced by RAP) and necrotic pathways (H 2 O 2 ) were 98.5% and 90.7% respectively. This can be attributed to the fact that the morphological changes in cells undergoing necrosis, observed using the scattering signature, are not very prominent, and therefore the network at times confused them with control cells. Out of 353 videos of necrosis (H 2 O 2 ), 320 of them were correctly predicted as necrotic and 31 of them were predicted as control (although all cells were necrotic as indicated by PI labeling) and only 2 of them were predicted as apoptotic. All the cells exposed to the high concentration of hydrogen peroxide showed staining with PI, indicating necrosis. In comparison, out of the 544 videos of apoptosis (RAP), 536 of them were correctly predicted as apoptotic and only 8 of them were predicted as live cells. For the control cells (viable cells), 227 out of 241 were correctly predicted as live, 3 of them were predicted as necrotic and 11 of them were predicted as apoptotic. The output results of the individual apoptotic and control cells were not validated using conventional methods, but none of them showed any PI staining, after adding the dye (thereby indicating no necrosis). The western blot assays, which were carried out by immunoblotting of caspase-3, cleaved caspase-3, caspase-7, cleaved caspase-7, Bak and Bax, with β-Actin as the loading control, also supports the overall observations. However, western blot gives an average result from all the cells analyzed and no information about individual cells. Thus, it is possible that some of the cells (8 out of 544) treated with RAP were still viable and did not undergo apoptosis. Similarly, it is possible that a small fraction of control cells (14 out of 241) were undergoing cell death (apoptosis) due to external factors.
This pre-trained network was also evaluated using cells undergoing apoptosis by treatment of Doxorubicin, a drug not used for training. The network was able to identify 100% of the cells (366 cells out of 366 cells) treated with Doxorubicin as apoptotic, as verified by the western blotting assays, which showed that Doxorubicin treatment leads to caspase cleavage, suggesting apoptotic cell death. Our results show that this pre-trained network can be used to classify the effects of compounds (drugs) that were previously not used for training. Therefore this method will be immensely helpful in testing new anti-cancer compounds [95][96][97] or treatment strategies such as nanoparticles [98], that are developed to effectively treat cancers, particularly the drug resistant cancers, during the initial drug discovery stage. Our choice of cell line was the TNBC cells because these types of cancers have a poor prognosis in comparison to other kinds of breast cancers [99,100]. It is challenging to treat this type of cancer due to the lack of expression of estrogen and progesterone receptors, as well as the human epidermal growth factor (HER2) [100,101], which are usually targeted in many conventional therapies. Therefore, there is a great thrust in the development of new treatment strategies for these types of cancers. This automated method presented here will be able to identify the viable from dead cells and also distinguish between apoptotic and necrotic cell death mechanisms, thereby providing real time information about the effectiveness of the treatment during the incubation step itself without the need for any additional steps during the drug discovery process.

Conclusion
Dark field microscopy was used to image the interaction of drug molecules and breast cancer cells and identify cell viability as well as cell death mechanism. Triple negative cancer cells were treated with different drugs to induce two different cell death pathways: apoptosis (using Raptinal) and necrosis (H 2 O 2 ). The hallmarks of each type of cell death, such as cell shrinkage (apoptosis), membrane blebbing (apoptosis), cell swelling, and membrane rupture (necrosis) were clearly observable in the time-lapse images. These temporal changes in cell morphology were then used to train a neural network, which was able to classify cell motility and death mechanisms with >95% accuracy. This pre-trained network could also identify the cell death mechanisms induced by a drug not used for training, in the same cell line. Our results show that dark field microscopy, coupled with deep learning, can be a useful tool for understanding cell death mechanism, which is an essential step for many different applications, especially drug screening.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).

Acknowledgments
A R is supported by University of Toledo Startup funds. A K T is supported by Susan G Komen Breast Cancer Foundation (CCR18548498) and Department of Defense (W81XWH210053). The views expressed in this article are those of authors and may not reflect the official policy or position of the Department of the Army, Department of Defense or the U S Government or Susan. G Komen Breast Cancer Foundation.