Neuromorphic cytometry: implementation on cell counting and size estimation

Imaging flow cytometry (FC) is a powerful analytic tool that combines the principles of conventional FC with rich spatial information, allowing more profound insight into single-cell analysis. However, offering such high-resolution, full-frame feedback can restrain processing speed and has become a significant trade-off during development. In addition, the dynamic range (DR) offered by conventional photosensors can only capture limited fluorescence signals, which compromises the detection of high-velocity fluorescent objects. Neuromorphic photo-sensing focuses on the events of interest via individual-firing pixels to reduce data redundancy and latency. With its inherent high DR, this architecture has the potential to drastically elevate the performance in throughput and sensitivity to fluorescent targets. Herein, we presented an early demonstration of neuromorphic cytometry, demonstrating the feasibility of adopting an event-based resolution in describing spatiotemporal feedback on microscale objects and for the first time, including cytometric-like functions in object counting and size estimation to measure 8 µm, 15 µm microparticles and human monocytic cell line (THP-1). Our work has achieved highly consistent outputs with a widely adopted flow cytometer (CytoFLEX) in detecting microparticles. Moreover, the capacity of an event-based photosensor in registering fluorescent signals was evaluated by recording 6 µm Fluorescein isothiocyanate-marked particles in different lighting conditions, revealing superior performance compared to a standard photosensor. Although the current platform cannot deliver multiparametric measurements on cells, future endeavours will include further functionalities and increase the measurement parameters (granularity, cell condition, fluorescence analysis) to enrich cell interpretation.


Introduction
Conventional flow cytometry (FC) is a vastly adopted high-throughput technology that is capable of measuring multiparametric features of cells in a population, including cell count, relative size, granularity, and can be combined with fluorescence detection for additional phenotypical characterisations [1,2].It registers the light scatter and fluorescence signals from cells that are excited by a laser beam into photodetectors.This paradigm entails a high-speed processing method, which can go over 10 000 events per second [3].However, such an expeditious approach is limited to lower-dimensional feedback and lacks subcellular resolution [4].Thus, numerous applications have been developed to optimise various emerging research needs.Imaging flow cytometry (IFC) is one of the remarkable strategies that combines features of high efficiency in FC with detailed spatial information and fluorescence intensity [3,5].It can comprehensively visualise cell area with more intricate metrics, including cell morphology, texture, correlation and marker localisation in high-resolution feedback [6].As with every invention, the frame-based photosensor adopted in IFC can suffer from inherent constraints that hinder its performance in delivering high throughput results while maintaining rich spatial information.
Considering the limited field of view provided by a microscope or interrogation point in IFC, cells in the flow can only remain within the bounds of the scene for an extremely brief period.The frame-based capturing techniques can be vulnerable against consecutive fast-moving targets, especially in equipment with high frame intervals.High-velocity objects between two frames can lead to motion blur, ghost detection and other motion-induced artifacts [7].Even though a professional-grade camera or sensor can be integrated to perform the image acquisition to reduce motion impact, the data redundancy caused by a high frame rate can obstruct the efficiency in data-intensive and delay-sensitive tasks [8].Also, such a device's expensive cost and maintenance can be overbearing for research and start-up projects.In terms of fluorescence analysis in IFC, which is an indispensable combination for revealing cell signalling, co-localisation, cell-to-cell interaction and DNA integrity in large-scale populations [6].The dynamic range (DR) of current photosensors can have difficulties perceiving limited fluorescence signals, and overexcitation from light sources can lead to phototoxicity, photobleaching and tissue heating on cells [9].These further exacerbate the trade-off relationship between speed, sensitivity and spatial resolution [10].Increasing the throughput of IFC without jeopardising spatial resolution and sorting purity remains one of the outstanding obstacles in the field of IFC [11].These inspired us to develop a data-and cost-efficient fluorescence-sensitive high-throughput neuromorphic imaging cytometry (NIC) to challenge these conundrums.
Neuromorphic vision can detect objects in motion by individually adapting brightness changes in each pixel, and when there is no motion included in the field, pixels will remain inactivated [12].This is compared to a frame-based pixel array synchronously timed to a global shutter.The unique mechanism in neuromorphic allows real-time processing with low latency, leading to great application in object tracking, recognition and motion analysis [13].As the maximum detection range of fluorescent signals is highly dependent on the DR of the applied sensor [14], the high dynamic range (HDR) (>120 dB) provided by an event-based vision sensor (EVS) can be an ideal candidate for visualising fluorescence-tagged objects, especially in low lighting or dark scenarios.To evaluate the feasibility of integrating EVS as a substitution for frame-based sensors in IFC, the experiments were carried out with different sizes of polystyrene-based microparticles and THP-1 cells flowed within a microfluidic channel to create the essential dynamic contrast between targeted objects and the background for neuromorphic imaging, counting and size estimation.Additionally, to appraise the performance of EVS in detecting fluorescent signals, Fluorescein isothiocyanate (FITC)-marked microparticles were assessed under the neuromorphic vision in comparison to a traditional Complementary Metal-Oxide-Semiconductor (CMOS) sensor and a Scientific Complementary Metal-Oxide-Semiconductor (sCMOS) sensor.Herein, we are delivering an early demonstration of NIC to perform object counting, size estimation and fluorescence analysis on microscale targets, presenting the first instance of neuromorphic cytometric-like measurements and building upon our previous endeavour on introducing neuromorphic architecture as an alternative to overcome the conventional frame-related challenges in IFC.

Relevant works
IFC can offer rich spatial information regarding cell interpretation; however, as with many imaging systems, IFC is also bound to the triangle of imaging constraints-speed, sensitivity and resolution.Increasing one of the parameters can lead to degradation in others [3].In contrast, EVS is more robust in handling low-lighting conditions and highly dynamic scenes owning to their asynchronous firing pixels, and the high-resolution event data can support over ⩾3 µs frame-rate [15].The study conducted by Howell et al [16] evaluated the performance of an EVS in detecting fluorescent objects flowing in a spiral microfluidic device and proved the visualisation.Adopting an event-focused vision and architecture can significantly reduce data redundancy and latency, possibly elevating the performance in throughput and sensitivity He et al [17] utilised the neuromorphic-enabled imaging classification, accomplished a mean average precision of 98.52% at a speed of over 1000 frames per second (fps) and performed 3D reconstruction of the measured subjects via intensity and contour extraction.Moreover, the recent work conducted by Abreu et al [18] utilising an EVS to conduct binary particle classification with a spiking neural network and achieved 98.45% testing accuracy, which further consolidates the feasibility of implementing neuromorphic architecture in the field of cytometry.To the best of our knowledge, our previous work, Zhang et al [19], is the first feasibility demonstrations of conceptualising and building a neuromorphic-enabled FC, delivering early imaging of microscale objects under an event-based vision.All these endeavours indicated the potential of adopting a neuromorphic vision and architecture in providing cell measurements, exhibiting a high level of accuracy and possible advanced throughput of NIC.

Sample preparation
Polystyrene-based microparticles were prepared in the initial testing to evaluate the performance of neuromorphic vision in capturing microscale targets.About 10 wt.% concentrated 8 µm and 15 µm microparticle solutions were acquired (Sigma-Aldrich, USA) to emulate the relative size of human blood cells (erythrocyte, leukocyte) and abnormal cells (e.g.cancer cells).8 µm particle solution was diluted in 1:250 with deionised water to maintain a sufficient count and spacing between particles, preventing statistical inadequacy in sample size and possible aggregation.As the concentration of the particles was distributed by weight, 15 µm particle solution was diluted in 1:38 to obtain a similar number with diluted 8 µm particles.Once the dilutions were completed, the sample particles were introduced into respective microfluidic channels to acquire neuromorphic feedback on counting and size estimation.To establish a baseline comparison, the same concentrations of the particles were prepared and analysed by an FC (CytoFLEX LX, Beckman Coulter) to obtain the measurement of counts and sizes to ensure the validity of the results.
Human monocytic cell line, THP-1 cells were collected from Barry Slobedman laboratory in the Charles Perkins Centre at The University of Sydney.The cells were maintained in a 37 • C and 5% CO 2 incubator and harvested in this experiment to examine the compatibility of neuromorphic vision with real sample cells and determine whether the complexity in cells can degrade or impact the visualisation and detection algorithm.The targeted cell line was prepared at the concentration of 1 × 10 6 cells ml −1 .The cell solution was first centrifuged at a speed of 400 g for 5 min for sample separation.Then, the cell pellet was collected cells and re-suspended with an FC buffer made of 0.02% sodium azide, 0.5% bovine serum albumin and 2 mM ethylenediaminetetraacetic acid in phosphate-buffered saline.The sample solution was vortexed and equally distributed into aliquots for neuromorphic analysis and CytoFLEX baseline comparison.
FITC-marked melamine resin microparticles with a diameter of 6 µm were purchased (Sigma-Aldrich, USA).2.5 wt.% concentrated solution was diluted into 1:100 with deionised water and allocated onto a glass slide for fluorescence observation under a microscope.

Microfluidic platform
A 60 µm-height and 100 µm-width microfluidic chip was fabricated utilising a standard photolithography protocol [20].The chip contains one inlet and outlet channel to create an essential delivery and visualisation pathway for the target samples.In addition, the channel employed a height of 60 µm to focus the particles into a relatively limited plane to avoid a broad depth range and excessive calibration caused by it.
The microparticles in the size of 8 µm and 15 µm and THP-1 cells were adopted in this experiment to investigate the capacity of asynchronously activated pixels in tracking microscale objects.The diluted sample solution was loaded into a 1 ml syringe and distributed into the channel by actuation of a syringe pump (LEGATO 200 Syringe Pumps, KD Scientific Inc., USA) at the flow rate of 10 µl min −1 .As the objects travelled through the section of interest, the physical characteristics of the targets were collected by the selected sensors and later analysed for object counting and size estimation.The same dilution factor was applied to the samples and analysed via CytoFLEX to provide outputs on total counts and estimated sizes for the baseline comparison.

Imaging setup
An event-based camera, Evaluation Kit 4 (EVK4, Prophesee, France) consisting of 1280 × 720 pixels, 4.86 × 4.86 µm pixel size, time resolution equivalent to >10k fps, >120 dB in DR and 92% quantum efficiency was mounted into the microscope (IX73 Inverted Microscope, Olympus, Japan) for 10× magnification of the microscale objects and monitoring the dynamics of the field of interest.Metavision Studio software was utilised to intimate recording and adjust the imaging setting regarding accumulation time and event-generated threshold to optimise the visibility.
An sCMOS camera, Prime BSI Express (Teledyne Technologies, USA) was ported into the microscope simultaneously to verify the event stream captured by EVK4.The camera consists of 2048 × 2048 pixels, 6.5 × 6.5 µm pixel size, 48 fps at full frame, 74.98 dB in DR and 95% quantum efficiency.Micro-Manger and ImageJ software were paired to control and facilitate image acquisition by auto-adjusting the brightness and contrast.
For the fluorescence analysis, an additional CMOS camera, Alvium 1800 U-240 m (Allied Vision, Germany) consisting of 1936 × 1216 pixels, 3.45 × 3.45 µm pixel size, 178 fps, 72 dB in DR and 64% quantum efficiency.The camera was employed to serve as a benchmark for evaluating the performance of traditional photosensors in registering fluorescent signals.Image Acquisition Toolbox in MATLAB was used to interface with the camera and deliver visual feedback.

Object counting and size estimation
The function of object counting and size estimation is accomplished by an open-access metavision_psm.pyfrom Prophesee.The code applied a Particle Size Estimation (PSM) algorithm to count and estimate the size of the ongoing objects.This algorithm utilises a series of lines of interest to record the triggered events and clusters the pixels within a certain distance as a single object.As the particles were observed under 10× magnification, the number of lines was set to 10 within the default section area.This measure was taken to ensure precise matching of detections across multiple lines and to prevent any potential ambiguity.
To optimise the integrity of objects in the recorded data, a Metavision pre-processing code on polarity, Polarity Filter Algorithm was implemented to include only one polarity to enhance object sharpness and avoid multiple detections on the same object.With particles as the main subject, polarity was configured to exclusively display OFF (decreased light intensity) events for optimal detection with the PSM algorithm.Moreover, accumulation time can be customised to improve the completeness of spatial distribution on the captured objects, which can pair with the polarity setting to maximise the performance outputs.In this work, as particles have different sizes and velocities during flow, accumulation time in a range of 100-1000 µs was selected according to various scenarios.
To reduce the noise caused by random light fluctuation while maintaining the accuracy in detecting particles, A Metavision noise filter, Activity Noise Filter Algorithm was adopted to eliminate the unwanted noise that can lead to misdetection.This filter validates events that occur similarly around its previous coordinates and within the selected time window.An adjustable parameter for the time window was defined in the range of 150-500 µs.Prior to the recording, an event threshold can be defined to determine the light contrast required for generating an event.Increasing this value can lead to a higher light contract to trigger an event, which leads to a reduction in noise but also compromises the sensitivity in capturing particles of interest.Deceasing this value can elevate the frequency of noise but increase the sensitivity in detecting objects with minor light fluctuation.In this experiment, the biases setting was left on default as the subjects were travelling at an acceptable velocity and sufficient illumination.By enumerating these objects and including their respective spatial extent with the temporal characteristics, the overall counts and size estimation are calculated.

Fluorescent signal analysis
To evaluate the efficiency of receiving fluorescent signals facilitated by HDR, 10 µl of FITC-marked microparticle solution was allocated on a glass slide and observed under the microscope.In this experiment, three lighting conditions were deployed, bright-field, and fluorescence mode with two distinct illumination intensities.The fluorescence illumination was activated by CoolLED pE300 white with light intensity set on 2 and 20 in the blue region to create different degrees of FITC excitation and emission.The view of the microscope under 20× magnification was captured by the aforementioned cameras under varying illumination conditions.

Results
The proposed schematic of our NIC is illustrated in figure 1 to demonstrate the working principle and expected future implementation.As the sample flow is introduced into the channel, the EVS will detect the ongoing subjects once it enters the view.The captured imaging will be fed to the processing centre for denoising, filtering, object counting, size and morphology estimation, determining whether the object is of interest and sending a signal to the dielectrophoretic sorter for sorting decisions.Once the DEF device receives the outcome, the electrodes gently pull the targeted object into the pathway to the collection outlet.The unaltered objects will enter the waste outlet by default geometrical design.
One of the novel attempts that should be continuously explored with EVS is the ability of HDR to capture high-speed fluorescent objects.Figure 2 illustrates the documented throughput of fluorescence IFC with their respective DR, and the work published by Vinegoni et al [14] highlighted that the maximum   [21], Holzner et al [22], Rane et al [23].capacity of the fluorescence detected is majorly dependent on the DR provided.Even though recent efforts have achieved a high throughput with high-resolution imaging in detecting fluorescent-tagged objects, these works, while remarkable, only adopted compensation strategies such as time-delay integration and virtual-freezing technique, the direct correlation and performance of HDR in registering fluorescent signals remain unknown.To our knowledge, there has not been an IFC that exploited such an HDR (>120 dB) compared to EVS, future endeavours on the subject can be highly informative and promising in developing next-generation IFC.
At the current stage, our platform can visualise and perform measurements on the basic physical properties of microscale objects.In figures 3(A)-(F), the precise contour and relative size difference in neuromorphic view can be observed and compared to conventional microscopic imaging captured by sCMOS sensor.Both imaging techniques were competent in delivering essential visual feedback without losing the key integrity of the subjects.Furthermore, when conducting microfluidic delivery with microscale targets, some of the travelling objects will inevitably adhere to the internal surface of the channel, rendering multiple imaging obstructions and deviating the focus from subjects.In neuromorphic, as the necessary movements are required to create the contrast between objects and the background, objects that remain in a static position will not activate the pixels.This unique architecture can mitigate that consequential image  effect caused by attached objects, significantly reducing the labour required for post-imaging processing and focusing on subjects of interest.
The sensitivity of EVS to fluorescent signals was examined in comparison to a standard CMOS, Alvium 1800 U-240 and sCMOS camera, Prime BSI Express.In figure 4, the visions of three cameras were revealed.In the context of a bright-field, all cameras can deliver visual feedback on FITC particles with essential features intact.As the illumination was turned into fluorescence mode, the traditional camera lost its ability to detect any object in a dark background regardless of the intensity of the excitation light.On the contrary, the sCMOS sensor and EVS can depict the spatial content with lower and higher excitation (2 and 20).The difference in the sCMOS camera under weak and higher excitation power is that when the object is excited by the weak illumination, the image shows more background noise during detection, and higher illumination can lead to a more well-defined sample imaging with minimised background noise.With EVS, both lighting conditions can lead to visualisation without a noticeable increase in noise.Owing to the high quantum efficiency (95%), DR (74.98 dB) and aid of auto-contrast provided by Micro-Manager software, the sCMOS sensor has focused and distinct visual feedback in presenting fluorescent targets.However, offering such high sensitivity leads to a compromised frame rate, which only tolerates 43 fps at full resolution (2048 × 2048, 16-bit) and a maximum of 1468 fps with reduced frame resolution (2048 × 128, 11-bit).On the other hand, EVS can support over 10k fps while maintaining its current sensitivity to fluorescent targets without any strategy of imaging processing.We believe that in the context of static imaging, a sCMOS such as Prime BSI Express can generate outstanding resolution and sensitivity, however, in a highly dynamic scene, the limited frame rate can be challenging to capture the ongoing subjects and EVS can be an optimal candidate.
With object counting and size estimation, the number of microscale objects with their respective sizes can be estimated during the flow.In a traditional FC, forward scatter (FSC) refers to the intensity of light that is scattered in the forward direction when an object passes through a laser beam, illustrating the relevant sizes of the detected objects.Side scatter (SSC) refers to the intensity of light that is generally perpendicular to the light source, delivering information on the granularity of the objects.Conventional cytometry scatter plots were provided in figures 5(A), (D) and (G), including the gated population of 8, 15 µm microparticles and THP-1 cells based on their sizes and granularity from FSC-A and SCC-A.For the 8 µm microparticle solution, in table 1 and figure 5(B), the total event count was 3189 with 95.6% gated for 8 µm microparticles, formulating a highly concentrated pattern of distribution around 1.1 M in FSC-A measurements with CytoFLEX; in figure 5(C), the neuromorphic detection was adopted and generated an overall count of 3086 with gating tolerance between ±2 µm in respective size, and yields 98.1% of the population as 8 µm microparticles.For the 15 µm microparticle solution, in figure 5(E), the total event count was 3220 with 92.9% gated for 15 µm particles with CytoFLEX, presenting a concentrated distribution around 3.4 M in FSC-A measurement; in figure 5(F), the overall neuromorphic count was 3898 with 98.7% gated as 15 µm microparticles.Both technologies yield remarkable counting similarities and purities in measuring pure polystyrene-based beads in different sizes, constructing a highly alike size of distribution in describing the sample population.As the sample solutions are prepared with only microbeads, our strategy can compete with conventional cytometry to provide higher purity and accuracy.For THP-1 cells, in figure 5(H), a total event number of 4429 was estimated by CytoFLEX with 40.6% identified as live THP-1 cells, 26.6% dead cells and 28.8% debris; in figure 5(I), the neuromorphic count was 3056 and condition of the cells and division into subsets remain inaccessible owing to the inability to contrast different level of granularity.Although the majority of the ongoing objects in varied sizes can be detected, such discrepancy in the total count can be caused by the significant amount of tiny debris in the cell solution, which can be easily recognised as noise-like events during the flow in neuromorphic.As a result, our platform achieved high correspondence with commercialised cytometry when conducting measurements on pure microscale objects; however, in the context of cell measurements in real practice, the debris, doublets, complexity, viability and cycle of cells can raise the parameters to be implemented to distinguish various components in cell culture.
During the early testing stage, MCF-7 cells were observed under the platform for testing the compatibility of neuromorphic vision with biological samples.In figures 3(M) and (N), in addition to the morphology acquisition, under phase contrast, the internal nucleus-like objects were identified with distinct clarity during the view without the aid of any staining techniques.Such a phenomenon can be supported by the HDR provided by neuromorphic vision.However, as this is the first instance of the event, we cannot conclude an absolute explanation on the matter.Further investigations and parallel studies on various cell lines are required to verify the specific mechanism behind it.Nevertheless, if the assumption is valid, this unique feature can contribute a deeper insight into NIC and can be utilised as an additional metric in conducting cell measurement and sorting.

Conclusion
In this work, we have concluded our first milestone in fluorescence assessment with EVS and performing cell population measurement in terms of object counting and size estimation.The high consistency and purity in describing microbeads in different sizes compared to commercialised cytometry indicated the measurement accuracy and alike characterisation of NIC.Although the function of classifying the cell condition and surrounding events is putting forward, the low latency and event-focused architecture in neuromorphic has the potential to outperformance the conventional IFC in throughput and data optimisation, enabling a prospect of a data-and cost-efficient fluorescence-sensitive high-throughput neuromorphic cytometry.

Figure 1 .
Figure 1.(A) Proposed schema of NIC.The target samples were introduced into the microfluidic channel in a flow.Features of the samples were captured by an event-based camera, sending the inputs into the processing unit for classification and sorting purposes.After decision-making, the target object was directed into the collection channel via a DEP sorter.(B) The current setup of the platform with an event-based camera integrated into a microscope for recording and capturing the sample flow.

Figure 2 .
Figure 2. Performance of the previously reported fluorescence IFC with their incorporated DR.The grey shade highlighted the unattained outputs and DR provided by the neuromorphic vision and sample imaging of different microscale targets.Cited papers in this figure are Huang et al [10], Zmijan et al [21], Holzner et al [22], Rane et al [23].

Figure 4 .
Figure 4. Microscopic imaging captured by Alvinum 1800 U-240 m (CMOS), Prime BSI Express (sCMOS) and EVK4 under varying illumination conditions.The first column on the left illustrated the images captured under a bright-field.The second column indicated the images taken in a fluorescence mode with light intensity set on 2 in the blue light region.The third column illustrated the images taken with light intensity set on 20 in the blue light region.

Figure 5 .
Figure 5. Cytometry and neuromorphic measurements on microparticles and THP-1 cells.(A) CytoFLEX interpretation of 8 µm microparticles based on forward scatter area (FSC-A) and sideward scatter area (SSC-A), gating 95.6% of the population being 8 µm microparticles.(B) Histogram of 8 µm microparticle size distribution based on FSC-A measurement.(C) Histogram of 8 µm microparticle size distribution based on neuromorphic outputs.(D) CytoFLEX interpretation of 15 µm microparticles based on FSC-A and SSC-A, gating 92.9% of the population being 15 µm microparticles.(E) Histogram of 15 µm microparticle size distribution based on FSC-A.(F) Histogram of 15 µm microparticle size distribution based on neuromorphic outputs.(G) CytoFLEX interpretation of THP-1 cells based on FSC-A and SSC-A, gating 40.6% as live THP-1 cells, 26.6% dead cells and 28.8% debris.(H) Size distribution of live THP-1 cells, dead cells and debris based on FSC-A.(I) Histogram of THP-1 cell size distribution based on neuromorphic outputs.

Table 1 .
CytoFLEX and neuromorphic output on the total event count of 8 µm, 15 µm microparticles and THP-1 cells in their sample solution.