Lensless dual-color fluorescence imaging device using hybrid filter

In this study, a dual-band hybrid filter that achieves high excitation light rejection performance in a lensless imaging system was fabricated and incorporated into an imaging device. The hybrid filter consisted of interference and absorption filters, and a fiber optic plate (FOP). The interference filters were attached to both sides of the FOP, which was placed on top of the absorption filter to suppress the decrease in spatial resolution. In addition, the lamination order was optimized to achieve a high fluorescence observation performance. The fabricated hybrid filter was mounted on an image sensor and had the ability to indicate the green and red fluorescence components.


Introduction
Fluorescence microscopy is an effective technique of imaging fluorescent molecules commonly used in many applications in biological studies, such as the visualization of small living cells, molecule-molecule interactions, and real-time protein conformation changes with high sensitivity and specificity. These features make it interesting and have many advantages over other techniques. 1,2) However, there are limitations to using a lens-based microscope in some experiments, such as long-term cultured-cell observation in an incubator and in vivo imaging with very low invasiveness. In these scenarios, the acceptable device dimensions are limited by the size of the incubators or host animals. Because a lensless microscope enables imaging without any lens elements, [3][4][5][6][7][8][9][10][11][12][13][14][15][16][17] it has many advantages in terms of lightweight, and small size. Simple optics and image processing techniques can simultaneously achieve a large field of view and high resolution. Thus, recently, the lensless microscope has been vital in various research applications, such as in-line digital holographic microscopy, super-resolution 3D imaging, dense image reconstruction, iterative phase recovery, and color imaging in lensless on-chip microscopy. However, most of them are bright-field observations, and there have been few reports of their application to fluorescence observations.
An important aspect of fluorescence imaging is fluorescence labeling. The luminescence intensity is typically weak with respect to the excitation light. Thus, the sensitivity of the imaging device must be considered. An interference emission filter with high wavelength selectivity is frequently used in lens-based fluorescent microscopes. However, in a lensless fluorescence imaging system, an interference filter is not used in the best condition because of the angle dependence of the transmission characteristics. In the lensless setup, the excitation light is scattered by an observation target and incident to the filter at various angles. Because the transmission spectrum shifts with the incident angle transmitted through the interference filter, the scattered component passes through it. The intensity of the scattered component is frequently higher than the target fluorescence intensity. Thus, it significantly affects the results of fluorescence observation and causes poor performance. Even if the interference filter entirely rejects normal incident light, it cannot operate properly in the lensless setup. Thus, it significantly affects the results of fluorescence observation and causes poor performance. 5) An alternative is to use an absorption filter because its transmission spectrum is independent of the incident angle. 8) However, autofluorescence can be emitted by this filter. Therefore, the use of these filters is still insufficient for lensless fluorescence imaging systems.
To overcome this problem, our research group has proposed a novel hybrid emission filter that combines the interference and absorption filters. [13][14][15] Both filters are combined via both sides of a fiber optic plate (FOP) attached to an image sensor to achieve a highly sensitive fluorescence detection performance. Recently, this hybrid emission filter has been used in many studies. A hybrid filter was developed to detect dual-color fluorescence with single-color excitation for the purpose of imaging fluorescence resonance energy transfer signals. 15) However, it is also important to detect multiple fluorophores to evaluate activities from different aspects using multiple fluorophores. [18][19][20] In such a scenario, a specific excitation light must be used to excite each fluorescent target. Thus, the emission filter must reject the excitation wavelengths and transmit wavelengths that match the fluorescent targets to understand the cell characteristics efficiently.
In this paper, we demonstrate a hybrid filter that enables dual-color fluorescence observation through a dual-color excitation light to observe green and red fluorescence components, which are the most commonly used fluorescent targets. A dual-band interference filter was realized by forming a bandpass interference filter on the FOP. In addition, absorption filter layers were fabricated on one side of the FOP, corresponding to green and red fluorescence excitation wavelengths (Fig. 1). Subsequently, the fabricated dual-color hybrid filter was mounted on a color CMOS image sensor. The performance of the hybrid filter was improved by optimizing the filter configuration compared with a previous study. 21) In addition, we demonstrated green and red fluorescence imaging and the improvement of fluorescence separation performance through image processing.

Hybrid filter structure for dual-color fluorescence imaging
Fluorescent dyes and proteins of various colors have been developed, and various activities of living organisms can be color-labeled using their specific labels for fluorescence observation. [18][19][20] In this study, green and red fluorescence components excited by dual-color excitation light were observed. The hybrid filter used to observe dual-color fluorescence was formed by combining the interference and absorption filters. Here, the order is important, and the top layer must be an interference filter. Even if it is easier to form with the absorption filter as the top layer, 22) the high performance of imaging cannot be achieved because the absorption filter itself emits fluorescence. In contrast, a highperformance hybrid filter was realized in our previous studies by forming an interference filter on the FOP. [13][14][15] In addition, an excitation filter for a small fluorescence imaging device was also manufactured by applying this technology. 23) An FOP was also used in this study. A longpass interference filter was attached to one side of the FOP, and a notch interference filter was attached to the other side to form a bandpass characteristic. Figure 2 shows the transmittance spectra of the longpass and notch interference filters. The excitation light wavelength was assumed to be approximately 450 nm for green fluorescence excitation and 594 nm for red fluorescence excitation. This bandpass interference filter was designed to have a high rejection performance for these wavelengths.
The order of the interference filters was considered to achieve high excitation light rejection performance. A part of the incident light is scattered in the FOP, and the transmission characteristic of the interference filter shifts with the incident angle of the excitation light. Thus, the light rejection performance of the interference filter below the FOP decreases because the highly angled excitation light transmits through the interference filter. Figure 3 shows the result of the excitation light scattered in the FOP with the interference filters observed when irradiated with a light source of 594 nm. It exhibited a high excitation light rejection performance when the notch interference filter faced the light source side [ Fig. 3(a)]. In the opposite direction, some of the components scattered in the FOP passed through the notch interference filter [ Fig. 3(b)]. Thus, the shape of the beam appeared bright. The same was observed for the blue light, and when the FOP was used as an interference filter substrate, the effective performance of the bottom-side filter degenerated.
In addition to these factors, when using an FOP, it is necessary to consider the autofluorescence of the FOP layer. Figure 4 shows the autofluorescence from an FOP with interference filters when irradiated with a 440 nm laser. The observation was performed using a yellow filter, and the excitation light component was removed. Blue light was irradiated from the notch interference filter [ Fig. 4(a)] and the longpass interference filter [ Fig. 4(b)]. Because the longpass and notch interference filters were directly attached to the surface of the FOP, the strong excitation light component could not be removed during irradiation. When the blue excitation light passed through the notch interference filter side, the autofluorescence of the FOP increased.
An absorption filter was also included to absorb the scattered components and transmit light at specific wavelengths to identify the dual-band characteristic. Generally, it  is difficult for an absorption filter to have a bandpass characteristic with high wavelength selectivity. In most scenarios, it is not possible to decrease the absorption of excitation light and increase the transmission of fluorescent components. This study aimed to observe the fluorescence, and a high reduction rate of excitation light was required. Yellow dye (Valifast Yellow 1108, Orient Chemical, Japan) was used as a filter with longpass characteristics. This dye has an absorber in the blue wavelength band and exhibits low autofluorescence. A dye with a sharp absorption peak (FDG-007, Yamada Chemical Co., Ltd., Japan) was used for the excitation light absorption of the red fluorescent protein. Figure 5 shows the transmittance spectra of the yellow and blue absorption filters. The blue filter exhibited a high absorption at approximately 594 nm. Because the high absorption bandwidth was narrow, it was necessary to shorten the wavelength of the excitation light.
The relationship between the transmittance characteristics and the absorption filter thickness of each filter is shown in Fig. 6. Figure 6(a) shows the results of the yellow absorption filter. The excitation light had a wavelength of 434 ± 8.5 nm. In this filter, there was an inflection point near 2.5 μm, but there was a significant deviation observed at approximately 1.5 μm owing to the addition of the FOP, which resulted in its autofluorescence appearing stronger than the autofluorescence of the absorption filter. In particular, when the light was incident on the notch interference filter side, strong fluorescence was emitted [ Fig. 4(a)]. The effective transmittance was reversed when the light was incident on the longpass interference filter side [ Fig. 4 As shown in Fig. 6(b), the blue absorption filter was irradiated with excitation light at a wavelength of 594 ± 5 nm. According to the Lambert-Beer law, the transmittance decreases exponentially as the thickness increases up to a thickness of 5 μm. In addition, the result of attaching the FOP   with an interference filter exhibits a similar curve, and the intensity of autofluorescence is small in this wavelength band.
The order of the absorption filter was yellow and blue from the excitation light incident side because the autofluorescence was generated at a wavelength of 550-610 nm by the yellow filter, and it was removed by the blue absorption filter. Figure 7 shows a comparison of autofluorescence in the order of the absorption filters. The yellow and blue absorption filters were attached to the FOP on the notch interference filter side, respectively, before being mounted to the image sensor to avoid strongly generated autofluorescence from the blue absorption filter.
The order of the interference filters must be appropriately determined based on the performance of each filter. In this study, the autofluorescence of the FOP irradiated by blue excitation light limited the effective transmittance (Fig. 4). Thus, a longpass interference filter was arranged on the top of the FOP, and a notch interference filter was arranged on the other side. Based on the above results, the filters were ordered as follows: longpass interference filter, notch interference filter, yellow absorption filter, and blue absorption filter. The thicknesses of the yellow and blue absorption filters were approximately 2 μm and 5 μm, respectively.

Filter preparation and device assembly
In the configuration of the proposed hybrid filter, the longpass and notch interference filters were attached to the uppermost and bottom sides of the FOP, respectively. The yellow and blue absorption filters were attached to the bottom-side. This hybrid filter was mounted on a commercially available color CMOS image sensor, which is used as the imaging device [ Fig. 9(a)]. A photograph of the fabricated hybrid filter with a lensless imaging device is shown in Fig. 9(b). The preparation of the hybrid filter and the device assembly process is described below.
(2) An amorphous fluoropolymer CYTOP-M (AGV Chemical, Japan) was coated on a cover glass (24 mm × 24 mm) through spin coating [ Fig. 10(a)]. This film was prepared as a substrate, enabling the absorption filter to be easily peeled off from the cover glass. yellow absorption filter using epoxy resin (Z-1, Nissin resin, Japan) and left for 24 h until it was solid and dry [ Fig. 10(c)]. Subsequently, the FOP was removed from the cover glass by cutting the edges of the cured epoxy and yellow dye film. This process was performed to obtain a yellow absorption filter on the FOP [ Fig. 10(d)].    7) The FOP with the yellow absorption filter surface was attached to the blue absorption filter using epoxy resin and left for 24 h to solidify and dry [ Fig. 10(f)], and it was then removed from the cover glass [ Fig. 10(g)].
This process was performed to obtain a hybrid filter.

Device assembly
The glass lid from the commercially available CMOS image sensor (IMX249, Sony, Japan) was removed from the package. The details are provided in the Appendix. The fabricated hybrid filter was mounted on a color CMOS image sensor using an epoxy resin. This image sensor has 1936 × 1216 effective pixels and a 5.86 μm ×5.86 μm pixel size. The glass lid that covered the surface of the sensor package was removed, and a 1.2 mm thick FOP was added before mounting the hybrid filter. Figure 11 shows the sensitivity spectrum of the proposed device. The stacked longpass and notch interference filters and yellow and blue absorption filters were mounted on an image sensor to realize the dual-band of the green and red fluorescence components. Thus, the sensitivity spectrum was measured using a spectrometer (micro-HR, Horiba Jobin Yvon) with a white light source (HPLS30-04, Thorlabs, USA). The curves of the emission spectra of green (eGFP) and red (mPlum) fluorescent proteins and the sensitivity spectrum of the proposed device are shown in Fig. 11. The monochrome light from the spectrometer was irradiated onto the proposed device in the normal incident direction. The device is not sensitive to the blue and yellow excitation light wavelength bands. In the proposed method, the interference filters are stacked with the FOP sandwiched between them. Because light is scattered in the FOP, the influence of multiple reflections between the interference filters was not observed in the measurement results. The proposed device is sensitive to green and red light at the expected transmittance wavelengths of approximately 510 and 650 nm, respectively.

Sensitivity spectrum of the fabricated device
As mentioned earlier, in this study, a commercially available color image sensor equipped with a color filter was used. The red pixels exhibited relatively high color selectivity, but the blue and green pixels had partial sensitivity to red in addition to the wavelength band of green fluorescence. Thus, even when the signals of each pixel were extracted, the image contained signals other than green or red. Image processing was required to obtain a clear colorseparated image.

Experimental setup
The proposed imaging device was placed on the stage of a fluorescence microscope (BX51WI, Olympus, Japan) under an objective lens (Mplan N5×/0.10NA, Olympus, Japan). This fluorescence microscope was used to obtain the reference images, and a mercury lamp (U-RFL-T, Olympus, Japan) was used as the light source. Excitation filters were also used with the fluorescence microscope to limit the central excitation wavelength for the blue (MF434-17, Thorlabs, Japan) and yellow (86-737, Edmund Optics, USA) excitation lights at 434 and 594 nm, respectively. Finally, the filter cubes in the fluorescence microscope were switched to the blue and yellow excitation lights during the experiment.
During the experiment, the proposed imaging device was irradiated with blue or yellow excitation light at a normal incident angle, corresponding to the excitation wavelength of the fluorescent targets. A schematic and photograph of the dual-color fluorescence imaging experimental setup are shown in Fig. 12.

Dual-color fluorescence imaging from fluorescence microbeads
In this fluorescence imaging, yellow-green fluorescence microbeads (F8844, Thermo Fisher Scientific, USA) and crimson fluorescence microbeads (F8839, Thermo Fisher Scientific, USA) were used to contribute a fluorescence light drop to the proposed hybrid filter. The excitation and emission spectra of the yellow-green and red (crimson) fluorescence microbeads were 505/515 nm and 625/645 nm, respectively. Both types of fluorescence microbeads were the same, with a diameter of 15 μm. The blue (434 nm) and yellow (594 nm) excitation lights were irradiated to excite the yellow-green and crimson fluorescence microbeads. The The Japan Society of Applied Physics by IOP Publishing Ltd imaging area was 11.2 mm × 6.8 mm obtained from the imaging device. Figure 13 shows images captured using the proposed device. The yellow-green and red fluorescence microbeads were spread on a cover glass, and the beads were placed on the imaging device. The fluorescence microbead images using blue and yellow excitation light were captured separately by the proposed device. The background of the captured images was almost black and strongly contrasted with both the yellow-green and red fluorescence microbeads. The results indicated that the proposed hybrid filter can reject both blue and yellow light excitation. Figure 13(a) shows an image when irradiated with blue excitation light. Red beads were arranged on the left side of the figure, and yellow-green beads were arranged on the right side. They appeared orange and yellow, respectively, owing to the fluorescence characteristics of the beads and the sensitivity characteristics of the sensor. The yellow-green beads had a fluorescent edge that extended to the red band. In addition, although the efficiency of the red beads was low, the absorption spectrum extended to the blue wavelength and exhibited a slight fluorescence. In addition, the fluorescence of the red beads was detected even in blue and green pixels because the blue and green pixels were sensitive to red (Fig. 11). Figure 13(b) shows an image when irradiated with a yellow excitation light. Here, only the red beads were bright fluorescent and appeared orange. This result indicates that the image of the red beads can be easily obtained by switching the excitation light.

Image processing to separate green and red microbeads
As the results in the previous section indicate, the color image sensor filter exhibited a difference in sensitivity for fluorescence from the green and red beads but could not detect them separately. Therefore, image processing was required to obtain an image in which the two types of fluorescence sources were separated. However, simple image processing was sufficient because an image of only red fluorescent beads could be obtained by yellow excitation [ Fig. 13(b)].
The red bead image was removed by subtracting the yellow-excited red pixel image from the green pixel image of the blue-excited image. Here, the red pixel image was multiplied by a coefficient that canceled the red bead image. The composite images of the red and yellow-green beads thus obtained are shown in Fig. 14. The image obtained by the image processing is shown in Fig. 14(a). Enlarged images of the part surrounded by the white dashed line in Fig. 14(a) were captured using the proposed device [ Fig. 14(b)] and a fluorescence microscope [ Fig. 14(c)]. Although the spatial resolution of the proposed device was low, the green and red beads were clearly observed.

Discussion
In this study, an imaging device for dual-color fluorescence observation was developed. A hybrid filter composed of longpass and notch interference filters was fabricated on both sides of the FOP of a device to realize its bandpass characteristic. Yellow (VY1108) and blue (FDG-007) absorption filters were also used in the proposed hybrid filter structure to absorb blue and yellow excitation light.

Transmittance characteristics and hybrid filter configuration
Typically, a certain amount of scattered light occurs even when light passes through the FOP, reducing the performance of the interference filter. Thus, the transmittance  characteristics of each filter were required to determine the configuration of the proposed hybrid filter.
In the hybrid filter configuration, the longpass interference filter was arranged on the top of the FOP, and the notch interference filter was arranged on the opposite side to avoid a decrease in the excitation rejection performance. This arrangement exhibited better characteristics than the reverse scenario. The yellow absorption filter (VY1108) and blue absorption filter (FDG-007) were arranged on the lower side of the FOP to absorb the excitation light.
For green fluorescence, the blue excitation light could be rejected by six orders of magnitude (Fig. 6). Although the green transmission wavelength band was limited by using a bandpass filter, VY1108 exhibited a higher wavelength selectivity than the yellow dye (VY3150) used in our previous report 13,14) and efficiently transmitted green fluorescence. The imaging results with fluorescent beads exhibited the ability to detect green fluorescence. Table I shows a comparison of the device characteristics and performance from this study with those from our previous report.
For red fluorescence, the yellow excitation light could be reduced by five orders of magnitude or more [ Fig. 6(b)]. In this wavelength band, the autofluorescence of the FOP and FDG-007 was low, and the rate of excitation light reduction by the absorption filter layer could be increased. However, the excitation light removal performance of the notch interference filter was not as high as that of the longpass interference filter in the blue band. In addition, the excitation light passed through the notch interference filter in the hybrid filter configuration after passing through the FOP. Each filter was formed on both sides of the FOP to avoid warpage due to filter formation. A problem remains in that the filter performance cannot be utilized because of scattering by the FOP. However, the characteristics can be improved if the notch and longpass interference filters can be arranged on one side without any problems.

Limitation of fluorescence separation
The green channel becoming red is a problem. In this study, although the image processing method was used for improvement, the sensitivity of red was higher than that of green, and artifacts tended to remain even after image processing in the vicinity of the region where red fluorescence was extremely strong. In this filter configuration, the filter did not remove the longer-wavelength side of the red-  The Japan Society of Applied Physics by IOP Publishing Ltd wavelength band. However, it could be improved by making only a specific area transparent and adjusting the green and red sensitivity balance. In addition, at a wavelength of approximately 700 nm, the sensitivity of the green pixels was approximately 40% of the red pixels, but the accuracy was expected to improve by reducing the sensitivity in this area. In this study, a commercially available color image sensor was used to observe green and red. However, it is also possible to apply a method that does not use a filter to observe intermediate wavelengths. 24) 6.3. Spatial resolution The spatial resolution decreases with distance in simple contact imaging devices such as the one demonstrated in this study. In particular, this effect is significant for fluorescence observation because fluorescence from the observation target is emitted isotropically. The resolution of the proposed method was lower than that of the fluorescence microscope [ Fig. 14(b)]. This can be improved by using a deconvolution method and an image restoration method based on sparse sampling. However, it is difficult to efficiently restore dense images. A method that combines image processing with a simple optical system using an incident angle limitation or a phase mask has been studied in recent years. [25][26][27][28][29][30] These optics have a relatively high compatibility with the proposed method. It is necessary to remove the FOP layer, but because a distance can be left on the sensor, it is expected that this can be solved by using an interference filter that uses the film formation or transfer of an interference filter on a thin glass substrate. 31,32)

Conclusions
This study aimed to develop a highly sensitive dual-color fluorescence observation device using a lensless system. A dual-band hybrid filter was fabricated to achieve a high excitation light rejection performance in the blue and yellow wavelength bands. In addition, the filter transmitted green and red fluorescence excited by these excitation wavelengths. The device was mounted on a commercially available color image sensor. Green and red fluorescent beads were successfully distinguished using image processing on the images obtained by switching the excitation light. From the results, it is expected that this device will be used to observe the activity of cultured cells owing to their small size.