Zero crossing detection algorithm based on an MLP neural network for differential confocal microscopy

Differential confocal microscopy is widely used because of its ultra-high axial resolution. The surface gradient results in light loss, which decreases the slope of the differential response signal at zero crossing. At this point, when the signal-to-noise ratio is fixed, the traditional linear fitting method to determine the position of zero crossing is subject to significant error influence. To solve these issues, this paper proposes a zero crossing detection algorithm based on a multilayer perceptron (MLP) neural network. Experimental results reveal that the proposed algorithm is more robust and capable of better zero crossing extraction. When numerical aperture (NA)=0.4, the average error is 16.9 nm, which is 55.4 % higher than that of the traditional linear fitting algorithm. The proposed algorithm has a high potential for use with the differential confocal sensor to measure unknown steep surfaces.


Introduction
Confocal microscopy has the unique ability to perform axial chromatography and has superior transverse resolution [1].Combined with laser measurement, Raman spectroscopy, microelectromechanical system (MEMS), and other related technologies, confocal microscopy can achieve high spatial resolution measurement of sample surface morphology and spectral information [2][3][4].When using confocal microscopy for microscopic measurements, the samples should be scanned axially layer by layer with a specific scanning interval to obtain the light intensity response curve.The maximum value of light intensity is determined by fitting the light intensity response curve.The axial position corresponding to the maximum value of light intensity is the focal position of the sample surface.However, confocal microscopes have many problems, such as power instability of the source and common-mode noise of the measurement circuit and photodetector.The team led by Wang Fusheng developed an optical probe for the differential confocal microscope in 2000 [5].In a traditional confocal microscope, a beam splitter mirror and a pair of photodetectors connected in differential form are added to the reflection path of the focusing lens.The high spatial resolution of the confocal microscope is enabled by the unique dualreceive optical path arrangement, which allows for bipolar absolute and high-precision tracking and aiming measurements.In addition to solving the abovementioned issues of the traditional confocal microscope, this work can utilize the linear interval operation mode of the differential confocal signal for fast scanning, making it an ideal choice for non-contact optical probes.During the data acquisition process, the non-contact probe is carried by a multi-dimensional motion mechanism for sampling.A three-dimensional reconstruction is then carried out to obtain the surface topography data.A few published literature in the field of surface reconstruction has proved that surface reconstruction using surface gradients and discrete three-dimensional coordinates has better detection ability than using only three-dimensional coordinates [6][7].In our previous manuscripts [8][9], we proposed a new measuring unit that can be used in the differential confocal microscope structure, as well as the corresponding algorithm that can accurately obtain the surface gradient of the surface to be measured while obtaining the three-dimensional spatial coordinates of the point to be measured.
However, when the surface of the measured object is tilted by θ°, the optical axis of the returned beam will be tilted by 2θ° according to the law of reflection.As a result, some light cannot return to the main light path through the pupil of the micro objective lens, resulting in energy loss.Furthermore, differential pinholes at the front end of the detector face this problem, as shown in Figure 1.Therefore, a system with a high numerical aperture (NA) utilization rate because of tilt will cause an overall energy decline, which will eventually lead to a change in the differential confocal curve and a decrease in the zero crossing slope, affecting the system resolution.Ultimately, it will affect the measurement accuracy of a differential confocal microscope optical probe system.Using a neural network is a common method for dealing with nonlinear problems.At present, neural networks have been widely used in the field of optics to determine the relationship between input and output, such as spectrometry [10], imaging, sensor calibration [11], and optical measurements.In this work, we propose a method for correcting the zero crossing drift of the differential confocal microscope signal caused by tilt.We obtain the measurement data for the differential confocal microscope at different tilt angles and the initial position of axial scanning through a large number of experiments.We then use the neural network method to perform multiple regression for surface errors and correct the system resolution and uncertainty.

Principles of Measurement
The structure of a differential confocal microscope measurement system is shown in Figure 2. The light beam is generated from a fiber laser and conducted through the fiber into the collimating lens, forming parallel light.It is worth noting that the light emitted from the single-mode fiber here can be approximated as an ideal point source.Because of the convergence of the microscopic objective lens, the parallel light is focused on the surface under test (SUT), forming point illumination.When the SUT has a slope of θ°, the optical axis of the returning beam will be shifted by 2θ° according to the law of reflection.The return beam passes through the microscope and becomes collimated again.After entering beam splitter B, the collimated beam is divided into pre-focus and postfocus measurement beams.The pre-focus measurement beam is shot into pinhole B placed at f-μm after passing through the focusing lens, where f is the focal length of the focusing lens.Similarly, the focus is measured before the beam enters f+μm at the position placed in pinhole A. The two measuring beams pass through the pinhole and enter the corresponding detector.The SUT is scanned along the axial direction of the system with the micro-objective carried by the PZT.IA and IB curves are formed at the output ends of the two detectors.
Assuming that the optical field distribution emitted by the fiber is U 0 (x 0 , y 0 ), the parallel optical field U 1 (x 1 , y 1 ) after passing through the collimator can be determined using the Huygens-Fresnel theorem: where the wave number k is 2π/λ, the focal length of the collimating mirror is f c , the constant coefficient is m 1 , and P 0 (x 0 , y 0 ) is the pupil function of the collimating lens.A light field U 2 (x 2 , y 2 , θ) is formed on the surface of SUT after the collimated beam passes through the microscopic objective.
where m 2 is the constant coefficient, P 1 (x 1 , y 1 ) is the pupil function during the focusing link of the microscopic objective lens, u is the defocus quantity, and f 0 is the focal length of the microscopic objective lens.It should be noted that the gradient of SUT is a two-dimensional tilt, which can be regarded as a vector composed of tilt direction and tilt depth.Therefore, tilt can be represented as θ ⃗ (x θ ,y θ ,θ).
Wang et al. changed the pupil function of the microscope after tilt to simulate tilt.When the depth of the sloped object to be measured is θ, U 3 (x 3 , y 3 , θ) can be derived from the light field distribution on the exit pupil surface of the micro objective: Here, m 3 is a constant coefficient, and P 3 (x 3 , y 3 , θ) is the pupil function of the microscopic objective for detecting optical links.For further information on the above expression, please refer to relevant literature.The distribution of the light field at a pinhole before and after the focal point can be obtained as follows:   4. As can be seen from Figure 4, the greater the depth is, the smaller the zero crossing slope of the differential confocal curve is.This means that the sensitivity and resolution of the sensor gradually become lower with an increase in depth.The measurement uncertainty of the system becomes higher, affecting the overall performance of the measurement system.

Experimental data collection
The experimental device is shown in Figure 5.A coherent light source ((LP642PF20, Thorlabs, Newton, NJ, USA)) generates a monochromatic beam at a wavelength of 642 nm with good Gaussian distribution, is transmitted through optical fiber, and becomes parallel after passing through the collimator.After the parallel light passes through the 9 mm microobjective (LMPLFLN 20x, Olympus, Tokyo, Japan) with an NA of 0.4, it is focused onto an ultra-precision six-axis displacement table (H-811.I2, ±10, Physik Instrumente, Karlsruhe, Germany) load on a plane mirror.After passing through a focusing lens (LA1207-A, Thorlabs, Newton, NJ, USA) with a focal length of 100 mm, the return beam is divided into pre-focus and post-focus detection parts by a beam splitter.The two detection parts use the same system parameters.The pinhole size is 50 μm, and the defocus is at 650 μm.The beam passes through the corresponding pinhole and is collected by a detector placed behind the pinhole.Firstly, a precise six-degrees-of-freedom displacement platform is used to realize a tilt with an acquisition interval in the range of 0-10° and a step length of 0.25°.At the same time, the objective lens driver performs axial scanning with a step size of 10 nm and a scanning interval of 100 µm to obtain data.Secondly, the above work is repeated 100 times to obtain 10 × 100 data sets.Thirdly, each group of data obtained in the previous step is split equally at an interval of 1 µm, and a total of 100000 data sets are obtained.Finally, 100000 groups are randomly divided into training, verification, and test sets in the ratio 8:1:1.

MLP neural network construction
A multilayer perceptron (MLP), developed from a single-layer perceptron, is a type of feedforward supervised learning neural network, including an input layer, an output layer, and at least one hidden layer.The different layers are fully connected; any neuron on the upper layer is connected with all the neurons on the lower layer, and the connections between neurons are given relevant weights.The training and learning algorithm constantly adjusts these weights during the iterative process so as to minimize the prediction error and provide good prediction accuracy.MLP has the characteristics of excellent nonlinear mapping ability, high parallelism, self-adaptability, and high fault tolerance.It can solve the nonlinear relationship between the differential confocal signal and the zero crossing position, as well as the interference phenomenon of various influencing factors under working conditions.In this study, the collected differential confocal signal data is considered as the sample data of the MLP neural network, and the exact axial position result, namely the corresponding label, is considered as the input variable.[U ⃗ p, q , F p, q ] is randomly divided into training sets [U ⃗ train , F train ] and test sets [U ⃗ test , F test ].Among them, the training set is used to optimize the parameters of each node in the neural network, and the test set is used to evaluate the predictive performance of the optimized network.The input of the training set U ⃗ train is sent to the neural network.After network calculation, the difference signal sampling points satisfying the difference signal strength of the t th sampling point is greater than or equal to zero, and the difference signal strength of the (t+1) th sampling point is less than zero.This corresponds to the distance between the displacement d t ' of the objective positioner and the zero crossing of the difference signal F pre .
Figure 6 shows the structural diagram of an MLP neural network built in this study.In the figure,  ,  , ⋯ ,  is the input, and o 1 is the output of the network.For the i th node in the network,  ,  , ⋯ ,  is the input from each node of the upper layer network, r is the number of neuron nodes of the upper layer network, a is the weight of each node, b is the bias of the node, f is the activation function of the network, and y is the output of the node.The network consists of an input layer, two hidden layers, and an output layer.The input layer contains 2N+1 neurons, and the input data is the difference signal sequence with a length of 2N+1.The numbers of neurons in the two hidden layers are 1024 and 32, respectively.The output layer contains one neuron, and the output is a distance prediction value.The ReLU function is selected as the activation function between each layer of the network.Parameters of each node in the network (parameters of the ith node) include weight a and bias b.Before network model training, the bias value of the node is initialized as zero, and the weight value of the node is initialized as a random number between (-1,1).During network training, the Smooth L1 loss function is selected as the network loss function, and the BP backpropagation mechanism is adopted to optimize the parameters of each node in the network.
The loss value between the real distance value F test of the set and the predicted value F pre of the neural network is denoted as: According to the loss value Loss train , the network parameters were optimized by backpropagation, and the optimization goal was to minimize the loss value Loss train .

Analysis of experimental results
During the training phase, with an increase in the number of iterations, the training loss of the training set becomes smaller and smaller and finally tends to be flat.In the verification set, the loss value is minimum at the 31 st training iteration, and the error increases gradually as the training continues.With the progress of iteration, the loss value can converge, and the result of the 31 st training iteration is used as the final network parameter.
Figures 7 (a) and (b), respectively, show the comparison of the error and standard deviation curves of MLP and LF algorithms with the change in the tilt angle.It can be clearly seen that the algorithm based on the MLP neural network is better than the traditional linear fitting algorithm in terms of its ability to obtain real position, stability, and robustness.In addition, with the increase in the tilt Angle, both algorithms have an obvious tendency to increase the error.However, the error increase trend of the algorithm based on the MLP neural network is smaller, indicating that the algorithm has stronger angle tolerance and will show better measurement ability and robustness in the case of severe surface tilt.Furthermore, the RMS values of the means and standard deviations are shown in Table 1.It can be seen from the table that the error and standard deviation of MLP are increased by 55.4% and 51.9%, respectively, compared with that of LF.

Conclusion
The differential confocal microscope is a good non-contact optical probe.However, when the surface of the test is not perpendicular to the optical axis, the traditional method of linear fitting and the zero point cause a significant error.To overcome this, a DCM orbital detection algorithm based on the MLP neural network is proposed in this article.Through the experiment, the zero detection method based on the MLP neural network shows good measurement accuracy and robustness.The angle of inclination has better allowable capacity compared with the traditional linear fitting, and the error and standard deviation increase by 55.4% and 51.9%, respectively.The proposed algorithm is of great potential for the measurement of unknown surfaces.

Figure 1 .
Figure 1.Cause analysis of optical path loss when slope exists on the cover side.In this work, we propose a method for correcting the zero crossing drift of the differential confocal microscope signal caused by tilt.We obtain the measurement data for the differential confocal microscope at different tilt angles and the initial position of axial scanning through a large number of experiments.We then use the neural network method to perform multiple regression for surface errors and correct the system resolution and uncertainty.

Figure 2 .
Figure 2. Systematic structure of a differential confocal microscope.When the SUT has a slope of θ°, the optical axis of the returning beam will be shifted by 2θ° according to the law of reflection.The return beam passes through the microscope and becomes collimated again.After entering beam splitter B, the collimated beam is divided into pre-focus and postfocus measurement beams.The pre-focus measurement beam is shot into pinhole B placed at f-μm after passing through the focusing lens, where f is the focal length of the focusing lens.Similarly, the focus is measured before the beam enters f+μm at the position placed in pinhole A. The two measuring beams pass through the pinhole and enter the corresponding detector.The SUT is scanned along the axial direction of the system with the micro-objective carried by the PZT.IA and IB curves are formed at the output ends of the two detectors.Assuming that the optical field distribution emitted by the fiber is U 0 (x 0 , y 0 ), the parallel optical field U 1 (x 1 , y 1 ) after passing through the collimator can be determined using the Huygens-Fresnel theorem: D f and D b are the effective detection areas of the pre-focal and post-focal pinhole detectors, respectively.The energy collected on the detector behind the pinhole located in the pre-focal and postfocal positions can then be determined as given below.From these, the differential confocal response curve can be obtained, as shown in Figure 3.

Figure 3 .
Figure 3. Differential confocal response curve.A comparison of the differential confocal microscope response signals at different tilt depths using the above-established algorithm is shown in Figure4.As can be seen from Figure4, the greater the depth is, the smaller the zero crossing slope of the differential confocal curve is.This means that the sensitivity and resolution of the sensor gradually become lower with an increase in depth.The measurement uncertainty of the system becomes higher, affecting the overall performance of the measurement system.

Figure 4 .
Figure 4. Photograph of the experimental setup.

Figure 5 .
Figure 5. Structural diagram of the zero crossing detection algorithm based on MLP neural network.

Figure 6 .
Figure 6.Plot of network training.In the training process, the Smooth L1 loss function is used to calculate the loss value between the real distance value F train of the training set and the predicted value F pre of the neural network, denoted by:

Figure 7 .
Figure 7. (a) Difference between the error values of the MLP and LF algorithms.(b) Comparison of the standard deviations of the MLP and LF algorithms.Table 1. RMS expectations and standard deviations of the extraction errors of different algorithms.

Table 1 .
RMS expectations and standard deviations of the extraction errors of different algorithms.