Independent bilateral-eye stimulation for gaze pattern recognition based on steady-state pupil light reflex

The participants sat on chairs and wore HMD devices (Vive Pro Eye, HTC Corp.) on which the stimuli were presented (figure 2(A). The horizontal/vertical field of view (FoV) of the HMD is about 98°/98°, and the resolution of the display is 2880 × 1600 pixels. The maximum size of the stimulation pattern of this study was under 59° horizontal FoV and under 28° vertical FoV. The device had an integral video-based eye-tracking function that worked under continuous infrared illumination at a sampling rate of 120 Hz.

Zoom In Zoom Out Reset image size

The luminance of the stimuli was calibrated using a chromameter (ColorCAL II, Cambridge Research System, Kent, UK). Visual stimulation and pupil data acquisition were programmed (Unity Technologies, USA) with SRanipal SDK (Ver. 1.3.2.0, HTC Corp.).

The participants gazed at a single stimulus per trial, and their pupil diameters were measured during this time. The pupil sizes were interpolated with a cubic spline fit during blinks. The data for the first 2 s were excluded from the analysis because of the large transient contractions that occur at the onset of visual stimulation [19, 49]. To account for individual variability in pupil size, the pupil data were normalized to a 200 ms baseline period prior to each stimulus onset (i.e. normalized pupil size = absolute/baseline pupil size) [50]. Differences were not observed in the pupil diameter between the left and right eyes of the participants. Considering the well-established consensual response of the pupillary light reflex and symmetry in pupil responses between the two eyes in healthy participants, we only report the diameter of the left pupils [43].

It is possible that different luminance modulation schemes induce larger pupillary oscillations in the pupil [51]. We evaluated the magnitude of the pupillary response to a visual stimulus presented in sinusoidal and square-wave forms to determine the modulation scheme of a stimulus that elicits a strong pupillary response. This preliminary experiment was conducted on four participants. Each stimulus was presented for 50 s and with five frequencies presented in random order. The stimulus was centered on the screen in the right eye. Participants were asked to overtly gaze at the target stimuli. A total of 20 trials (two conditions × five stimulation frequencies × two trial) were conducted for each participant. Discrete Fourier transform (DFT) was used to examine the effect of the visual stimulus on the overall data. The pupil response was divided into 20 s windows with a frequency resolution of 0.05 Hz, and the PSDs were additively averaged at a 90% overlap rate.

The square sequence, ${s_{{F_i}}}\left( t \right)$, for the signal blinking with frequency ${F_i}$ Hz was described as follows:

$$\begin{equation}\begin{array}{*{20}{c}} {{s_{{F_i}}}\left( t \right) = A\left( {127} \right){\text{square}}\left( {2\pi {F_i}t} \right),{\mkern 1mu} i = \left\{ {1,{\mkern 1mu} 2,{\mkern 1mu} \ldots } \right\},} \end{array}\end{equation} \tag{ 1 }$$

where ${\text{square}}\left( \cdot \right)$ indicates a square waveform with a period of $T = 1/f$ s,

$$\begin{equation*}{\textrm{s}}quare\left( {2\pi ft} \right) = {\textrm{}}\left\{ {\begin{array}{*{20}{l}} {1,{\mkern 1mu} {\mkern 1mu} n/f \leqslant t < n/f + 1/2f}\\ {0,n/f + 1/2f \leqslant t < n/f + 1/f} \end{array}} \right.\end{equation*}$$

$$\begin{equation*}n = \left\{ {0,\,1,\, \ldots } \right\}.\end{equation*}$$

The luminance of the targets, $A\left( c \right),\,c \in \left\{ {0,1, \ldots ,255} \right\}$, was indicated using an RGB color. The color of the target was (R, G, B) = (127, 27, 127), henceforth referred to as A(127). The display luminance values at the pupil position were A(0) = 0 cd m⁻², A(127) = 21 cd m⁻², and A(255) = 122 cd m⁻².

As shown in figure 3, the square wave,

Zoom In Zoom Out Reset image size

$$\begin{equation*}s_{f{^{^{\prime}}}}^{{\text{square}}} = A\left( {127} \right){\text{square}}\left( {2\pi f{^{\,^{\prime}}}t} \right),\end{equation*}$$

exhibited a larger amplitude of the pupil light reflex than the sinusoidal stimulus

$$\begin{equation*}s^{{\text{sin}}}_{f^{^{\prime}}}= A\left( {127} \right){\text{sin}}\left( {2\pi f^{^{\prime}}t} \right),\end{equation*}$$

when ${f^{^{\prime}}}$={0.3, 0.75, 0.9, 1.5, 2.25}. The ratio of the peak values to the effective value (which is called the crest factor) of a square wave is lower than that of a sinusoidal wave. Therefore, it can efficiently induce a pupillary light reflex even in HMD displays in which the maximum luminance is limited.

2.3.1. Experiment 1

Here, we investigated the differences in pupil response between the binocular-independent condition or the monocular-superposed condition. When a stimulation signal is generated as the sum of two sinusoidal waves, the frequency components will not change if the response of the pupil diameter is linearly related to the light intensity. However, if the non-linearity of the pupil response is high, certain component in the frequency domain other than the input frequency will appear in the pupil response. As shown in in the left side of figure 4(A), the shape of the stimulus target was a square (9° viewing angle). The stimulation patterns of the binocular-independent condition were ${F_1}:{F_3}$ and ${F_2}:{F_3}$, where ${F_i}:{F_j}$ indicated that the ${F_i}$ and ${F_j}$ stimuli were administered to the left and right eyes, respectively. The stimulation patterns of the monocular-superposed condition were ${F_{1 + 2}}:{F_{1 + 2}}$ and ${F_{2 + 3}}:{F_{2 + 3}}$. Here, the stimulation wave, ${s_{i + j}}$, for ${F_{i + j}}\,$obeyed the equation

$$\begin{align}&{s_{i + j}}\left( t \right)\nonumber\\ &= \left\{ {\begin{array}{*{20}{c}} {A\left( 0 \right),{\mkern 1mu} {\text{square}}\left( {2\pi {F_i}t} \right) + {\text{square}}\left( {2\pi {F_j}t} \right) = 0} \\ {A\left( {127} \right),{\mkern 1mu} {\text{square}}\left( {2\pi {F_i}t} \right) + {\text{square}}\left( {2\pi {F_j}t} \right) = 1} \\ {A\left( {255} \right),{\mkern 1mu} {\text{square}}\left( {2\pi {F_i}t} \right) + {\text{square}}\left( {2\pi {F_j}t} \right) = 2} \end{array}}\right.\!\!\!\!.\end{align} \tag{ 2 }$$

Zoom In Zoom Out Reset image size

Three frequencies were assigned to the stimulation pattern ${F_1}$, ${F_2}$, ${F_3} = 0.5,\,1.0\,,1.5$ Hz. Participants were asked to overtly gaze at the target stimuli. As shown in figure 2(B), after 10 s of darkness, visual stimuli were presented for 45 s. Three measurements were taken for each condition, so 4 × 3 = 12 trials.

2.3.2. Experiment 2

2.3.3. Experiment 3

2.4.1. Time-frequency analysis

2.4.2. Signal-to-beat ratio (SBR)

We called the peak of the unexpected frequencies the beat frequency and used the SBR to evaluate the effect on the observed pupil response. The beat signal is not observation noise; it is rather generated by a biological mechanism. However, it should be avoided whenever possible because the existence of beat frequency signals degraded the command classification performance of the proposed interface. The SBR was calculated by dividing the mean amplitude of each of the two stimulus frequencies by the mean of the highest beat frequency:

$$\begin{equation}\begin{array}{*{20}{c}} {{\textrm SBR} = 10{{\log }_{10}}\left( {{P_{{\text{singal}}}}/{P_{{\text{beat}}}}} \right){\mkern 1mu} {\mkern 1mu} {\text{d}}B.} \end{array}\end{equation} \tag{ 3 }$$

The SBR was virtually the same as the signal-to-noise ratio (SNR) when the power of the beat frequency was significantly higher than that of the other spectrum components.

2.4.3. Signal processing and classification

To recognize the target being gazed at, we used a simple classification with a threshold. First, we obtained the PSD of the pupil diameter and normalized it to the highest PSD of one. The frequency of the first highest peak was always considered a pattern gazed at. Second, if the frequency of the second highest peak was over the threshold, it was considered a pattern gazed at. The threshold was common between different participants, and the value was obtained with a 0.01 resolution from the measured data by using cross-validation (see 2.5).

Figure 5 shows the determination for each target when the threshold was 0.5. As indicated in the upper panels, only one frequency peak was beyond the threshold when the gazing targets were identical (${F_1}:{F_1}$, ${F_2}:{F_2}$ and ${F_3}:{F_3}$). Oppositely, as shown in the lower row, two peak frequencies appeared beyond the threshold when the gazing targets were different (${F_2}:{F_1}$, ${F_3}:{F_2}$ and ${F_1}:{F_3}$).

Zoom In Zoom Out Reset image size

The classification accuracy was estimated using k-fold cross-validation. The dataset was divided into a training set for choosing the optimal parameter (threshold) and a test set for estimating the accuracy. Three measurements were taken for each stimulus condition. Therefore, parameter k of the cross-validation was three. The accuracy rate was defined as

$$\begin{equation}\begin{array}{*{20}{c}} {{\textrm acc} = \frac{1}{3}\mathop \sum \limits_{k = 1}^3 \frac{{{P_k}}}{{{N_k}}} \times 100\,\% ,} \end{array}\end{equation} \tag{ 4 }$$

where ${P_i}$ is the number of correctly classified attempts and ${N_i}$ is the number of attempts when the fold number was k.

In addition to accuracy, the ITR is a common metric used to evaluate BCI performance. It measures the information transmitted by the system per unit time and is defined as follows:

$$\begin{align}\begin{array}{*{20}{c}} {{\textrm ITR} = \frac{{60}}{T}\left[ {{{\log }_2}N + P{{\log }_2}P + \left( {1 - P} \right){{\log }_2}\left( {\frac{{1 - P}}{{N - 1}}} \right)} \right],\,} \end{array}\end{align} \tag{ 5 }$$

where $N$ is the number of stimuli, $P$ is the target identification accuracy and $T$ is the time required to make a selection (s). We assumed that the processing time was negligibly short and the rest time as initial stimulation was not included in T; therefore, the ITR calculated in this study was higher than that in a practical application.

In experiment 1, significance levels were calculated using a paired t-test. The p-values below 0.05 were considered significant. In experiment 2, we analyzed the difference in PSDs depending on the stimulus condition on the basis of multiple comparison tests (paired-t-tests) with Bonferroni correction. The significance levels were set at p < 0.05. In experiment 3, we analyzed the difference in accuracies depending on the stimulus condition on the basis of multiple comparison tests with Bonferroni correction. Moreover, we also used multiple comparison tests (paired-t-tests) with Bonferroni correction to analyze the difference in ITRs, the stimulus condition, and the number of stimulation patterns. The significance levels were all set at p < 0.05.

Figure 6 shows the mean $\pm \,{\text{SD}}$ PSD of the pupillary responses generated by the synthetic waves or independent stimuli in seven participants. Figures 6(A) and (B) show the results for the stimulation frequencies of 0.5 and 1.5 Hz, and figures 6(C) and (D) show those for 1.0 and 1.5 Hz. Peaks appeared at the stimulation frequencies in the plots for each condition. Comparing figures 6(A) and (B), the PSD at 1.5 Hz was not significantly different whereas that at 0.5 Hz was significantly lower for the binocular-independent condition (0.5 Hz:$t\left( 6 \right) = 6.58,\,p = 0.0006;$ 1.5 Hz: $t\left( 6 \right) = 0.33,\,p = 0.75$). Comparing figures 6(C) and (D), the PSD at 1.0 Hz was not significantly different whereas that at 1.5 Hz was significantly lower for the binocular-independent condition (1.0 Hz:$t\left( 6 \right) = 0.21,\,p = 0.84;$ 1.5 Hz:$t\left( 6 \right) = 5.58,\,p = 0.002$). Peaks appeared at the beat frequencies of the stimulation frequencies. Comparing figures 6(A) and (B), the PSD at the beat frequency (1.5 − 0.5 Hz = 1.0 Hz) was significantly lower for the binocular-independent condition ($t\left( 6 \right) = 10.17,\,p = 0.00006$). Comparing figures 6(C) and (D), the PSD at a beat frequency (1.5 − 1.0 Hz = 0.5 Hz) was significantly lower for the binocular-independent condition ($t\left( 6 \right) = 5.02,\,p = 0.003$).

Zoom In Zoom Out Reset image size

As shown in figure 7(A) for 0.5 Hz: 1.5 Hz, the SBR of the binocular-independent condition was significantly higher than that of the monocular-superposed condition at 1.5 Hz ($t\left( 6 \right) = 5.92,\,p < 0.002$) and was not significantly different from that of the monocular-superposed condition at 0.5 Hz ($t\left( 6 \right) = 1.80,\,p = 0.12$). The mean of the binocular-independent condition was higher than that of the monocular-superposed condition. As shown in figure 7(B) for 1.0 Hz: 1.5 Hz, the SBR of the binocular-independent condition was significantly higher than that of the monocular-superposed condition at 1.0 Hz ($t\left( 6 \right) = 8.63,\,p < 0.0$002) and 1.5 Hz ($t\left( 6 \right) = 6.88,\,p < 0.0005$).

Zoom In Zoom Out Reset image size

These results showed that the binocular-independent condition was more suitable for discrimination using the stimulus frequency because it was less affected by the beat component. A previous study on an SSVEP-based BCI employed the beat frequency for classification [52] because it was significantly higher than the amplitude of the stimulation frequency. However, the present results for the pupil light response indicated that the amplitude of the beat frequency was not significantly higher than that of the stimulation frequency. Subsequently, we employed independent binocular stimulation to identify stimulus patterns, unless otherwise specified.

We determined if the stimulus patterns that were not gazed at but were in the field of view affected the pupil responses using time–frequency analysis when six stimulus pattern was displayed. Three of the six stimuli were for the monocular-single condition (${F_1}:{F_1}$, ${F_2}:{F_2}$, and ${F_3}:{F_3}$), and the other three were for the binocular-independent condition (${F_2}:{F_1}\text{, }{F_3}:{F_2}\text{, and }{F_1}:{F_3}$). Figure 8 shows the time–frequency analysis results when participant P4 gazed at each target for 40 s by independent bilateral stimulation. We observed spectrum peaks at the stimulus frequency corresponding with the gazed-at target. Each gaze stimulus elicited a different and unique pupil response. No effects from the beat frequency or ambient light from the non-gazed-at target were observed.

Zoom In Zoom Out Reset image size

Figure 9 shows the mean and standard error of the spectral peaks generated when the participants gazed at each visual stimulus. The PSDs tended to be high when the stimulation frequencies of both eyes were identical. In most cases, PSDs of the monocular-single condition were significantly higher than these of the binocular-independent condition, except in the case of between ${F_2}:{F_2}$ ${\text{and }}{F_2}:{F_1}$ (t(5) = 2.54, p = 0.10).

Zoom In Zoom Out Reset image size

In this system, the width of the time window represented the approximate time from when the participant began to gaze at the pattern. Therefore, the width of the time window should be shorter to increase the ITR. Figure 10(A) shows the mean classification accuracy rate across the participants when the time window width was varied from 3 to 20 s. The mean classification accuracy rate increased with an increase in the time window widths from 3 s (26.4%) to 7 s (80.1%). The highest mean accuracy rate was 92.6% at 12 s. A window wider than 12 s appeared not to improve the accuracy. Figure 10(B) shows the simulated mean ITR. On average, the theoretical maximum ITR of 12.5 bits min⁻¹ was achieved when the window width was 7 s.

Zoom In Zoom Out Reset image size

Finally, we compared the accuracies and the ITRs of each condition of experiment 3 to confirm the feasibility of the dual-frequency method (figure 11). The window width for classification was set to 6.0 s. The mean accuracy for the five stimulus patterns was 66.7% for binocular-independent, 77.8% for monocular-superposed, and 95.6% for monocular-single, with ITRs of 8.3 bits min⁻¹, 11.9 bits min⁻¹, and 20.2 bits min⁻¹. The mean accuracy for the ten stimulus patterns was 62.8% for binocular-independent, 64.4% for monocular-superposed, and 87.7% for monocular-single, with ITRs of 12.7 bits min⁻¹, 13.1 bits min⁻¹, and 25.3 bits min⁻¹. The mean accuracy for the 15 stimulus patterns was 58.9% for binocular-independent, 59.6% for monocular-superposed, and 78.5% for monocular-single, with ITRs of 14.4 bits min⁻¹, 14.5 bits min⁻¹, and 23.7 bits min⁻¹.

Zoom In Zoom Out Reset image size

The accuracy of the monocular-single condition for each stimulation pattern case was significantly higher than the binocular-independent condition (t(5) = 3.69, p= 0.042; t(5) = 4.25, p= 0.024; t(5) = 5.08, p= 0.012), and the monocular-single condition for the ten-stimulation pattern case was significantly higher than the binocular-independent condition for the ten-stimulation pattern cases (t(5) = 7.00, p= 0.003). On the other hand, there were no significant differences between the binocular-independent and the monocular-superposed conditions.

The ITR of the monocular-single condition for the 15-stimulation pattern case was significantly higher than the binocular-independent condition for the five-stimulation pattern and the monocular-superposed condition for the ten-stimulation pattern case (t(5) = 6.76, p= 0.039; t(5) = 6.93, p= 0.035), and the monocular-single condition for the ten-stimulation pattern case was significantly higher than the monocular-superposed condition for the ten-stimulation pattern case (t(5) = 6.44, p = 0.048). The highest ITR through all conditions and number of stimulations was obtained under the monocular-single condition for the ten stimulation patterns (25.3 bits min⁻¹) (confusion matrices of each participant are shown in figure S3(1)–(6)).

The results of experiment 1 indicated that the independent stimulation of both eyes suppressed the spectral leakage of the beat frequency component, whereas superposition stimulation increased it. This result is consistent with the results of Howarth et al who investigated nonlinear processes within the pupillary pathway [47]. In frequency coding, which extracts stimulus frequencies from pupil oscillations and decodes the gaze target, the gaze target would be misidentified if energy is observed at a frequency that should not be included in the stimulus signal. When the SBR < 0, it indicated that the beat frequency spectrum exceeded the stimulus frequency spectrum, and the system misinterpreted the beat frequency as the stimulus frequency. Suppressing the difference component using independent stimulation would facilitate the identification of the stimulation frequency, and there would be no need to avoid using stimuli with frequencies that match the beat frequency.

The results of experiment 2 indicated that the response of pupil reflex in the monocular-superposed condition (same stimulus on-offset pattern shown in both eyes) was stronger than the independent condition (different stimulus on-offset pattern shown in both eyes). This suggests that the effect of lateral suppression was stronger than the orienting responses in this experimental condition.

The results of experiment 3 indicated that, although we carefully selected the stimulus frequencies so that the beat frequencies were not affected by the classification result, the accuracy and ITRs of the monocular-superposed condition were not higher than those of the binocular-independent condition.

For a pupillary light reflex-based BCI with single-frequency coding, Muto et al achieved over 98% discrimination performance when the difference in the blinking frequency between the stimuli was larger than Δf= 0.12 Hz [53]. When the range of the stimulation frequency was 0.7–1.9 Hz and the difference between the frequencies was 0.12 Hz, the number of stimuli would be approximately 11 at most. Our experimental system was able to identify 15 frequencies with an identification rate of 78.5% (ITR = 23.7 bits min⁻¹). The difference in the shape (sine or square wave) of the stimulation signal may have contributed to increasing the ITR of our system. Achieving a higher ITR with the PFT method, which we call monocular-single condition, is not an objective of this paper, but the highest ITR of our system was 25.3 bits min⁻¹ (single method/10 stimuli). This was higher than the 21.26 bits min⁻¹ reported by Muto et al.

The results of experiment 3 indicated that the binocular-independent and monocular-superposed conditions could be classified fifteen patterns with five different frequencies. However, we do not believe that these approaches are the optimal solution. For example, a gradual 'zoom-in' method has the advantage of easily increasing the number of classifiable targets without decreasing the ITR, even with fewer frequencies that can be stimulated. Mathôt et al reported a zoom-in method for classifying eight targets with repeated binary segmentation; the accuracy of this method was 90% (chance level = 12.5%), and the mean ITR was 4.86 bits min⁻¹ [25]. The mean of the classification time in each trial was 28 s. It was higher than Naber et al [24], who used unique frequencies for each target (PFT method); the accuracy was 73 ± 20% (chance level = 25%), and the mean ITR was 2.32 bits min⁻¹ when they classified four patterns. However, the zoom-in method changes the content being displayed with each zoom-in, resulting in a slight delay in user recognition and, consequently, a slight decrease in ITR with each trial. If it were possible to stimulate with only five frequencies, the ITR for the binocular-independent condition would be 14.4 bits min⁻¹ (see figure 11). On the other hand, the zoom-in method requires four repetitions of binary classification for 15-pattern classification. An accuracy of 97% or better within 3.0 s is required to obtain the same ITR as the binocular-independent condition when the user recognition delay is assumed to be 1.0 s. This suggests that the zoom-in method is not a gold standard.

The zoom-in method might be extended to reduce the number of classification trials by using several unique frequencies. For example, if five patterns are classified in the first trial, and three patterns are classified in the second trial, then 15 patterns can be classified in these two classification trials. In experiment 3, in the monocular-single condition, the five-stimulus pattern was classified with 92.2% with a 4 s window width (see figures S1 and S2, in the supplementary materials). In this setting, the ITR for 15-pattern classification can be 21.4 bits min⁻¹ even if a 1.0 s delay time exists for user recognition. Although the interface design will be complex, the ITR of an extended zoom-in method could be higher than that of the binocular-independent condition. Meanwhile, the mean ITRs for the dual-frequency conditions were higher for the 15-stimulus pattern (14.4/14.5 bits min⁻¹, binocular-independent/monocular-superposed conditions) than for the ten-stimulus pattern (12.7/13.1 bits min⁻¹) or the five-stimulus pattern (8.3/11.9 bits min^-1). At least, the ITRs did not decrease as the number of simultaneously displayed patterns increased. Therefore, the extended zoom-in method may not be suitable in combination with dual-frequency conditions.

The presentation of different visual stimuli to both eyes may induce BR [54]. BR is a phenomenon in which conscious perception spontaneously alternates between left and right images. There have been recent reports that BR slightly affects the pupil [23]. However, in the present experiment, the effects of BR were negligible because the stimuli were simple shapes and not two colors, and BR is unlikely to occur when spatially identical stimuli that flash at different frequencies are presented [55].

The non-linearity of the pupillary light reflex was deeply involved in this study. The beat frequency component due to superposition stimulation can be easily explained by the hypothesis of Howarth et al [47], which states that it is caused by a non-linear transformation located before the confluence of the left and right eye stimuli. However, Howarth et al discussed the reasons for generating the beat frequency component of the pupil response in the frequency domain, as opposed to the time domain. Therefore, the effects of the independent and superposition stimulation of the pupil response have not been fully explained. They chose a stimulus frequency so high that the stimulus did not generate oscillations. Therefore, they possibly did not model well at low frequencies. Baitch and Levi observed that there was a beat frequency component in the evoked potentials in the visual cortex caused by independent stimuli [56]. Their results differed from those of the pupil's responses. This study is for engineering. Therefore, it is not possible to describe the detailed structure and response characteristics of the binocular summation from the viewpoint of neuroscience and electrophysiology. Considering that the non-linearity of the pupillary light reflex may affect BCI performance, it will be necessary to investigate the nature of the pupil oscillation in detail from a neuroscientific point of view.

As a future experimental plan, we may improve BCI performance by implementing visual feedback [57, 58]. In addition, we need to consider the idling state of the BCI system. One way to deal with this is to use thresholds [7, 59]. The BCI system can be activated when the pupil response exceeds a pre-defined threshold. Another possibility is to build a hybrid system with an eye tracker or similar system. Gaze estimation by eye tracking requires accurate calibration, and there are fears that it will select non-gazed-at (but in the field of view) targets. In addition, when the target is dense and small, BCI using the biological signals generated by blink stimuli has been observed to perform better than eye tracking [60]. Therefore, by combining eye-tracking measurements with SSVEP measurements using blinking stimuli, it is possible to identify the object that is truly being gazed at [61–64]. In the pupillometry-based system, pupil measurement and eye tracking can be combined in a simple system with only a camera.

Certain limitations existed in this study. First, we selected a combination of stimulus frequencies that generated a beat frequency component and investigated only a limited number of frequencies. Second, we showed that at least 15 stimulation patterns can be applied; however, it was not determined how many visual stimuli can be increased without degrading discrimination performance. Next, there were individual differences in the selection accuracy. These individual differences could have caused differences in attention, which could have been improved by minimal attentional training. Finally, lesions such as relative afferent pupillary defect and amblyopia could affect the results.

This study replaced a cognitive estimation component of BCI with the pupil light reflex from the SSVEP of surface EEG. The eyes may reflect various brain activities and attention [65–67]. We believe that the pupil response denotes several brain activities through the central nerve. In the future, more brain activities will be inferred from pupil response.