Drowsiness Detection using Galvanic Skin Response and Electro-occulograph

Galvanic Skin Response (GSR) is widely used in psychological applications, mostly stress detection. Hence, people always put a limitation on GSR for stress detection only. Therefore, the challenge in this study is to expand the usage of GSR in drowsiness detection. Workers, students and drivers face a sleep deprivation problem due to never-ending work. Hence, this drowsiness detection is needed to detect the drowsiness to prevent unforeseen accidents from occurring. However, existing GSR application on sleep deprivation detection needs to improve with data reliability since the recording always took place on the wrist, and an external source like hand movements may influence the reading. Therefore, another drowsiness detection method is needed for reliable data or tasks. Hence, this study aims to detect GSR and EOG from behind the ear. The earpiece has been designed to make data recording of both GSR and EOG easier. By doing so, this study able to detect the skin conductance response (SCR) and skin resistance level (SCL) of GSR also eye activity which reflect the drowsiness seen from behind the ear of the user. The study found that the SCR and SCL levels increase with increasing sleepiness or drowsiness. Moreover, EOG shows a sudden spike in the signal when the user is in a drowsy state.


Introduction
Electrodermal Activity (EDA) is a measuring method of the electrical properties of the skin.It is used to detect or define the changes across the skin surface by applying electrical potential.One of the most widely used measurements is skin conductance (SC).Two points of skin contact are involved in the measurement, and the resulting current between them will give the necessary reading.The point commonly used is the medial or distal phalanxes at the user's non-dominant hand.However, for this research, the method is unsuitable due to the excessive movement of the hand while driving.Hence, the detection method of the EDA will be taken behind the ear of the user.
Electrooculography is a method for measuring the cornea-retinal standing potential between the front and the back of the human eye.It is used to record eye movements during electronystagmography testing.It is based on the cornea-retinal potential, with a dipole of the long axis of the eye.An electrical signal is produced by the eye's movement relative to the surface electrodes placed around the eyes.The electrical signal corresponds to the eye position.However, the electrooculography electrodes will be located at the mastoid bone in this study.
To detect EDA and EOG behind the ear of the user, a suitable device must be designed to aid signal detection.In this study, an earpiece is designed to help the signal detection of EDA and EOG behind the ear.The earpiece will come in pairs for both ears.One sensor of EDA and EOG will be implemented at each pair of earpieces.The EOG sensor will be located at the mastoid bone of the user.Since an earpiece detects the EDA and EOG signal, a suitable method is necessary to maintain good contact of electrodes with the user's skin.The design and material chosen for the earpiece will be explained further later.
Workers' and students' most common problem is sleep deprivation due to never-ending work and assignments.Hence, drowsiness while working or, even worse, while driving will be a significant problem since they may face an unfortunate incident.Moreover, research by the Malaysia Institute of Road Safety (MIROS) indicates that 54% of applicant drivers were involved in a road accident after drifting off to sleep, and 61% of drivers reported that they almost got into an accident due to drowsiness.Additionally, existing drowsiness detection encounters a problem with data reliability due to external source influences.Hence, this study proposed a method to detect EDA and EOG behind the ear of the user using the earpiece.Some notable problem is previous GSR detection on drowsiness use of a wearable wrist device.Due to that drowsiness detection, the driver faces a problem from external sources.The system was affected by the excessive movement of the hand and air-conditioning.Furthermore, Electrooculography alone cannot be used to determine the drowsiness level since eye activity during driving is influenced by many external sources.

Galvanic Skin Response Measurement
A GSR measurement works by detecting the changes in ionic (electrical) activity resulting from changes in sweat gland activity.As for that, the electrode must be sensitive to the changes and able to transmit the information to the recording devices.The most used electrodes in the modern GSR are Ag/AgCl (silver-chloride), which contacts the skin.They are chosen because it is robust, cheap, accurate to transmission and, most importantly, safe for human contact.
The electrical resistance of the GSR is recorded between the two electrodes that place on the testing point, which is most commonly on the hand with the feeble current running across them.The autonomic tone changes resulting in the GSR alteration.For example, a slight increase in sweating was observed after the sympathetic simulation.It is enough to lower the SC due to the water and electrolytes in the sweat.The water and electrolytes in the sweat will increase the skin's conductivity.Hence, a rise in sympathetic tone can be indicated by the fall in the Galvanic Skin Response [1].The available methods of GSR measurement are exosomatic and endosomatic measurements.The exosomatic size can either be done using alternating current (AC) or direct current (DC) through a circuit that contains a galvanometer and electrical battery and is subject to measure changes in the GSR.However, the most common method for exosomatic measurement is using DC currents, as it is more straightforward than AC currents.As for the endosomatic measurement, it only uses a galvanometer and subject to measure the voltage changes of the skin.Among those two methods, the common use method is the exosomatic DC measurement [1].
Using those two methods, two processes of GSR can be measured: the tonic and phasis processes.Phasis processes are called electrodermal responses (EDR), which are more event-related and have a shorter time course.Primarily, it results from eliciting stimuli; however, it also can be non-specific with unidentified sources and is reported in amplitude and frequency.As for tonic responses, it is described as electrodermal levels (EDL), which are slower to change, have a longer time course and usually tell in amplitude [2].
Back to the exosomatic DC, which is the most commonly used in GSR measurement, it has two types of terms and measurement: Skin Resistance (SR) and Skin Conductance (SC).The relation between those two measurements is that the increase in SC will decrease SR and vice versa.Those two measurements, SC and SR, are both phasis and tonic processes.Therefore, Skin Conductance Level (SCL) and Skin Conductance Response (SCR), as well as Skin Resistance Level (SRL) and Skin Resistance Response (SRR), are all measurements of GSR.However, endosomatic only consists of one measurement, which is Skin Potential (SP), and it can be described as a phasic process which is Skin Potential Response (SPR) or tonic process, Skin Potential Level (SPL).All due to that, GSR continues to become one of the frequently used methods in psychophysiology, and it is respected as the gold standard [1].

Electrooculograph Measurement
Electrooculography is a method of measuring the standing potential between the front and the back of the eye.There is a presence of steady electric potential across the positively charged and negatively charged cornea.The possibility has a magnitude that ranges between 0.4-1.0 mV [3].The occurrence of higher metabolic activity in the retina was the cause of the generated potential in the cornea [4].
Three electrodes need to be used to measure that potential to determine the subject's EOG signal.The electrode placement is at the outer canthi and above, below the right eye.The electrode placement on the outer canthi detects the horizontal eye movement, while the placement above and below detects the vertical eye movement or eyelid movement, which is blinking.Lastly, the electrode is a clip on the left earlobe, which acts as the ground point and ground [5].
The principle behind the detection is that the eye acts as a dipole where the anterior pole and posterior pole are positive and negative, respectively.The EOG signal measurement can be measured in two channels: the horizontal and vertical channels.It can be done by placing the electrodes around the eyes vertically and horizontally.The horizontal channel records the horizontal eye movement, while the vertical channel records the vertical eye movement and the eyelid movement [5].However, several studies [6][7] have shown that EOG signals also can be detected behind the ear.

Drowsiness Detection
The standard method for drowsiness detection is Polysomnography (PSG) [8] and camera-based solutions [9].In particular, PSG with the Maintenance of Wakefulness Test (MWT) is considered the medical gold standard of microsleep detection.The method is based on the human head's electrical signals, such as eyeball movements, brain waves, and behaviours of closing eyelids and eye blinks.However, this method required a performance setup that was complicated, required trained technicians and needed to undergo a controlled clinical environment.
Another standard method of drowsiness detection is by using a camera.This method is commonly used due to its affordability and is the most common method to detect microsleep among drivers [9].However, this method's flaw is that it can only detect the outer reflection of sleepiness while ignoring other physiological signatures of sleeping, such as muscle and brain activities [10].Thus, this method is unreliable in microsleep while the subject's eyes remain open, which often occurs [11].Moreover, camera usage also raises significant privacy concerns while being limited by environmental light conditions.Indeed, these concerns can easily be erased by using the wearable camera; however, the usage of the wearable camera is not practical for everyday use.It is due to the mixed criticisms of Google Glass regarding its privacy and form factor [12].Those reasons have shown that wearable camera is not a device that the public can easily accept in their daily life.

BTE Earpiece Design and Working Principle
The design for the behind-the-ear (BTE) earpiece must be comfortable, cost-effective, reliable, and able to collect signals behind the ear continuously.Moreover, the earpiece must also be safe for the skin and will not irritate.After several studies of existing research [13][14], the chosen material is the silicon material of Dragon Skin 10.Those materials were selected due to their nature which can easily fit at the curve created by the mastoid bone behind the ear of the user.Moreover, it also fulfilled the requirement that the material is safe for the skin and will not cause irritation.Memory wire also was included in the BTE earpiece to ensure good contact between skin and electrode.It is because the memory wire creates a grip on the user's ear and will maintain good contact between the skin and electrode.Additionally, using the memory wire, the earpiece can be customisable as the wire is flexible and will be used for different ear sizes and shapes, as shown in Figure 1.

Figure 1 Design of BTE Earpiece
The design of the BTE hardware must be a susceptible circuit that can detect the polarisation signal created by the eyeball activities (EOG) and electrodermal activities (EDA).However, the most concerning issues in a wearable device are motion artifacts and environmental noise.Hence, a Threefold Cascaded Amplifier (3CA) was chosen to address those two issues.3CA consists of three stages: Stage 1 -Unity Gain Amplifying (Buffering), Stage 2 -Feed Forward Differential PreAmplifying (F2DP) and Stage 3 -Adaptive Amplifying.Stages 1 and 2 were implemented on the earpiece, while the sensing circuit and firmware comprised stage 3.
The design of the overall hardware of BTE shown in Figure 2 consists of a sensing circuit and a BTE earpiece.

Figure 2 Overall Hardware Design
The BTE algorithm is implemented on the host device, which is the laptop.The data collected from the BTE earpiece will be separated into different streams upon receiving it for further processing.There are three main data streams: contact impedance signals, EOG signals and EDA signals.The DC, electricity, and other noises were removed during pre-processing using DC removal, median, notch and outlier filters.The results of the clean signal of EDA and EOG obtained will then be used for microsleep classification.The data extracted from the signal is later used with machine learning algorithms.

Execution Phase
This study aims to produce a new drowsiness detection method by integrating Galvanic Skin Response and Electrooculography.MSP-430 microcontroller will be used in this system with the whole system consisting of earpiece, hardware, firmware and algorithm.
In the execution phase, low power and precision AD8244 input buffer was used to implement Stage 1 of 3CA.AD8244 was chosen due to its characteristics of high input resistance (i.e.20TΩ), unity gain and very low input capacitance (i.e.12pF), which ensure that the fluctuation of the electrode can be minimised as a state before.To implement Stage 2, which is F2DP, the dual-channel instrumentation and precision amplifier AD8222 was chosen.To utilise the full range of the ADC (i.e., 2.5V to 2.5V), the preamplifying gain was set at 100.Later, the signal was digitised with the help of 24-bits ADC chip ADS1299 and ultra-low noise integrated amplifiers (1μV peak-to-peak internal noise).
The next step of OOA was executed by oversampling the signal ten times the frequency at its maximum interest (i.e.100hz) while averaging a ratio of 5. Finally, it will yield a signal with a final sample rate of 200Hz, which will be transmitted over Bluetooth.To ensure good skin contact with the electrode, the contact impedance was measured at 250Hz.The microprocessor, MSP432, primarily uses to drive the analogue front end of the sensing circuit, adjust amplifier gain, and stream the data to the laptop with the help of Bluetooth.

Results
This section will cover the data and results obtained from the design system of BTE.However, the signal from the user was obtained from another research and will be considered the expected outcome once tested on the users.The actual study on the user could not be executed due to the research being done during the pandemic.

Analysis of EDA Signal
The raw data will undergo a deconvolution operation to decompose into SCR and SCL [15].The target is to get the SCR to d in the system developed.By considering y[n] as the raw data, the convolution using the means of discrete convolution is done to get the filtered signal,  ̂[] as below: After that, a deconvolution process decomposes the data into SCR and SCL.It is used to reverse the combination of the signals.

𝑦 ̂[𝑛] = (𝑟 × 𝑙)[𝑛] = ∑ 𝑟[𝑛 − 𝑖]𝑙[𝑖] 𝑁 𝑖=0
(2) Where,  ̂[] is filtered EDA signal, [] are SCR components and [] is an SCL component.Then, the [] components are computed using the cubic spline fit to estimate it.The [] value or SCR component can be calculated using the value obtained.The [] value is preferable to be calculated using the frequency domain to reduce the complexity involved, thus by using the Fourier Fast Transform (FFT), the equation is as follows: The preliminary results [15] based on the calculations above are shown in Figure 3, Figure 4, and Figure 5.The objective is to obtain the SCR component, as shown in Figure 6.   Figure 6 The SCR component obtained by using the deconvolution process under frequency domain.
The data of the SCR component will be used as the indicator for drowsiness detection.From Figure 6, we can analyse that there is a sudden spike in the SCR readings at a sample rate of 0 to 100 and 500 to 700.Thus, by using drowsiness detection, we can assume that the user is in a state of drowsiness during those times.Therefore, for the reliability of the data, the result is compared with EOG signals to ensure the drowsiness detection reliability.Moreover, the data also will be used to verify the integrated system to make sure the integrated system is working well.

Analysis of Electrooculography Signal
The figures below, which are Figure 7, Figure 8, Figure 9, and Figure 10, are the expected outcome of EOG when placing electrodes around the eyes [3].By comparing the EOG signal reading that has been taken at the eyes [3] and ears [16], we can see that ear also can be a site for EOG signal readings.From Figure 10 and Figure 11, the signal readings at the eyes and the ear are proximately similar to each other.We can see that at around 1.30 seconds, there is a spike in signal readings due to the double blink at both sites of signal readings.Moreover, the time at which the signal detected that the user gazed left and right was also proximately similar for both sites.Hence, the EOG signal obtained behind the ear is considered reliable for drowsiness detection, which aligns with the theory explained previously.

Conclusion
The study revolves around these three terms, which are electrodermal activity, Electrooculography, and drowsiness.Ample knowledge of these terms and technical skills are needed in the development of this study.The previous research used in the study helped me understand the concept and analysis involved in the study.
Even though the study is still not practically used by the user, the study is deemed successful in the theoretical phase.However, some limitations can be further improved in the future.The experimental user phase is hard to implement due to the user's environmental research, in which the user must drive while the measurement is taken for better results.The experimental phase while driving can be changed to another practical environment that can produce drowsiness, like reading, which we can conclude that the equipment can detect drowsiness in every situation, like during driving.

Figure 3
Figure 3The raw EDA data obtained by the EDA

Figure 4
Figure 4The filtered EDA data after the convolution process.

Figure 5
Figure 5The estimation of EDA filtered data to obtain SCL component by using the cubic spline fit.