Brain waves-based index for workload estimation and mental effort engagement recognition

The advent of the communication systems and considering the complexity that some impose in their use, it is necessary to incorporate and equip these systems with a certain intelligence which takes into account the cognitive and mental capacities of the human operator. In this work, we address the issue of estimating the mental effort of an operator according to the cognitive tasks difficulty levels. Based on the Electroencephalogram (EEG) measurements, the proposed approach analyzes the user’s brain activity from different brain regions while performing cognitive tasks with several levels of difficulty. At a first time, we propose a variances comparison-based classifier (VCC) that makes use of the Power Spectral Density (PSD) of the EEG signal. The aim of using such a classifier is to highlight the brain regions that enter into interaction according to the cognitive task difficulty. In a second time, we present and describe a new EEG-based index for the estimation of mental efforts. The designed index is based on information recorded from two EEG channels. Results from the VCC demonstrate that powers of the Theta [4-7 Hz] (θ) and Alpha [8-12 Hz] (α) oscillations decrease while increasing the cognitive task difficulty. These decreases are mainly located in parietal and temporal brain regions. Based on the Kappa coefficients, decisions of the introduced index are compared to those obtained from an existing index. This performance assessment method revealed strong agreements. Hence the efficiency of the introduced index.


Introduction
The ambition behind communication systems is evolving the communication between human and machine in order to improve his/her everyday life. With the innovative aspect brought by the Artificial Intelligence (AI), communication systems have increasingly integrated the human's everyday life. In the context of rehabilitation and health care, these systems represent innovative tools for the assistance of people with disabilities or an extreme dependence. However, designing systems communicating with humans must incorporate dynamic and adaptive characters. In the context of assistive technologies, a crucial aspect in designing such systems is to take into account the specificities of each user and to propose solutions adapted to his/her mental and motor abilities [1]. In order to measure affective status of an individual, several approaches have been proposed in literature; the more robust are that based on biological signals [2], [3]. Amongst these methods we distinguish Heart Rate Variation (HRV), Galvanic Skin Response (GSR) and the brain electrical activity [4]. However, amongst these approaches, only analysing the electrical brain activity has the potential to precisely reflect the affective status of an individual. For about two decades, studying and analysing the human's brain activity have been closely related to the concept of Brain-Computer Interface (BCI). In its definition, a BCI represents a direct communication and control system between the human and any electronic device [5]. BCI provides a means of interacting and acting with the physical surrounding without using the standard neuromuscular channels [1], [6]. The user's intentions are mediated through brain signals in place of nerves and muscles. This type of peripheral, which is completely different from the usual Human-Machine Interfaces (HMI), can be designed to assist, improve or repair human cognition functions [7], [8].
In the context of cerebral interfaces, especially those based on EEG signals, features allowing distinguishing mental tasks and mental states vary from the amplitude of the brain signal, the Power Spectral Density (PSD) and time-frequency features. Approaches using these features take advantage of the spectral aspect of the EEG signal.  [9].
In this work, we aim at developing a passive BCI to estimate the cognitive load level and the mental effort engagement of an individual. Our experimental protocol aims at measuring EEG signals on four subjects while answering matrices products with different levels of difficulties. At a first time, the proposed approach combines the power spectral density analysis of EEG signals and a variances comparison-based classifier (VCC) in order to illustrate changes in the frequency features according to the cognitive task difficulties. In a second time, we introduce a new indicator of mental effort engagement which reacts sensitively to changes of the cognitive task difficulty levels. Our approach is based on both the and frequency components. This paper is organized as follows. In the second section we present a background of related works from literature. In the third section we describe our approach for estimating the cognitive workload and mental effort engagement. Experimental results are detailed and discussed in the fourth section. Finally conclusions are drown in the fifth section.

Background and related works
In psychology, the use of EEG measures represents the basis background of a large number of researches in the cognitive workload analysis. However, there is still a doubt on the effect of cognitive workload on electroencephalographic signals. In works of Smith and Gevins [10], an approach is presented to assess EEG recordings of subjects performing n-back tests and other computer interaction tasks. This approach is based on the extraction of spectral characteristics of and frequency bands in four seconds time-windows, and then, neural networks are applied for the discrimination of three different levels of cognitive workload. In another work related to the cognitive workload assessment, Putze [11] proposed a multimodal approach to estimate the different levels of the cognitive workload. Authors propose, in addition to EEG data, measuring the skin conductance and breathing. These measures are then classified on time-windows of 60 seconds. This approach was applied in the context of assessing the cognitive workload in a driving simulator. In the realm of using EEG signals to estimate mental states, Pal et al., have introduced an unsupervised approach, based on the Mahalanobis distance, to recognize departs in drowsiness of drivers in an environment based on Virtual Reality (VR) [12]. This approach generates, in each driving session, a statistical model of the driver's alertness based on and frequency bands from the occipital brain area. Through this work, authors demonstrated that deviating from de alert model varies according to the driving performance.
Estimating the cognitive workload from EEG requires objective methods to determine the cognitive overload of the brain mental fatigue during the work performance. In [13], authors have determined an index , based on the frontal ( ) oscillation and the parietal ( ) oscillation: The index increases with the number of tasks to perform simultaneously and time awake. The main results of this approach are focused on the information from EEG spectrum and particularly the oscillation increasing in the frontal area and the oscillation decreasing in the parietal area.
When performing certain cognitive tasks, some changes are observed in EEG signals. We talk about Event Related Synchronization (ERS) and Desynchronization (ERD). According to Antonenko et al., [14] and Fink et al., [15] ERS and ERD rates measure the oscillatory dynamics. According to the Antonenko, the ratio reflects the decrease (ERD) or the increase (ERS) of the power during a time window which is compared to a reference time window. The ratio is defined as follows: where represents the band power during a given time window. is the band power during the reference time window. According to Antonenko, a positive ratio reflects an increase in the power, i.e. ERS. According to Grabner et al., [16], performing a cognitive task of a high level of difficulty increases the ERD in the band. This implies a decrease in the power spectrum.

Protocol and experimentations
To induce different levels of cognitive workload we used as cognitive task solving matrices products with different difficulty levels. The experiments conducted in this study consist of two sessions. The cognitive task difficulty varies in an ascending way from the first session to the second one. Arising the difficulty level lies in increasing sizes of matrices.
EEG data used in our study were recorded on students of the Department of Mathematics and Computer Science at the Mohammed First University, Morocco. The population consists of four male subjects right handed and aged between 23 and 33years old. EEG data were measured in clinical conditions at the Neurology Department, Mohammed Sixth Hospital Center, Morocco. EEG measures were recorded based on the EBNeuro acquisition system with 7 electrodes at sampling rate of 128 Hz. The EEG data were filtered using the band-pass [1-30Hz].

Cognitive workload estimation approach
The cognitive workload is characterized particularly by evolutions of brain activities in the frequency range of . Therefore, we suggest focusing on the evolution of and brain waves in the   [17]. The signal is segmented into several overlapping and equal parts. Therefore the STFT is computed on each segment. The result is the arithmetic average of the segments transform. This method is applied every 10 seconds and on each EEG channel with a moving window fixed at 2s. This window duration seems appropriate in the frequency range of  for the STFT computation. The use of the BBS method at the beginning of our approach aims to reduce the effect of artifacts on the calculation of the spectral power on the band. To estimate the workload difference (changes of and band powers) between the two sessions, a comparison is made using a VCC applied to each considered band power.
Let and be two independent samples of the power spectral density of each of and (represented by ) on a given EEG channel. We denote by ̅̅̅̅̅ and ̅̅̅̅̅ the means of and respectively. Sizes of samples are respectively and . We denote by and the experimental variances of these samples respectively. Through the VCC we aim to determine whether the two samples belong to the same population. Therefore, two hypotheses are defined. The Null Hypotheses corresponds to ; while the Alternative Hypothesis corresponds to the case of . Hence the statistic of the VCC is defined as follows: The defined statistic is observed along a Fisher law with degrees of freedom. The comparison of variances can be tested based on a Confidence Interval . If we can reject with a risk of having reject it. If we accept . The theoretical values of the boundaries are extracted from the Fisher-Snedecor table according to values of and .
Using this classification method allows to determine whether the mental efforts produced during the two sessions of the experimental protocol are of the same class or not. Moreover, it allows to determine brain regions in which cognitive workload changes mainly took place. In order to better estimate and assess the mental efforts of each participant, we introduce the new index which is based on the parietal and the frontal oscillations:

Results and discussion
Results presented in Fig. 2 demonstrate the sensitivity of the index in terms of estimating the mental efforts and the cognitive workload expended to perform the cognitive tasks of the used experimental protocol. These results show, for all participants, a significant decrease of the index value during the second session, i.e. when increasing the cognitive task difficulty. Through a representative example (subject 2), presented in Fig. 3, we illustrate the index behavior while performing the first session. Indeed, during the first time window, i.e. the first 10 seconds, the index revealed a large value. During this time window, the subject does not perform any arithmetic operation; he read the session statement. Hence, no high level of vigilance is required during this time window. During the second time window, the user started resolving matrices products. This transition from reading the session statement to the performance of the first computations has strongly decreased the index. From the second time window to the fifteenth one, the subject performed matrices products with a low level of difficulty. During time windows 16 and 17 the subject wrote his responses on the answers sheet. These two time windows are characterized by large values of the index. Starting from the time window 18, the subject performed matrices products with a significant high level of difficulty. One can note that the index values during these time epochs are more decreased than those when the subject performed cognitive task with a low level of difficulty. Finally, during the last four time windows the subject reported his responses on the answers sheet. This is reflected by values significantly large. Hence the sensitivity of the index to describe changes in the cognitive workload levels.
In order to evaluate our new index, a comparison to the index, introduced by Holm [13], is made. Results from Fig. 2 present, in averages, values of the two indices, and , during the two sessions. These results show similar behaviors of the two indices except the case of subject 1 for whom the index increased during the second session. Unlike the Holm's assumption, results from Fig. 3 show a decrease in the index. This implies that the wave decreases in the parietal brain area when increasing the difficulty of the cognitive task. Our assumption is illustrated by the obtained results presented in Fig. 2. To better assess the performance of the introduced index, we used the Kappa coefficient [18] to compute the agreement rate between decisions obtained from our index and those obtained from the index. The use of the Kappa coefficient identifies, beyond chance, the quality of a classifier. Evaluation decisions are made by computing the coefficient presented in Eq. 5. represents the degree of agreement between decisions of the two indices.
represents the probability of a random agreement between the compared indices. Values of the Kappa coefficient range from -1 to 1. The higher the value, the stronger the agreement.  Result of this assessment process revealed a value which reflects a strong agreement between our index and the ratio.

Conclusion
In this paper we have presented an EEG-based index to estimate brain workload and mental effort engagement. This indicator is based on spectral density using the Welch periodogram. EEG powers were calculated in the and bands in even central, parietal and temporal brain areas. Results obtained on four subjects have shown that increasing the cognitive task's difficulty reduces the power spectral density of and bands in the specified areas. In a future extension of this work, a large data set should be considered in order to accurately validate the obtained results. Moreover, we plan to determine thresholds on and bands which can be used to label the subject's cognitive states.