Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human–robot collaboration

Objective. Brain–computer interface (BCI) technology is poised to play a prominent role in modern work environments, especially a collaborative environment where humans and machines work in close proximity, often with physical contact. In a physical human robot collaboration (pHRC), the robot performs complex motion sequences. Any unexpected robot behavior or faulty interaction might raise safety concerns. Error-related potentials, naturally generated by the brain when a human partner perceives an error, have been extensively employed in BCI as implicit human feedback to adapt robot behavior to facilitate a safe and intuitive interaction. However, the integration of BCI technology with error-related potential for robot control demands failure-free integration of highly uncertain electroencephalography (EEG) signals, particularly influenced by the physical and cognitive state of the user. As a higher workload on the user compromises their access to cognitive resources needed for error awareness, it is crucial to study how mental workload variations impact the error awareness as it might raise safety concerns in pHRC. In this study, we aim to study how cognitive workload affects the error awareness of a human user engaged in a pHRC. Approach. We designed a blasting task with an abrasive industrial robot and manipulated the mental workload with a secondary arithmetic task of varying difficulty. EEG data, perceived workload, task and physical performance were recorded from 24 participants moving the robot arm. The error condition was achieved by the unexpected stopping of the robot in 33% of trials. Main results. We observed a diminished amplitude for the prediction error negativity (PEN) and error positivity (Pe), indicating reduced error awareness with increasing mental workload. We further observed an increased frontal theta power and increasing trend in the central alpha and central beta power after the unexpected robot stopping compared to when the robot stopped correctly at the target. We also demonstrate that a popular convolution neural network model, EEGNet, could predict the amplitudes of PEN and Pe from the EEG data prior to the error. Significance. This prediction model could be instrumental in developing an online prediction model that could forewarn the system and operators of the diminished error awareness of the user, alluding to a potential safety breach in error-related potential-based BCI system for pHRC. Therefore, our work paves the way for embracing BCI technology in pHRC to optimally adapt the robot behavior for personalized user experience using real-time brain activity, enriching the quality of the interaction.


Introduction
Mishaps and accidents occur in overloaded conditions due to time pressure or hecticness and also in underloaded situations due to boredom or carelessness [1].Most work environments are risk assessed to identify and mitigate these factors.In work environments involving human-machine interactions, safety is normally ensured by maintaining spatial separation between robot and the user.However, these classical safety measures may be insufficient for modern work environments, especially for collaborative environments where humans and machines work in close proximity, often with physical contact.
In physical human robot collaboration (pHRC), brain-computer interface (BCI) technology could play a prominent role in enabling occupational safety by providing a direct channel for communicating the cognitive state of the user.With BCI, the robotic partner gets access to the user's brain activity affording a direct communication pathway rather than relying on traditional communication routes like manual or speech input, which might distract the human partner from the task at hand.Embracing passive BCI technology in pHRC affords the ability to optimally adapt the robot behavior for each individual user based on real-time brain activity, enriching the quality of the interaction [2,3].
To assess the users' brain activity, electroencephalography (EEG) can be used.EEG is a method to record the electrical activity of the human brain [4] with high temporal resolution at the same time scale as cognitive processes unfold.This method has previously been employed for BCI technology to decode the cognitive state of the user [5,6] and adapt the interaction accordingly.This method could be employed to assess the user state [7][8][9] and to detect errors in the interaction with technical systems [10,11].
In a pHRC, the robot might perform complex motion sequences with unexpected changes in velocity, assuming unpredictable positions in space [12].Unpredicted or erroneous robot behavior or a faulty interaction can be measured from the EEG as errorrelated potential naturally generated by the brain when the human partner perceives an unexpected behavior or error occurrence [13][14][15].An errorrelated potential consists of a time-locked negative peak or error-related negativity around 200 ms after the error occurrence [16,17].It is mostly followed by a positive peak around 300 ms after the error onset [14,16], and this error positivity is related to the user's error awareness [18].As error-related potentials occur at short latency, when detected online using machine learning methods [19][20][21], it can be used as implicit human feedback to optimize robot behavior [22].Based on the interaction context, the detection of error-related potential can aid the automatic detection of higher-level user intention and achieve personalized rule-based robot control [23] for a safe and efficient pHRC.
In previous research, error-related potentials have been employed with reinforcement learning to train robots to mirror the gestures of a human user [22,24].An error-related potential was also used in a closed-loop system to switch the trajectory of a robot arm between two targets [25].Further, Ehrlich and Cheng employed error-related potential in a closedloop system to adapt the robot gaze based on the intention of the user [26] and later compared the error-related potential detection performance in case of unexpected movement of a robot head or cursor [27].However, in these studies, human users were merely passive observers of the error unlike in realworld applications.Recently, Ehrlich et al [28] showcased how EEG measures capture partner anticipation during task transitions, using these signals to train a reinforcement learning algorithm for dynamic subtask assignment in robot-human collaborations.
In settings with active movement of participants, the prediction error negativity (PEN) was observed that appears during the execution of a motor task, prior to any external error indication.The PEN was first observed in a cognitive conflict study with active physical movement using Mobile Brain/Body Imaging [10,11,29].In 2019, Aldini et al observed a PEN during an unexpected robot behavior in pHRC [30].They evaluated the intuitiveness in a pHRC by comparing the modulations in the amplitude of PEN under different degrees of mechanical resistance in the transition to an unexpected robot stopping.Recently, they evaluated the performance of six classifiers in PEN detection as a step in the direction of intuitive pHRC [31].
However, the integration of BCI technology with PEN for robot control demands failure-free integration of highly uncertain EEG signals, particularly influenced by the physical and cognitive state of the user.Mentally demanding tasks might not only compromise access to cognitive resources [32] but might completely deplete resources [33] needed for successfully executing the task.Mental workload reflects the tuning between the task demand and the cognitive capacity of the user [34].Researchers have reported an increase in the frontal theta power [35][36][37][38][39] and a decrease in the parietal alpha power with increasing mental workload [40,41].
As task demands and the user's cognitive capacity continuously vary over time, the mental workload experienced by a user at any point in time could be unpredictable, and therefore, a one-size fits all adaptation approach may not accommodate the inter-subject variability.Further, there has been no systematic study on how mental workload variations impact cognitive conflict response.The detrimental effects of workload on human performance and situational awareness are well-known in the literature [42][43][44], however, its influence on error awareness might also raise safety concerns in pHRC.
We employed MoBI methods, which combines various approaches to examine human brain activity, motor behaviors, and other physiological data linked to cognitive processes that engage active behavior [45][46][47] in this work.We hypothesize that with increasing workload, there will be a diminished error awareness, reflected in reduced amplitudes of the accompanying PEN and Pe.We study the impact of workload variations on error awareness or cognitive conflict.We also explored the effects of an error occurrence on the mental workload of the user by evaluating the spectral variation after the error occurrence at the regions known to be indicators of mental workload-the frontal and the parietal region [35][36][37][38][39][40][41] and the region known to reflect sensorimotor processing-the central region [48][49][50].
In this study, the EEG features reflecting workload variations prior to the error occurrence were used to evaluate whether workload variations could predict error awareness.We employed different levels of complexity-based learning methods-ShallowConvNet, DeepConvNet [51] and EEGNet [52] for estimating the PEN and Pe values based on the EEG data prior to the error.We hypothesize that the EEG dynamics prior to reaching an obstacle or the unexpected robot stopping can predict the PEN and Pe amplitude with high resolution, thereby, paving the way for safe and reliable deployment of BCI technology in pHRC.
We designed our experimental paradigm as an interactive clock game with a task that emulates the abrasive blasting operation of the collaborative robot.Abrasive blasting is a demanding job that involves using pressurized air or water to propel abrasive materials onto a surface.This process is used to either smooth, roughen, reshape, or remove substances like rust, paint, or graffiti.In the abrasive blasting task, a human worker collaborates with a robot to control the movements of the tool.The robot assists the operator by providing guidance for the tool and sharing the physical workload.We considered abrasive blasting in our experiment because it effectively combines the strength, precision, and endurance of the robot with the skills, intuition, and decision-making capabilities of the human operator.The ANBOT [53], an intelligent industrial robot for abrasive blasting featuring a UR10 robot arm by Universal Robots, was the collaborative robotic partner in this experiment.
The mental workload variations were achieved using arithmetic tasks with low, medium and high task difficulty.The physical load was maintained constant across these conditions to investigate how mental workload variations influence physical performance.An error occurrence was achieved by an unexpected robot stopping at an invisible obstacle during the blasting operation.The NASA Task Load Index questionnaire [54] was employed to validate the mental workload variations by incorporating a subjective measure of workload.

Participants
The experimental protocol was approved by the institute ethics committee (ETH21-6346).We had 24 participants (age 23 ± 4, 13 males and 11 females, all right-handed) in this study.Participants received a reimbursement of 60 AUD for their time and involvement.All participants selected for the study had no neurological or psychological disorders that might impact the experiment's results; they could wear glasses for vision correction.Additionally, the chosen participants were non-smokers and were asked to avoid drinking tea or coffee for at least 6 h before the experiment began.

Equipment
The EEG data were collected using a wireless MOVE system (Brain Product GmbH, Germany) with 32 channels.The channels recorded were: Fp1, Fp2, F7, F3, Fz, F4, F8, FT9, FC5, FC1, FCz, FC2, FC6, FT10, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, TP9, P7, P3, Pz, P4, P8, TP10, O1 and O2 with AFz as the reference.The impedance of the electrodes was maintained below 20 kΩ, and the sampling rate was 1 kHz.The impedance of the electrodes was checked and confirmed to be below 20 kΩ approximately one hour after the start of the experiment while the participants rested after the second block of the experiment.The events from the experiment scenario were synchronized with EEG data with Lab Streaming Layer [55].
The ANBOT [53], an intelligent industrial robot for abrasive blasting featuring a UR10 robot arm by Universal Robots, was the collaborative robotic partner in this experiment.The participants were in contact with the robot arm through a handle mounted on the robot end-effector (see figure 1(A)).The interaction forces and torques are measured and controlled by the ATI mini-45 transducer, placed between the robotic arm and the end-effector.

Experimental procedures
The game-like primary task emulated a blasting task and employed a secondary task, mental calculation of varying difficulties, to manipulate the workload of the participants.The task was presented as an interactive game called the 'Clock Game' on a 65-inch-wide LED TV.During the game, the point where the nozzle of the ANBOT pointed to was represented by a blue dot on this screen, and for this experiment, the ANBOT was limited to 2-DOF.The participants were instructed to grasp the button on the back handle of the robot end-effector to move the robot arm.They also had to grasp the button on the front handle to blast the area where the nozzle moved.The area being blasted was shown on screen in blue and this kept the participants engaged in this task, and they used both hands to move the robot arm.
Initially, the participants performed a training session where they performed 30 trials of the task to familiarize themselves with the task and move the robot arm.The experiment was divided into three blocks with 90 randomized trials (figures 1(B) and (C)).Each block started with a fixation period of 10 s, where the participants looked at a fixation cross on the screen.After this, the initial starting arithmetic equation was shown on the screen.For the first trial, the participant only performed the calculation in the allocated five seconds and had not moved the robot.This initial blasting trial was discarded in the EEG, task performance and physical performance analysis.The arithmetic equation was either adding or subtracting one-digit, two-digit, or three-digit numbers.
In the low workload condition, the arithmetic equation was a one-digit number addition or subtraction (e.g. 9 − 1).For medium workload condition, the arithmetic equation was the addition or subtraction of two-digit numbers with no carryover (e.g. 13 + 24).For the high workload condition, the arithmetic equation was either the addition or subtraction of three-digit numbers.Also, the addition involved carryover in the one's and ten's places (e.g.135 + 168) and the subtraction of three-digit numbers with borrowing from hundred's and ten's place (e.g.423 − 187).
In the first trial, the participants performed the calculation, after which, a large circle with a red dot at the center and four green dots along the perimeter was shown.Each of the green dots in the perimeter had a number next to it.The participants were asked to identify the current target, which was the green dot, with the answer to the arithmetic equation that they had previously solved.This 'Identify Target' phase was presented for 2 s, within which the participants had to identify the target green dot.
Afterwards, the participants were instructed to move the blue dot to the center of the big circle or the red dot.When the participants brought the robot's nozzle represented by the blue dot to the center, the participants received instructions to 'Move to Target' and 'Calculate' along with an arithmetic equation of random difficulty.The participants then moved the robot arm while performing the calculation, and they had five seconds to complete the blasting task or the trial reset.In order to maintain participants' interest in the game, we displayed a score on the screen, which was calculated based on the sprayed area and the target identification accuracy.The arithmetic equation disappeared from the screen as soon as the robot arm reached the target position ensuring that participants performed the calculation simultaneously while moving the robot arm.
The unexpected robot stopping was achieved by placing an invisible obstacle placed three-fifths on the path to any potential target or green dot.The screen also turned black when the participants reached the obstacle to corroborate the occurrence of the error.We also ensured that the participant would not be able to move around the invisible obstacle to try and reach the target, as the invisible obstacle takes the form of an annular barrier that restricts any movement of the robot arm.The robot arm remained stuck until the blasting phase resets.The probability of occurrence of the invisible obstacle in the blasting path was limited to 33% to prevent participants from generating an internal cognitive anticipating an unexpected robot stopping.Therefore, within each block, there were 90 trials of each workload condition with 60 trials of normal blasting and 30 error trials, and the trials with unexpected robot stopping were completely randomized.
After the blast phase, the participant received feedback with messages, such as 'Yay!Correct target' , 'Oops.Wrong target' or 'Sorry.Time is up' in case of the normal trials.They received feedback such as 'Aborting trial.Correct Direction' , 'Aborting trial.Wrong Direction' or 'Sorry.Time is up' in case of obstacle trials.We provided such feedback messages to keep the participants engaged in the task.After the feedback, which is shown for 1.5 s, the participants were asked to identify the target again based on the equation's answer before they were instructed to 'Get back to center' .During each block, the participants were randomly asked to complete the NASA-TLX questionnaire to record their subjective workload ratings.The NASA-TLX questionnaire evaluated the mental demand, physical demand, temporal demand, performance, frustration and effort measures.The participants provided a score between 0 and 100 for each of these measures every time the NASA-TLX questionnaire sought their subjective workload ratings in a trial.Following the participant's identification of the target during the 'Identify Target' phase, the experiment paused to record the participant's NASA-TLX ratings.Afterward, it resumed with the 'Calculate and Blast' phase.In our analysis of the Behavior data, we excluded the trials where NASA-TLX ratings were gathered to calculate accuracy.For each workload condition within a block, we collected NASA-TLX ratings twice from every participant.
Every participant performed three blocks and each block had randomized workload trials with Normal and Obstacle conditions.Every block was approximately 18-20 min, and participants rested for 10 min before proceeding to the subsequent block.All participants performed the clock game for approximately 1.5 h.

ANBOT control system
The control system that achieved pHRC in our experiment is outlined in [30,31].We employed an admittance-based controller [56] to convert the forces applied by the operator on the handles into the desired velocities of the robot in Cartesian space.To maintain the robot's admittance without introducing a singularity avoidance strategy, we restricted the task to a specific area within the robot's workspace.This ensured that the Jacobian matrix remained invertible in the designated workspace without requiring an active singularity avoidance algorithm, which could have led to unintended opposing forces.The calculation for the desired Cartesian velocities is shown below: Here, Ẋd represents the desired velocities of the robot's end effector in Cartesian space.A stands for the diagonal matrix holding the admittance gains, f H indicates the forces exchanged with the human operator, f D refers to a force vector that smooths out the Cartesian velocities, and f R represents the resistive force introduced to manipulate cognitive conflict.The admittance gains are specifically set at 0.01 m sN −1 for the linear parts and 0.4 rad sN −1 for the angular parts.To calculate f D , we multiply the actual Cartesian velocity of the robot's end effector, Ẋr , by a diagonal matrix of gains serving as a damper D.
In our experiments, we restricted the movements of the robot end effector in certain directions and orientations using a simple P controller.This controller manages the end effector's position at specific set points within those directions.Using i to represent the directions and orientations in Cartesian space (where i = 1, 2 for the two active directions in the planar task), we can summarize the control system for the experiment as follows: In this setup, x i R represents the real Cartesian position in the ith direction, while x i D is the targeted Cartesian position.The operator solely presents f H , and their perceived admittance is A * = Ẋd fH .The resistive force, f R , acts like a compressed spring, opposing movement and influencing the admittance in our experiment.To adjust both the admittance and its rate of change, we utilize the resistive force f R , which is calculated based on Hooke's law: ( The stiffness gain, represented by K R , gets multiplied by x ∼ , which stands for the difference between the cursor's position (the point aimed at by the ANBOT's nozzle), x p and the position where the resistive force begins, x s .The position where the robot's movement is restricted is x t , corresponding to the target in Normal trials or the hidden obstacle in Obstacle trials.As the cursor moves from its starting point, x f sits just before the target or obstacle, positioning the start of the force at x s = x t − x f .Consequently, ) Once x p reaches x t , the robot's movements come to a halt.To create a sudden resistance upon reaching the obstacle or target, x f is positioned 20 pixels (equivalent to 2.3 cm) before the center of either the target or obstacle.

Subjective measures analysis
The NASA-TLX scores were randomly collected from the participants for all the six conditions: low, medium and high workload conditions, for both normal and obstacle trials in a block.The mean scores of each subscale were then calculated for each condition.

Behavioral and performance data analysis
The performance in the obstacle trials of the blasting task was evaluated by (i) the target identification accuracy and (ii) the time to reach the obstacle.Target identification accuracy for each workload level was the ratio of the number of trials where the participant correctly identified the target to the total number of trials, Accuracy = n_correct n_total (5) where n_correct was the number of trials where participants correctly identified the target and moved in the correct direction before reaching the obstacle, and n_total was the total number of obstacle trials.The time taken to move the robot arm from the center of the circle to the invisible obstacle in the blasting path to any potential targets was also considered for the different workload conditions.Further, the mean applied force and mean end-effector velocity was extracted from the force-torque sensor (ATI Mini-45) placed between the handle and blasting nozzle.

EEG processing
EEG was preprocessed using EEGLAB v2022.0toolbox [57] in MATLAB R2020b (The Mathworks, Inc., Natick, MA, USA), as shown in figure 2. The data was down-sampled to 250 Hz, before band-pass filtering between 1 and 45 Hz using eegfilt function in EEGLAB, which performs two-way least-squares finite impulse response filtering.Line noise was removed using the clean_line function of EEGLAB which estimates and removes sinusoidal noise from the data using Thompson F-statistic and multitapering.The channels with flat lines for over three seconds were eliminated using clean_flatline function in EEGLAB.Subsequently, using the clean_channels function, the noisy channels were also identified and removed.On average, 2 ± 0.4 channels were removed, which were later restored using the spherical spline method, which interpolated the data from neighboring channels.A copy of this dataset was made, and using pop_rejcont, continuous artifactual regions were cut off.Then, the clean_windows function was used to perform window cleaning.After these artifact removal steps, independent component analysis (ICA) [58] was performed using EEGLAB's AMICA algorithm [59,60].
After decomposing each participant's data by ICA, we employed ICLabel plugin [61] in EEGLAB to identify and reject all the components with less than 30% of brain activity.Subjects had 5 ± 0.76 ICs remaining after this step.Then, the ICA information (icawinv, icasphere, and icaweights) was copied back to the original dataset and component data was backprojected to the sensor level.All the blasting epochs were extracted, and three datasets corresponding to the three workload levels of blasting were created.We further extracted the obstacle trials from these three blasting datasets to form another three datasets with obstacle trials under low, medium and high workload conditions in order to study the PEN and Pe modulations across the three workload conditions.

IC clustering
The data from multiple participants were managed and processed using the EEGLAB STUDY structure [62].A Study was created with a group (of 24 participants) with six conditions corresponding to the three workload levels and the presence or absence of obstacle.The components for all participants were clustered using k-means clustering algorithm [63] based on three equally weighted (weight = 3) criteria: (1) dipole locations, (2) dipole orientation, and (3) scalp maps to produce six clusters.The frontal, central and parietal clusters were identified using Talairach coordinates [64] of the fitted dipole sources of these clusters.
For the frontal, central, and parietal clusters, the power spectral density (PSD) of each component was evaluated using the spectopo function, considering each 1 s segment using a Hamming window.For each IC of the frontal, central and parietal clusters, the mean (PSD) at delta (1-4 Hz), theta (4-8 Hz), alpha (8-12 Hz) and beta (12-30 Hz) and gamma (30-45 Hz) bands were examined for the 1 s immediately after reaching the target in normal conditions and for the 1 s immediately after reaching the obstacle in the obstacle trials.The calculated PSD at each IC was normalized using the fixation period as the baseline to compare the changes in PSD upon reaching either target or obstacle in different workload conditions.
We also created a simple ERP Study, implemented in EEGLAB, with a group (of 24 participants) with three conditions corresponding to the three workload levels.The components for all participants were clustered using k-means clustering algorithm [63] based on three equally weighted (weight = 3) criteria: (1) dipole locations, (2) dipole orientation, and (3) scalp maps.The channel-based ERP data was evaluated at all the 32 channels.PEN was measured as the minimum error negativity amplitude in the 100-350 ms window after reaching the obstacle, and Pe was measured as the maximum error positivity in the 350-450 ms window after reaching the obstacle.

Prediction of PEN and Pe amplitude from EEG data
We employed deep learning algorithms to predict PEN and Pe amplitudes from the EEG data prior to reaching the obstacle.The two seconds of preprocessed EEG data prior to reaching the obstacle was used as the input to EEGNet, ShallowConvNet and DeepConvNet [65] to predict the PEN and Pe amplitude at the Fz channel.We used 80% of the data for training and it was in the (1026 × 500 × 29) format and 20% for validation (342 × 500 × 29) and another 20% for testing (342 × 500 × 29).Using hyperparameter optimization, we identified and employed the optimal batch size = 128, dropout rate = 0.5 and Adam optimizer.

Statistical analysis
SPSS (IBM SPSS 26.0; Chicago, IL, USA) statistical tool was used to conduct all the statistical analyses in this study.Using the Shapiro-Wilk test, it was ascertained that the data was normally distributed.The variations of the NASA-TLX scores with the workload levels and the presence and absence of obstacle was evaluated using two-way repeated measures analysis of variance (ANOVA) test with workload levels and trial type (normal or obstacle trial) as within-subjects factors.The variations of EEG parameters, task and physical performance at three workload levels of the task were investigated by conducting a one-way repeated-measures ANOVA with the workload level as the within-subjects factor.We used Mauchly's test to test for sphericity, and if sphericity was dissatisfied (p < .05),Greenhouse-Geisser correction was performed.Posthoc comparisons were conducted with Bonferroni corrections (α = 0.05) if the main effect of ANOVA was significant.

Subjective workload measures
The subjective workload measures of the NASA-TLX increased with increasing difficulty levels in the arithmetic task in the clock game task with unexpected robot stopping, as shown in figure 3. The results of the two-way repeated measures ANOVA revealed that there was significant main effect of workload on participants' mental demand scores (F(2, 48) = 1499.764,p < .01,η p 2 = .980).Also, the ANOVA revealed that mental demand scores during trials with unexpected robot stopping were significant (F(1, 24) = 163.748,p < .01,η p 2 = .845).Further, there was a significant interaction between workload and unexpected robot stopping (F(2, 48) = 3.191, p = .048,η p 2 = .096)such that participants reported increasing mental demand scores with the increasing workload during conditions with unexpected robot stopping than the normal conditions.There was a significant main effect of workload on participants' physical demand scores (F(2, 48) = 403.112,p < .01,η p 2 = .931).Also, the ANOVA revealed that physical demand scores during trials with unexpected robot stopping were significantly higher compared to normal trials (F(1, 24) = 408.133,p < .01,η p 2 = .932).Further, there was a significant interaction between workload and unexpected robot stopping (F(2, 48) = 26.118,p < .01,η p 2 = .465)such that participants reported greater physical demand scores with the increasing workload during conditions with unexpected robot stopping than the normal conditions.
The results of the two-way repeated measures ANOVA revealed that there was a significant main effect of workload on participants' temporal demand scores (F(2, 48) = 1177.726,p < .01,η p 2 = .975).Also, the ANOVA revealed that temporal demand scores during trials with unexpected robot stopping were significant (F(1, 24) = 12.462, p = .001,η p 2 = .293).Further, there was a significant interaction between workload and unexpected robot stopping (F(2, 48) = 15.432,p < .01,η p 2 = .340)such that participants reported increasing temporal demand scores with the increasing workload during conditions with unexpected robot stopping than the normal conditions.There was a significant main effect of workload on participants' performance scores (F(2, 48) = 1016.185,p < .01,η p 2 = .971).Also, the ANOVA revealed that performance scores during trials with unexpected robot stopping were significant (F(1, 24) = 925.325,p < .01,η p 2 = .969).Further, there was a significant interaction between workload and unexpected robot stopping (F(2, 48) = 100.151,p < .01,η p 2 = .769)such that participants reported higher performance scores with the increasing workload during conditions with unexpected robot stopping than the normal conditions.
There was a significant main effect of workload on participants' effort scores (F(2, 48) = 1091.580,p < .01,η p 2 = .973).Also, the ANOVA revealed that effort scores during trials with unexpected robot stopping were significant (F(1, 24) = 202.924,p = .001,η p 2 = .871).However, there was no significant interaction between workload and unexpected robot stopping as the participants reported similar increasing effort scores with the increasing workload during unexpected robot stopping and normal conditions.There was a significant main effect of workload on participants' frustration scores (F(2, 48) = 1368.842,p < .01,η p 2 = .979).Also, the ANOVA revealed that frustration scores during trials with unexpected robot stopping were significant (F(1, 24) = 334.303,p < .01,η p 2 = .918).Also, there was no significant interaction between workload and unexpected robot stopping as the participants reported similarly increasing frustration scores with the increasing workload during unexpected robot stopping and normal conditions.

Behavioral results
The target identification accuracy deteriorated significantly with increasing workload during the unexpected robot stopping conditions, [F(2, 48) = 228.770,p < .001,η p 2 = .916],as shown in figure 4(A).Target identification accuracy significantly decreased for low and medium (p = .004),low and high (p < .01)and medium and high (p < .01)conditions.The time to reach the obstacle increased with the increasing workload in the unexpected robot stopping conditions, [F(2, 48) = 113.896,p < .01,η p 2 = .792],as shown in figure 4(B).The increase was significant for low and medium (p < .01),low and high (p < .01)and medium and high conditions (p < .01).

Physical performance results
The applied human force decreased significantly with the increasing workload during the unexpected robot stopping conditions, [F( 2

Independent source clusters
The frontal, central and parietal clusters were identified based on the dipole locations and the cluster centroid [50].The Talairach coordinate of the frontal cluster centroid was at (4,38,39) and included 19 components from the 19 participants.The Talairach coordinate of the central cluster centroid was at (3, −5, 51) and had 18 components from 18 participants.The Talairach coordinate of the parietal cluster centroid was at (11, −49, 45), including 24 ICs from 24 participants.

Frontal cluster
The frontal cluster's ICs' PSD for the different workload conditions during blasting is shown in figure 5(A).There was significant main effect of workload on frontal theta PSD (F(2, 38) = 37.894, p < .01,η p 2 = .643)during unexpected robot stopping condition.Frontal theta PSD increased significantly in the low-medium (p < .01),medium-high (p = .003)and low-high (p < .01)conditions shortly after unexpected robot stopping.Further, the frontal theta power was significantly higher after reaching the obstacle as compared to reaching the target for the low (p < .01),medium (p = .029)and high (p < .01)workload conditions.However, there were no significant differences in the frontal cluster's delta, alpha, beta or gamma PSD.

Central cluster
The central cluster's ICs' spectral power for different workload conditions soon after reaching the obstacle in the unexpected robot stopping condition and after reaching the target in the normal conditions is shown in figure 5(C).There was a significant main effect of workload on central alpha PSD (F(2, 36) = 15.274,p < .01,η p 2 = .540)during unexpected robot stopping condition.Central alpha PSD increased significantly in the medium-high (p = .038)and lowhigh (p < .01)conditions shortly after the unexpected robot stopping.Further, the central alpha power varied significantly only for the low workload condition (p < .01)upon reaching the obstacle as compared to reaching the target.The central beta PSD varied significantly after reaching the obstacle across the three workload levels (F(2, 36) = 18.453, p < .01,η p 2 = .552).Central beta PSD increased significantly in the medium-high (p = .002)and low-high (p = .001)conditions shortly after the unexpected robot stopping.Further, the central beta power increased significantly for the low (p < .01)and medium (p < .01)workload conditions upon reaching the obstacle as compared to reaching the target.No significant changes were noted in the other frequency bands across the workload conditions.

Parietal cluster
The parietal cluster's ICs' spectral power for the three workload levels upon reaching the target in the unexpected robot stopping condition or reaching the obstacle in the normal condition is illustrated in figure 5(B).There was a significant main effect of workload on parietal alpha PSD (F(2, 48) = 59.458, p < .01,η p 2 = .730)during unexpected robot stopping condition.Parietal alpha PSD decreased significantly in the low-medium (p < .01),mediumhigh (p = .003)and low-high (p < .01)conditions shortly after the unexpected robot stopping.Further, the parietal alpha power varied significantly only for the low workload condition (p < .01)upon reaching the obstacle as compared to reaching the target.No significant changes were noted in delta, theta, beta or gamma PSD at the parietal cluster.

Prediction of PEN and Pe amplitude from EEG data
We employed deep learning algorithms to predict PEN and Pe amplitude from the EEG data prior to reaching the obstacle.Figure 7 shows the mean   As shown in figures 7(C) and (D), EEGNet can predict the amplitudes of PEN and Pe with great resolution based on the EEG data prior to the erroneous robot behavior at a latency of 0.0008 s.Therefore, relying on a two-second window prior to the error, which holds crucial indicators of the workload experienced by the user, we can reliably predict the PEN and Pe amplitude and thereby, the error awareness of the user.

Discussion
In this study, we designed an abrasive blasting task with the unexpected robot stopping under three workload levels.The mental workload in the blasting task was manipulated by a secondary mental arithmetic task of varying difficulty.Performance in the blasting task, assessed by the target identification accuracy and time needed to reach the obstacle, degraded notably with increasing workload.Therefore, the workload manipulation during blasting successfully evoked substantial performance variations.The physical performance assessed by the applied human force and the resultant end-effector velocity decreased significantly with increasing workload levels prior to reaching the obstacle.Hence, the increasing mental workload resulted in significant physical performance deterioration.
In order to investigate the neural correlates of error processing under three level of mental workload variations, the independent components were extricated from the scalp EEG data, and significant correlations between the task load variations and the spectral powers at frontal, central and parietal clusters were observed.During blasting, the participants performed a secondary mental arithmetic task, manipulating one-digit, two-digit and three-digit numbers, adding to their mental load.Our results revealed a considerable increase in the frontal theta power with the increasing mental workload immediately after reaching an obstacle or unexpected stopping as compared to reaching the target or expected stopping.These results suggest the increase in memory load [38] and executive control involving error detection [66][67][68] with the increasing workload levels after reaching the obstacle.This increase in the memory load might also be due to unexpected removal of the arithmetic equation from the screen when the participants reach the obstacle, requiring the participants to rely on their working memory to continue with the calculations.This was further supported by an increase in the mental demands score of the NASA-TLX in the obstacle condition as compared to the normal blasting condition.However, the frontal region showed no significant change in any other frequency band immediately after reaching the obstacle as the error processing is primarily reflected only in the theta frequency [69].
Our results further demonstrate an increased parietal alpha power with the increasing workload soon after reaching the obstacle as compared to reaching the target.The decrease in the parietal alpha power with increasing workload might be related to the desynchronization of alpha power with increased item maintenance in the working memory [40,[70][71][72] and memory load [41].Our claims were also validated by the increased mental and temporal demand score with the increasing workload in the obstacle condition as compared to the normal condition.Further, the absence of any significant changes in other frequency bands at the parietal region could signify that error processing adds to the workload experienced by the operator.
In our experiment, once the participants reach either the target or the obstacle, the robot arm stops and cannot be moved until the end of the blasting period of five seconds.We saw an increase in the alpha and beta power at the central region with increasing workload immediately after reaching the obstacle as opposed to the similar alpha and beta power across workload conditions after reaching the target in the normal condition.The alpha and beta power at the central region is indicative of sensorimotor processing [48][49][50], and desynchronization of alpha and beta power at the central region during dynamic movements of the upper limbs is well known in the literature [73][74][75].The participants might have attempted to move the robot arm to the target after the unexpected robot stopping resulting in a decrease in the alpha and beta power suppressions at the central region with increasing workload, which is further substantiated by the lack of any changes at any other frequency bands after the error.This might explain why the NASA-TLX scores show an increased perceived physical demand with an increasing mental workload during the obstacle condition despite maintaining a uniform physical load across trials.
This work investigated the neural correlates of error processing under three workload levels.We detected PEN and Pe after the unexpected robot stopping despite the load on the cognitive resources induced by performing arithmetic calculations.PEN and Pe amplitudes significantly decreased with increasing workload, indicating a shortage of cognitive resources for error processing in increased workload conditions despite similar error type and severity.Therefore, the results indicate that increased workload depreciates the error awareness of the human operator.However, our experiment does not provide any behavioral or subjective measure of error awareness to further substantiate that the error awareness was compromised at high workload conditions.Nevertheless, some prior research groups have observed a reduction in P300 amplitude at high workload or fatigued conditions [6,76].
As PEN and Pe have low amplitudes of few microvolts, they are more prone to distortion from artifacts introduced in the measured EEG signals during active physical interaction with the robot [77].In higher workload conditions, our results demonstrate that PEN and Pe amplitudes were further diminished, raising potential safety concerns when error-related potential is relied upon for ensuring a safe and efficient pHRC environment.We evaluated whether the EEG signal prior to reaching the obstacle or unexpected robot stopping could predict the PEN and Pe amplitudes.EEGNet yielded good performance for PEN and Pe prediction using the EEG data prior to the error as the input.The model could be solely based on the tonic theta power that might be driving the relationship between before and after the error.However, our results demonstrate that during high workload conditions, theta power increased substantially after the error while the PEN and Pe amplitudes were relatively diminished as compared to the low workload conditions.These findings add to the credibility of the conclusion that the EEG prior to the error could predict the modulations in the PEN and Pe amplitudes.Therefore, this prediction model could be employed to strategically forewarn that the human user's error processing capacity has been compromised, possibly allowing a safety breach that might escape the attention of the human user.
The estimation of the PEN and Pe amplitudes or error awareness of the human operator serves to grant an opportunity to adapt the environment dynamics or robot behavior to lower the operator's workload and improve their error awareness.Further, the low latency of the offline model in predicting error awareness opens possibilities of developing a real-time system for predicting error awareness.We are in the process of developing a real-time system with minimal data processing for predicting error awareness of the operator engaged in the pHRC.Therefore, going beyond the conventional safety measures, deploying BCI with the continual prediction of the operator's error processing ability throughout the collaboration can protect the operator from harm by employing an appropriate safety scheme.A robust safety scheme could be multi-level with simple control strategies at the first level and more severe safety measures such as safely transitioning the robot to a safe configuration and stopping any further movement.Such a concrete and efficient safety scheme rushes forth an era of effective and reliable BCI deployment in the new frontiers of pHRC.

Conclusion
This work presented a systematic investigation of the impact of workload variations on the perception and processing of cognitive conflict in a physical human-robot collaboration.Twenty-four participants' EEG signals, subjective workload rating, task and physical performance measures were examined while they moved the robot arm and performed arithmetic calculations of varying difficulty.Task performance, physical performance and subjective measures revealed the effectiveness of workload manipulation in the task.Task and physical performance degraded while the perceived workload increased with increasing workload levels.Furthermore, the perceived workload was greater in error trials in which the robot arm stopped unpredicted as compared to normal trials.We observed a decrease in the amplitude of the error-related potential with increasing workload levels.Deep learning models were able to predict the PEN and Pe amplitude from the EEG data prior to the error, allowing an opportunity to adapt the system dynamics to lower the workload of the human user.Additionally, we observed significant increases in the frontal theta, central alpha and beta power soon after the error as compared to the normal condition.This work demonstrates how EEG measures can be employed for mental workload assessment and PEN and Pe amplitude prediction, and the results are encouraging considering that our study emulated unrestricted, real-world physical humanrobot collaboration.This work exposes the significance of measuring and maintaining the human user's workload at an optimal level to ensure the safe and reliable use of BCI technology for intuitive physical human-robot collaboration.

Figure 1 .
Figure 1.Experiment design with (A) a schematic illustration of a person operating ANBOT, (B) shows the participant performing an experimental block and (C) shows a detailed representation of the block design.

Figure 3 .
Figure 3. Shows the NASA-TLX scores during the normal and obstacle conditions for the three workload levels ( * * denotes p < .01).

Figure 4 .
Figure 4. ANBOT collected data: (A) shows the target identification accuracy, (B) shows the time to reach the obstacle, (C) shows the normalized applied force and (D) shows the normalized end-effector velocity for the three workload levels with unexpected robot stopping condition due to obstacle in the blasting path.( * * denotes p < .01).

Figure 5 .
Figure 5. (A) shows the frontal cluster-(A1) scalp map, (A2) dipole locations of the components, the Talairach coordinate of the cluster centroid was at (4, 38, 39) and (A3) normalized theta PSD for the one second immediately after reaching the target or the obstacle at the ICs selected in the frontal cluster during blasting.(B) Shows the parietal cluster-(B1) scalp map, (B2) dipole locations of the components, the Talairach coordinate of the cluster centroid was at (11, −49, 45) and (B3) normalized alpha PSD for the one second after reaching the target or the obstacle at the ICs selected in the parietal cluster.(C) Shows the central cluster-(C1) scalp map, (C2) dipole locations of the components, the Talairach coordinate of the cluster centroid was at (3, −5, 51), (C3) normalized alpha PSD, and (C4) normalized beta PSD for the one second immediately after reaching the target or obstacle at the ICs selected in the central cluster ( * * denotes p < .01).

Figure 6 .
Figure 6.(A) Error-related potential at the Fz channel, (B) PEN and (C) Pe upon unexpected robot stopping at the obstacle for the three workload levels ( * * p < .01).

Figure 7 .
Figure 7. (A) PEN prediction performance with mean squared error (mse) and mean average error (mae), (B) Pe prediction performance with mean squared error (mse) and mean average error (mae), (C) PEN prediction and (D) Pe prediction using EEGNet.