A causal perspective on brainwave modeling for brain–computer interfaces

Our real world constantly evolves over time and we are usually forced to adapt, either actively (by choice) or passively, to these changes. Modeling reality has been an important goal for humankind and the aspiration behind the invention of many modern scientific tools. Physical phenomena and physical systems were among the first mechanisms that scientists managed to successfully model using mathematics.

A commonly-used mathematical tool is differential equations, for example in motion equations, providing a comprehensive description of physical/mechanical systems while capturing their underlying casual structures. Reliable predictions under different changes to their variables is a feature that all physical models share. Despite their provable robustness to changes or shifts that can take the model outside of its original setting, they usually require a human expert to understand the system that needs to be modelled (its structure, its dynamic processes, the causal relationships between its different parts, etc) and come up with these analytical equations.

For certain systems or problems, though, it is often impossible to reach analytical equations that accurately describe them, no matter the available expert knowledge at hand. In these cases, statistical models (widely used in the field of ML) are proved to be useful. This type of models heavily relies on observational data and can uncover associations between the input and output data. In these models, it is possible to infer some accurate predictions based only on observational input data as long as there are no changes in the system or the problem that is attempted to be modeled.

Models learnt using ML algorithms belong to the category of statistical models, since these algorithms try to make accurate predictions from a given amount of observations. Nowadays, the great success of ML (or Deep Learning) in various fields is the result of four main factors, as described in [26]: 1. large amount of data, 2. massive computational power, 3. high-capacity systems and 4. independent and identically distributed (i.i.d.) tasks.

Today's hardware and research practices guarantee almost always factors 2. and 3. On the other hand, factor 1. tends to be valid for the majority of the problems tackled by ML models, although there are still areas where a lengthy data acquisition process and/or lack of human annotations prohibit the availability of large amounts of (labelled) data for particular tasks.

Lastly, the i.i.d. assumption (factor 4.) seems to be the greatest vulnerability for the majority of ML models. When this assumption is violated, by introducing changes or distribution shifts that take the models outside of their current settings, it is empirically proven that these algorithms are bound to fail (tend to not generalize well—in ML terms). These distribution shifts vary from simple perturbations to the input data (adversarial samples) to different data distributions being used during development and deployment.

Unlike physical models, statistical models are not always robust to distribution shifts when at least one of the previously mentioned factors is violated. Generalization across problems and distribution shifts is an open challenge in the ML community and a gap that causal reasoning promises to bridge.

Causal reasoning is the analysis of a task/problem in terms of cause-effect relationships between the different variables of interest [25]: if a variable A is a direct cause of variable B, we express it as $A \rightarrow B$ (causal diagram: A causes B or B is the effect of A).

Causal modeling lies somewhere in-between these two types of models (physical and statistical). On the one hand, it takes into account causal relationships between the different variables of interests, like physical models, and on the other hand it is data-driven (learnt from observations), like statistical models. Figure 1 graphically demonstrates the difference between a statistical and a causal model.

Zoom In Zoom Out Reset image size

Given an input X and a target prediction Y, a statistical model tries to estimate $\displaystyle P(Y|X)$. A causal model, although data-driven, extends the statistical setting by considering additionally the cause-effect relationships between the various variables of interests. Using causal terminology, a task can be either [25]:

• Causal: when $X \rightarrow Y$, prediction of effect from cause or
• Anti-causal: when $Y \rightarrow X$, prediction of cause from effect.

According to the principle of independent causal mechanisms [26–28], the causal breakdown of a system consists of independent variables that do not influence or inform each other ⁶ . In other words, a sudden change or distribution shift to one of these variables would not change the mechanism of data generation, which is invariant and can be used to improve model robustness.

In order to demonstrate the power of causal models, let us consider the following scenarios where we have two observable variables X and Y. According to the common cause principle (Reichenbach's Principle) ⁷ , we have the following scenarios [26]:

• (i)  
  $X \rightarrow Y$ where X causes Y or Y is the effect of X
• (ii)  
  $X \leftarrow Y$ where Y causes X or X is the effect of Y
• (iii)  
  $X \leftarrow Z \rightarrow Y$ where both variables X and Y are caused by another variable Z

A statistical model would not be able to differentiate between these three cases, simply due to the fact that the joint observable distribution $P(X,Y)$ is the same in all three cases. By taking into account, the causal relationships between the variables of interest, a causal model has automatically more information—which in turn if appropriately used by ML algorithms—can lead to models which are more robust to certain types of distribution shifts.

Understanding the brain is arguably one of the most interesting problems of our time. EEG technology has made it possible for researchers to measure the brain activity of a subject in a non-invasive manner, while various tasks are performed. Since the underlying organizational structure of the brain still remains a puzzle to be solved, there is no analytical model (physical modeling) that can accurately describe its functionality for the entirety of human behavior ⁸ . Therefore, the success of BCIs mainly relies on the recent advances in ML (statistical models).

BCI models can be distinguished between encoding and decoding models (examples are shown in figure 2), known in ML terms as generative and discriminative models:

• Encoding models : given a known performed task/ stimulus Y, the goal of these models is to estimate the brain activity X (brainwave) corresponding to that particular task, i.e. $P(X|Y)$
• Decoding models: given an observed brain activity X (brainwave), the goal of these models is to predict the performed task Y corresponding to that particular brainwave, i.e. $P(Y|X)$

Zoom In Zoom Out Reset image size

By introducing causal (cause-effect) relationships between the observed brain activities and their corresponding performed tasks, these two types of brainwave models can be identified with contrasting causality terms:

• If $X \rightarrow Y$: encoding models are anti-causal and decoding models are causal
• If $Y \rightarrow X$: encoding models are causal and decoding models are anti-causal

Although encoding BCI models (BCI systems that modulate the current neural activity so as to bring it within a desirable range associated with some targeted label/state) can be met in real-life application like transcranial direct current stimulation for limb rehabilitation [45] or Parkinson's disease [46], the vast majority of BCI systems are decoding models, therefore we will focus only on BCI brainwave decoders in this work.

To make use of causality models, as described in the previous section, it is crucial to break down the task of brainwave analysis in a number of independent variables, according to the Principle of Independent causal mechanism, and determine the underlying causal relationships between them. To effectively perform this analysis, the various tasks of BCI paradigms need to be first categorized. Unlike previous research works [47], we argue that this categorization is bound to two core properties—the presence of task stimulus and the voluntary engagement of the subject.

Based on the presence or absence of stimulus, a BCI paradigm can be identified as:

• Exogenous: when the brain activity is modulated by the presence of stimuli which can take any form of sensory input.
• Endogenous: when the brain activity is inherently modulated, without the intervention of any external stimulus.

On the other hand, based on the subject's voluntary engagement, a BCI paradigm can be identified as:

• Voluntarily Engaged: when the subject willingly generates a particular brain activation or coactivation pattern like in the case of motor imagery [48].
• Involuntarily Engaged: when the subject has no control of the generated brain activation pattern. In other words, the ongoing brain activity is modulated without any conscious effort from the subject [49].

Therefore, all BCI paradigms can be categorized using a combination of the above mentioned categories. For example, P300-speller [31] (a type of BCI that allows users to spell words and sentences by focusing their attention on specific characters) is exogenous (since there is a visual stimulus) and voluntarily engaged (since the subject initiates the brain activity). On the other hand though, P300 Lie Detector (LD) [32] (a type of BCI that uses event-related potentials (ERPs) to detect deception) is again exogenous and involuntarily engaged (since the subject has no control over the generated brain patterns from the visual stimulus). In table 1, we demonstrate examples of some BCI paradigms based on this proposed causal framework.

In our causal analysis of all BCI paradigms, we are mostly interested in determining the causal relationship between the observed EEG brainwave signal X and the labelled task Y. But before analyzing each one of these four types of BCI paradigms separately in terms of their independent components and causal relationships, we will introduce a variable that all categories share: the true underlying unobserved BCI-relevant brain activity Z. Inspired by [25], the EEG singal X can be considered a noisy measurement of the true unobserved neural activation patterns Z i.e. $Z \rightarrow X$ [29, 50].

Given a BCI brainwave decoder that is exogenous, when a stimulus S is applied, there are two possible scenarios:

• (i)  
  The task Y causes the stimulus S which in turn generates the corresponding brain activation pattern Z contained in the measured signal X (exogenous and voluntarily engaged). In this case, the causal diagram of the variables of interest is: $Y \rightarrow S \rightarrow Z \rightarrow X$.
• (ii)  
  The task Y and the stimulus S together—as a pair—cause the generation of the corresponding neural activation pattern Z measured within X (exogenous and involuntarily engaged). In this case, the causal diagram of the variables of interest is: $(Y,S) \rightarrow Z \rightarrow X$.

For example, in the case of P300 speller, the user chooses which letter they want to focus on and in turn that letter causes the right stimulus which results in the observed EEG brain signal. On the other hand, in the P300 lie detector case, the participant-suspect is presented with a crime-related image (stimulus) and depending on whether he/she is aware of it or not, the P300 response may involuntarily be emitted. Given these two causal diagrams, in both cases, a decoding brainwave model of this category is anti-causal.

When the system is endogenous, there is no stimulus S applied that can initiate any task. Therefore, we need to investigate further and determine the true source of the neural activation pattern Z as well as the causal sequence of all involved modules.

In the case of endogenous and involuntarily engaged BCIs, no stimulus can initiate a task's brain activity and at the same time the subject has no control over the generated brain signals. Therefore, in order to determine the brainwave source in this case, we will go through two different BCI examples of this category, namely epileptic seizure detection [19, 20] and driver's drowsiness detection [51]. In the first example, a brainwave decoding model needs to be able to distinguish between normal and epileptic brain activity. In the second example, a brainwave decoding model needs to be able to distinguish between the brain activity of alertness and drowsiness. In both cases, the true underlying brain activity changes by factors that are unknown to and uncontrollable for the subject and essentially affect their current state. As a result, in this category we will introduce the environment E variable. The changes of E variable determine the task label Y (e.g. normal or epileptic brain activity) which in turn generates the corresponding neural activation pattern Z measured by X, i.e. $E \rightarrow Y \rightarrow Z \rightarrow X$. Given this causal diagram, a decoding brainwave model of this category is anti-causal.

Finally, in the case of endogenous and voluntarily engaged BCIs, producing the causal diagram is not a trivial task to perform. Here, no stimulus can initiate a the brain activity of a task, but the subject has full control over the generated brain signals. Therefore, the intention I of the subject, can be considered the source of the BCI-relevant neural activation pattern. In this case though, different parts of the brainwave sequence can be used during modeling. We will elaborate with the following scenarios. Let Dec be a decoding model that uses brainwaves to discriminate between executed left and right hand movements. If Dec uses the brain signal during the execution of the movement, the intention has caused the prediction label Y which causes the brain activity, i.e. $I \rightarrow Y \rightarrow Z \rightarrow X$. If Dec uses the motor preparation brain signal X [52], then the signal precedes the final result (the movement execution) but still the prediction label Y of the pre-movement signal is the cause of the brain activity, i.e. $I \rightarrow Y \rightarrow Z \rightarrow X$. In both cases, there is one common causal diagram and the decoding model is anti-causal.

The causal diagrams of figure 3 show how different factors can influence brainwave activity. Based on these diagrams, we can develop causal factorizations, which are mathematical models that represent the relationships between these factors, to further explore the problem of brainwave modeling. For convenience, we will focus only on the case of brainwave decoding models. In other words, given an input EEG signal X we want to predict the associated task Y i.e $P(Y|X)$ ⁹ .

• Voluntarily Engaged—Exogenous ($Y \rightarrow S \rightarrow Z \rightarrow X$):
  $$\begin{align} \begin{aligned} P\left(Y,S,Z,X\right) & = P\left(Y\right)P\left(S|Y\right)P\left(Z|S,Y\right)P\left(X|Z,S,Y\right) \\ & = P\left(Y\right)P\left(S|Y\right)P\left(Z|S\right)P\left(X|Z\right) \end{aligned} \end{align} \tag{ 1 }$$
• Involuntarily Engaged—Exogenous ($(Y,S) \rightarrow Z \rightarrow X$):
  $$\begin{align} \begin{aligned} P\left(Y,S,Z,X\right) & = P\left(Y,S\right)P\left(Z|Y,S\right)P\left(X|Z,Y,S\right) \\ & = P\left(Y,S\right)P\left(Z|Y,S\right)P\left(X|Z\right) \\ & = P\left(Y\right)P\left(S\right)P\left(Z|Y,S\right)P\left(X|Z\right) \end{aligned} \end{align} \tag{ 2 }$$
  since Y and S are independent.
• Involuntarily Engaged—Endogenous ($E \rightarrow Y \rightarrow Z \rightarrow X$):
  $$\begin{align} \begin{aligned} P\left(E,Y,Z,X\right) & = P\left(E\right)P\left(Y|E\right)P\left(Z|Y,E\right)P\left(X|Z,Y,E\right) \\ & = P\left(E\right)P\left(Y|E\right)P\left(Z|Y\right)P\left(X|Z\right) \end{aligned} \end{align} \tag{ 3 }$$
• Voluntarily Engaged—Endogenous:
  $$\begin{align} \begin{aligned} P\left(I,Y,Z,X\right) & = P\left(I\right)P\left(Y|I\right)P\left(Z|Y,I\right)P\left(X|Z,Y,I\right) \\ & = P\left(I\right)P\left(Y|I\right)P\left(Z|Y\right)P\left(X|Z\right) \end{aligned} \end{align} \tag{ 4 }$$

Zoom In Zoom Out Reset image size

The experimental conditions (under which a BCI decoding task takes place) are crucial and they need to be designed carefully to avoid any possible distribution shift between the training and test EEG sets. BCI researchers need to take into account the following factors:

• (i)  
  Demographics. Demographics (e.g. sex, age and handedness) are considered to be the cause of differences in brain activation patterns among individuals during the execution of the same task. It is advisable that the subjects' demographics in training EEG set resemble the population represented in the test EEG set. If the goal is a global population-agnostic brainwave decoder, the training EEG set need to be diverse in terms of available subjects in order to capture as large part of the population as possible.
• (ii)  
  Stimuli Shifts $P_\mathrm{S}(\mathrm{S})$. Brainwave decoders, that rely on the presence of stimuli to modulate the necessary brain activity, are heavily dependent on the exact type and form of this external sensory input. Therefore, the stimuli used for the collection of the training EEG set and the stimuli used during the deployment of the ML model need to be perfectly aligned and any possible shifts should be avoided. For example, in the case of a steady state visually evoked potentials (SSVEP) BCI-speller, the specific frequencies that correspond to each alphabetical letter needs to be the same in both training and test EEG sets.
• (iii)  
  Shifts on Subject's Intention $P_\mathrm{S}(\mathrm{I})$. Subject's intention is the starting point, and therefore vital, in all voluntarily engaged and endogenous brainwave decoders. It is important though that these intentions are aligned between the training and test EEG sets since different intentions can lead to different BCI-relevant neural activity. For example, in a MI BCI, a subject can imagine raising his/her right hand while another subject can visualize of closing his/her right fist. Although both movement include the movement of the right hand, they can lead to very different brain activities which might mislead the brainwave decoder. It is the BCI researcher's duty to ensure a baseline behaviour for all involved participants during the data collection as well as the deployment phase in order to avoid any unpredictable distribution shifts.
• (iv)  
  Shifts on Environmental Conditions $P_\mathrm{S}(\mathrm{E})$. The environment factor is essential for involuntarily engaged and endogenous brainwave decoders and therefore, as in the case of stimuli and subject's intention, the environment in the training EEG set should resemble as close as possible the environment represented in the test EEG set.

4.2.1. Data scarcity

4.2.2. Acquisition shift $P_\mathrm{S}(X|Z)$

Our brain is a complex system that is not only structured but also functions differently from one person to another, a phenomenon that is often referred to as population shift or inter-subject variability. Each subject has a unique brain anatomy and functionality of each individual that results in a variety of different neural activity patterns that can be observed on the surface of the brain using EEG signals. Additionally, brain functionality does not only differ between subjects but across time as well. According to various studies [71], a certain variability in the acquired EEG signal can be observed when the same subject attends the same experiment in different times (sessions), a phenomenon known as inter-session variability. In the case of BCI brainwave decoders, subject shifts can occur in all possible identified scenarios in our causal framework (i.e. $P_\mathrm{S}(Z|S)$, $P_\mathrm{S}(Z|Y,S)$, $P_\mathrm{S}(Z|Y)$, $P_\mathrm{S}(Z|I)$). Therefore, we should turn our attention to domain-specific (BCI-specific) methods that are specifically designed to tackle these issues, found only in the area of brainwave analysis.

Over the last few years, there has been an increasing interest towards this research direction with many strong empirical evidence of increasing the performance of various BCI models. [72] projects data into a subject-invariant space while [73] proposes an adversarial inference network that learns subject invariant features. In an effort to design enhanced subject-independent ML models, [74] describes a novel technique to explicitly align feature distributions at various layers of the deep learning model. By utilizing both statistical and trainable methods (inspired by convariance-based alignment methods used in Riemannian geometry [72]). While [29, 50] approximates the subject distribution shift in the true BCI task-related brain activity among different subjects by using dynamic convolutions based on an input-dependent attention vector.

4.3.1. Artifacts

It is important to make sure that the training set for a ML model is as similar as possible to the deployment set, in terms of the distribution of classes (task class balance). This is because if the training set is imbalanced, the model may not be able to generalize well to new unseen data during deployment.

In the cases where this is not possible, specific techniques should be employed to correct the bias in estimating the training loss. For instance, [84] proposes a BCI method for seizure detection that effectively moderates the bias in performance caused by imbalanced class distribution by assigning different weights to the EEG samples.