Quantification of anticipation of excitement with a three-axial model of emotion with EEG

Our motivation for this study originated from an idea to quantify a putatively complex state of mental representations, 'Kansei'. In the Asian languages, 'Kansei' (in Japanese, a direct translation would be 'sensitivity' or 'sensibility') is a widely accepted concept that reflects a feeling that is exogenously triggered by something and that is often accompanied by mental images of a target [16]. Kansei expressions often reflect a mixture of affective and cognitive states, as exemplified in recent views on human emotion [6]. For instance, 'Waku-Waku' is one of the onomatopoetic states. It is typically defined as an emotional state in which one is excited emotionally while anticipating an upcoming pleasant event(s) in the future cognitively. In Russell's circumplex model, the best approximation of 'Waku-Waku' would consist of placing the 'excited' emotional state between 'alert' and 'elated' with a moderate level of high valence (pleasant) and a moderate level of high arousal. The closest synonyms for 'Waku-Waku' in English may be 'anticipatory excitement' or 'a sense of exhilaration', in which one envisages on upcoming future. As our eventual goal is to quantify such a putatively complex state, it was hypothesized that the addition of an extra dimension of cognitive state might better explain the state of 'Waku-Waku'. Therefore, we hypothesized that a combination of multidimensional axes that incorporates both affective and cognitive dimensions, including prediction, could be a sensible model to reflect the state of Kansei. In a broad sense, Kansei not only includes a meaning for ones' state but also reflects one's traits or preferences based on one's experiences. For the sake of clarity, we will focus on the state of one's affect and cognition in this article. The other aspect of trait shall be treated elsewhere.

Here, we first aimed to build a psychological model for a Kansei expression, mainly focused on the state of 'Waku-Waku.' Similar to the expression 'Waku-Waku,' other onomatopoeic expressions are commonly used in the Japanese language to reflect an emotional state. Therefore, neural-based quantification of Kansei would be of great advantage for many industrial applications. Based on a conventional two-dimensional model of affect [1], we hypothesized a three-dimensional model composed of 'valence' and 'arousal' axes as well as a third dimension of prediction, 'expectation.' It should be noted that 'expectation' was defined as only having the cognitive aspect of anticipation, putatively forming explicit imagery of a future under uncertainty. It may provoke a degree of emotional valence; however, such potential overlap was treated mathematically (see Methods). By definition, 'Waku-Waku' (anticipatory excitement) holds emotional valence and arousal when expecting a pleasant event in the future as described above; therefore, it was expected to be loaded with both affective and cognitive states of mind.

Recent developments in the neurosciences allow us to build an interface to monitor our neural status in real-time by presenting the neural state on a screen or a device. Today, the demand for techniques such as neurofeedback or BCI is increasing in clinical settings and even in industrial applications [17–19]. Several variations of neurofeedback methods exist; some visualize one's neural state by using functional magnetic resonance imaging (fMRI) for training and therapeutic purposes [20, 21]. BCI with electroencephalography (EEG) has also been widely employed to detect a locus of attention with the P300 component [18, 19] or as a means to assess the conscious level with α-waves [22–24]. These classical models typically acquire data from one or a few electrode channels and focus only on a specific frequency range of interest. With improved computational resources, BCIs focusing on limb movement can acquire real-time feedback from independent components (or common spatial patterns) recorded from whole-scalp electrodes [25–27] rather than from data from only one or a few channels as was common in the past. Currently, one can effortlessly expect an even more complex BCI to be achieved, such as a neurofeedback system incorporating multiple neural indices such as the multidimensional model proposed here.

In this study, we first modeled 'Waku-Waku' with three dimensions, namely, valence, arousal, and expectation, given the psychological model for 'Waku-Waku' as an intermixed state of higher-order affective and cognitive functions [7]. Having confirmed the significance of all axes in the psychological model, we then derived electrophysiological markers using EEG corresponding to each axis. Finally, based on the outcomes, we proposed a three-axis linear equation model of a 'brain-emotion interface (BEI)' to quantify 'Waku-Waku' in real time by incorporating the psychological model with corresponding neural markers. Below, we show the resultant psychological model and the EEG markers for each axis and propose a prototype 3-D model for the quantification of 'Waku-Waku.' Potential applications of the BEI as a tool of Kansei engineering and the limitations in this study are discussed.

Thirty-six healthy young adults (19 females) aged between 19–27 years were recruited locally. Due to technical errors or failure to attend to all three sessions in the lab, some participants were rejected. As a result, the data from 28 participants (16 females; age [mean ± SD]: 22.17 ± 1.79) are reported here. None of the participants reported a history of neurological or psychological disorders. All participants had normal hearing abilities with either normal or corrected-to-normal vision. All participants gave their informed consent to participate in this research. The study was approved by the local research ethical committee of Hiroshima University.

Participants performed a picture rating task in which they were asked to mentalize an upcoming novel picture appearing on a computer monitor. One of three kinds of auditory cues preceded picture onset (see figure 1). These three cueing conditions were as follows: 1) a low-frequency tone predicting a positive picture ('Predictive Pleasant'), 2) a high-frequency tone predicting a negative picture ('Predictive Unpleasant'), and 3) an intermediate-frequency tone indicating equal probabilities of seeing a pleasant and an unpleasant picture ('Unpredictive'). The auditory tone was played for 250 msec followed by a blank delay period with an inter-stimulus-interval of 4000 msec. The conditional assignments for the high- and low-frequency tones were counterbalanced across participants. This assignment was fixed throughout all three visits. While arousal ratings of the pictures varied picture by picture (see figure 1 and supplementary figure 1 (avaliable online at stacks.iop.org/JNE/17/036011/mmedia)), these auditory cues were unrelated to the arousal toward the upcoming image. Notably, we categorized each picture to counterbalance picture sets across different sessions, referring to the subjective ratings of valence and arousal reported in the original article [29] (supplementary table 2 present the detailed ratings of valence and arousal. The appendix in the supplementary lists all picture codes used in this study). Participants also rated the pictures based on their subjective feelings after the experiment (see below for details).

Zoom In Zoom Out Reset image size

During the 4000 msec blank period, participants were asked to imagine what kind of image would be displayed. After the delay, an emotion-triggering picture was displayed at the center of the screen for 4000 msec (for the details of the selected pictures, see the Stimuli section below). Following the picture display, a red fixation cross appeared at the center of the screen for 1000 msec. During this response period, participants were asked to rate their subjective feeling of valence by answering the question 'how were you moved by seeing that picture' on a 4-point Likert scale: 'strongly pleasant', 'pleasant', 'unpleasant' and 'strongly unpleasant'. They reported their response by pressing either the corresponding keyboard button with the thumb, index finger, middle finger, or ring finger (for EEG) or the response button (for fMRI). Before the main task, participants underwent a brief practice session for all three conditions with a picture set independent of that used for the main task.

As a predictive cue, one of three different auditory tones (500, 1000, or 1500 Hz) was played for 250 msec via a pair of headphones worn comfortably. Before the experiment, during the practice session, the volumes of the tones were checked with the participants and adjusted to comfortable levels. The 500 Hz (the low-frequency tone) or 1500 Hz (the high-frequency tone) stimuli could predict either a pleasant or unpleasant picture, and the 1000 Hz (the intermediate tone) stimulus was used as the unpredictive cue. The assignment of high and low-frequency tones was counterbalanced across participants. The intermediate tone was fixed for the unpredictive condition across all participants.

Novel images were carefully selected from 1182 pictures from the International Affective Picture System [IAPS; 29] with the following criteria. Pictures that might cause an excessively negative affect, such as corpses or the like, or that may be counter to our local ethics were discarded. In addition, the following pictures were disregarded: pictures consisting of multiple objects that could each be the object of focus of the participant; pictures with an intermediate valence that may not evoke an adequate intensity of either positive or negative valence such as plain scenery or objects (valence ratings between 4–6, see supplementary figure 1); and pictures containing text, symbols or items that may have cultural discrepancies (i.e. guns). The resulting 320 pictures were divided into two sets of 160 pictures (one for the tests for each of the two MRI sessions with 80 trials each and the other 160 pictures for the tests for the EEG session) randomly while controlling for average ratings (as reported in the IAPS dataset) of valence and arousal. Each set of 160 pictures was counterbalanced across participants. For each set of 160 pictures, half were pleasant, and the other half were unpleasant. Each picture appeared at least once for the two predictive cue conditions ('predictive pleasant' or 'unpleasant'). Half of the 160 pictures (both pleasant and unpleasant) were displayed twice for the unpredictive cue condition ('intermediate tone'). In the selected pictures, the brightness and spatial frequency (high or low; split at 140 Hz) of the visual features were adjusted such that they were not significantly different among different sets. In addition, the contents of each picture had been visually determined (human, animal, scenery, or others), and the categorical information was also equally distributed across the sets. See supplementary figure 1 for the details of valence and arousal ratings used in this study. All IAPS pictures were novel to the participants.

A Dell 24-inch LCD monitor was used to display pictures at a 1920 × 1080 pixel resolution. A chin rest placed 56 cm away from the monitor was used to stabilize the distance between the monitor and the eye for the participants. The sizes of the pictures varied. Some were oriented in landscape, and others were in portrait; however, the original pictures were displayed to fit the monitor. All auditory and visual stimuli were delivered by Presentation software version 17.2 (NeuroBehavioral Systems, San Francisco, USA).

As noted above, the trials consisted of fMRI and EEG sessions. All behavioral tasks remained the same; however, a total of 120 trials (including 40 repeated-pictures) were conducted for each of the fMRI sessions, and 240 trials (including 80 repeated-pictures) were conducted for the EEG sessions. On each visit, different sets of pictures were used to maintain the novelty of the pictures. For the EEG sessions, each cueing condition was presented for 80 trials, resulting in a total of 240 trials for the experiment. Regardless of cueing type, the participants judged 120 pleasant and 120 unpleasant pictures.

At the completion of the main task, the participants reported their subjective feelings by rating the pictures on a 0–100 visual analog scale for all conditions. Each question was displayed at the center of the screen, and a VAS was displayed underneath. Participants answered each question on the computer monitor by moving a pointer to the left (0) or the right (100) on the scale with the left or right arrow keys on the keyboard. The participants were instructed as follows: 'The following questions concern the evaluation of your feelings during the experiment. Once a question appears on the screen, please answer the degree of feeling experienced when you heard each tone on a scale between 0–100'. For each condition, participants rated the degree of 'valence' (unpleasant to pleasant; 'Kai' in Japanese), 'arousal' (low to high arousal; 'Kassei' in Japanese), and 'expectation' (low to high expectation; 'Kitai-kan' in Japanese), as well as 'Waku-Waku' ('a sense of exhilaration' or 'anticipatory excitement' in Japanese). For example, the sentence below appeared in the middle of the screen for the 'predictive pleasant' condition: 'Please rate the degree of "pleasure" (wellness of the feeling) when you heard the low-frequency tone.' A similar sentence was used for "arousal" (degree of liveliness), but for "expectation", the word was presented with no added explanation.

Here, we selected the term 'expectation' rather than a specific word such as 'prediction' or 'time'. This was because 1) we assumed that a predictive sense would be hard to report consciously and accurately; 2) we determined that 'time' (how soon they felt/expected) was inappropriate to ask, as the timing of the upcoming picture presentation was fixed to 4 sec. Furthermore, the definition of 'Waku-Waku' might depend on how the participant would conceptualize it. For the sense of anticipatory excitement, the term 'expectation' (Kitai-kan in Japanese) is supposed to reflect anticipation. However, it should be noted that one might not be able to adequately separate and ignore his or her own emotions that are covertly attached to the expectation when rating the expectation. The feeling of expectation might unconsciously induce emotional valence. We discuss the interpretation of these results with the derived correlations in results section 3.1 and the Discussion section; however, our choice of statistical procedure—a mixed linear model—would account for such putatively correlated inputs (see below for details).

As described above, each participant rated the pictures a total of three times for each of the experimental sessions (once for the EEG and twice for the fMRI experiments). All ratings were recorded for the analysis. Notably, the creation of a model with data from only 28 participants from one EEG experiment (one data point per condition) did not meet our statistical criteria for constructing a psychological model. We decided to include the data from all three sessions to ensure that all three axes achieved the desired level of significance.

The primary aim was to construct a formula with the anticipation of excitement ('Waku-Waku') as the dependent variable and three independent variable axes, namely, valence, arousal, and expectation. We first constructed a simple linear equation with the corresponding coefficients for each axis with no intercept. It should be noted that our primary aim was to obtain a starting point of reference to quantify the feeling of anticipatory excitement, 'Waku-Waku'. A mixed linear model was applied to obtain the weights for each axis (valence, arousal, and expectation) based on the subjective ratings for the anticipation of excitement with the 'Mixed Model' function in IBM SPSS version 22. The mixed linear model analysis is similar to a regression, but it can handle complicated situations similar to those in our case, where each subject participated in tests across multiple occasions (repeated design) and in different orders or with different picture sets (between-subject variability). Compared to the general linear model (GLM) that assumes independence across the data, the mixed linear model has an advantage in handling putatively correlated and unbalanced data. Notably, some degrees of correlations and random effects among the subjective ratings across the axes were assumed; a typical GLM approach might not be suitable in this case. The resultant psychological model, therefore, takes the putatively correlated effects into account. The assignment of counterbalanced tones, picture sets, the examined domains of the measurements (two MRI measurements and one EEG measurement), and the order in which the MRI or EEG measurement experiments were performed were included as covariates of no interest. In the mixed model function, repeated covariance type was set to 'scaled identity', and the maximum likelihood method was used to estimate the linear mixed model with 100 iterations. Estimates of the fixed effects were standardized and taken as coefficients for each independent variable.

As described above, the inclusion of at least valence and arousal from the original circumplex model was expected to be fundamental to the model. To validate the observed coefficients, the likelihood ratio test (χ2 statistic) was evaluated for each axis. All axes met the significance level (p <.05) as reported in section 3.1. The arousal axis did not meet the criteria when including only one of three sessions. That was one of the reasons to include the behavioral results from the MRI sessions, which otherwise could have been part of an independent study.

2.6.1. Recording procedures.

2.6.2. Analytical procedures.

2.6.3. Rejection criteria.

2.6.4. Clustering independent components.

2.6.5. Spectrogram computation.

2.6.6. Evaluation of the proposed BEI model.

See figure 2 for a summary of the subjective ratings for each cueing condition ('predictive pleasant,' 'unpredictive,' and 'predictive unpleasant'). Based on the ratings for valence, arousal, expectation, and Waku-Waku, the linear models with fixed weights were tested with valence and arousal as independent variables for a conventional two-dimensional model and with valence, arousal, and expectation as independent variables for a three-dimensional model. For both models, Waku-Waku was the dependent variable.

Zoom In Zoom Out Reset image size

With the 2-axis model, the result of the mixed linear model analysis (see Method section 2.5) revealed that 'Waku-Waku ("W")' was modeled with the estimates of the fixed effects of valence and arousal as follows (adjusted R² =.90):

$$\begin{equation}{\text{W }} = {{ }}.67{\text{*Valence }} + {{ }}.34{\text{*Arousal}}\end{equation} \tag{ 1 }$$

The 95% confidence intervals for valence and arousal were.66–.80 and.24–.35, respectively. Similarly, the mixed linear model for the three-axis model with the addition of the expectation fixed effect is depicted as follows (adjusted R² =.93).

$$\begin{equation}{\text{W }} = {{ }}.38{\text{*Valence }} + {{ }}.11{\text{*Arousal}} + {{ }}.51{\text{*Expectation}}\end{equation} \tag{ 2 }$$

The 95% confidence intervals for valence, arousal, and expectation were.27–.46,.04–.16, and.46–.64, respectively. As expected, the three-dimensional model outperformed the two-dimensional model in terms of the accuracy by 3% of the variance. Notably, the added third axis of expectation was significant and highly loaded. When including only one of the experimental sessions, the coefficient for the arousal axis did not meet our criteria (at p <.05), but that of the valence and expectation axes did. For completeness, here are the fitted formulas for each session: [MRI1: W =.53*V +.04*A +.45*E; MRI2: W =.30*V +.12*A +.52*E; EEG: W =.29*V +.16*A +.61*E], where W, V, A, and E correspond to Waku-Waku, valence, arousal, and expectation, respectively. The values of the Akaike Information Criteria (AIC), an index that assesses the goodness-of-fit of each model, computed for each session (MRI1, MRI2, and EEG) with the three-axis models were 670, 686, and 669, respectively. This result implies that the formula for the EEG session might have been the best compared to those for the MRI sessions. However, the differences in the values among the three sessions were and negligible, indicating that the AIC values validate the comparability of the scores for all three sessions.

In addition, direct pairwise correlations were examined between Waku-Waku and each of the three axes. The resulting correlation coefficients (r²-values; p-values in parentheses) for valence, arousal, and expectation were.59 (p <.001),.04 (p <.005), and.66 (p <.001), respectively. These direct correlations roughly reflect the weight balance of the coefficients derived in formula (2). The correlations among the three axes were also examined. The resulting correlation coefficients for the valence and arousal, valence and expectation, and arousal and expectation pairs were.00 (.37),.55 (<.001), and.05 (<.001), respectively. Please note that the mixed model should have taken care of these correlations as well as the other unbalances or random effects from the samples. Supplementary figure 2 shows the scatterplots of Waku-Waku ratings as a function of the different axial counterpart.

From the GMM analysis on the inverse weights of the 949 ICs, a comparison of the BIC values for 1–60 clusters suggested that 15 clusters should be extracted. Figure 3 shows the BIC values across all examined numbers of clusters to be extracted (left panel) and the topographical IC maps for each common IC cluster (right panel; supplementary figure 3 also depicts the centroid coordinates of the dipole location for each cluster). Notably, these clusters of ICs are irrespective of the order of the identified ICs from the final ICA procedure. These clusters represent the shared ICs across subjects based on the topographical maps (inverse weights) of all ICs.

Zoom In Zoom Out Reset image size

Spectral power was examined for each IC cluster (see figure 4 for a summary of significant and marginally significant ICs; supplementary table 1 contains the statistical results of all IC clusters for completeness). For the valence (figure 4(a)), arousal (figure 4(b)), and expectation (figure 4(c)) axes, 5, 2, and 2 IC clusters emerged as significant, respectively. Emotional valence was represented by several EEG features that dissociated the pleasant and unpleasant conditions after picture onset, such as the θ-band of IC clusters 1, 5, and 7, the α-band of IC clusters 5 and 6, and the β-band of IC clusters 7 and 14. Of these, IC cluster 5 was shared by all sampled participants. Arousal was instead quantified by two IC clusters that differentiated the high- and low-arousal conditions after picture onset: the α-band of IC cluster 7 and the β-band of IC cluster 11. Of these, the α-band of IC cluster 7 was more reliable, shared by a greater percentage (86%) of participants. As expected, the θ-band of IC cluster 10 significantly dissociated the predictive pleasant and unpredictive conditions during the precue interval, and it was slightly more reliable than IC cluster 15, which also dissociated the above conditions.

Zoom In Zoom Out Reset image size

Supplementary table 4 shows the results for all ICs, including marginally significant ICs, as well as additional supplementary comparisons focusing on 'predictive pleasant vs. predictive unpleasant' and 'both predictive conditions vs. unpredictive'. Some IC clusters were shared by all participants (100%), but some were not (i.e. valence cluster 11 for 79% of the participants). Across all comparisons, some IC clusters were shared on different axes, such as cluster 7 for valence and arousal at different frequency bands. No single IC cluster (and frequency band) survived the correction across all three axes.

Given the three-dimensional psychological model (2) and corresponding neural correlates selected for each axis, we first propose a 'conceptual' BEI model to estimate Waku-Waku ('We') below. Assuming that formula (2) is valid and that the corresponding EEG features could estimate each axis value (ranging between 0–100), the psychological axes in formula (2) consist of subjective ratings that may be replaced with corresponding EEG features.

$$\begin{equation}{{\text{W}}_{\text{e}}} = .38 * {\text{EE}}{{\text{G}}_{{\text{Valence}}}} + .11 * {\text{EE}}{{\text{G}}_{{\text{Arousal}}}} + .51 * {\text{EE}}{{\text{G}}_{{\text{Expectation}}}}\end{equation} \tag{ 3 }$$

where EEG_Valence, EEG_Arousal, and EEG_Expectation correspond to the normalized (0–100) spectral power of the ICs for the valence, arousal, and expectation axes, respectively. Here, a conversion of EEG spectral power from its standard unit (dB) to a normalized unit (0–100) is necessary because the unit acquired for the psychological model ranged between 0–100, while the spectral density of EEG signals does not span this range.

Given the statistical results, we selected the most robust and reliable IC candidates that survived our criteria because multiple EEG features were identified. The selection criteria were a combination of the highest Z-score and the highest proportion of participants who were covered by the selected IC cluster. The final selected formula for the estimated Waku-Waku (W_e) is expressed as

$$\begin{equation}{{\text{W}}_{\text{e}}} = .38 * {\boldsymbol{IC}}5\left( \theta \right) + .11 * {\boldsymbol{IC}}7\left( \alpha \right) + .51 * {\boldsymbol{IC}}10\left( \theta \right)\end{equation} \tag{ 4 }$$

where IC5, IC7, and IC10 correspond to the IC clusters reported in figure 3, figure 4 and supplementary table 1; the θ and α in parentheses correspond to the frequency ranges of interest for each IC. It should be noted that as described in the above section, each EEG feature assumes that the values are normalized (0–100) within its power distribution. An example workflow using formula (4) is depicted in figure 5.

Zoom In Zoom Out Reset image size

In figure 5, we propose an example BEI workflow. First, the raw EEG data from an epoch of 4000 msec is imported. The imported data are first preprocessed (including 4–20 Hz filtering and data mining) for subsequent quantification of the valence, arousal, and expectation axes. The filtered data are then sphered to extract the independence across channels. The IC weights of interest are extracted from the sphered data, and the spectral power density for each target frequency band (θ of IC5 for valence, α of IC7 for arousal, and θ of IC10 for expectation, as determined in section 3.2; see also figures 3 and 4) is obtained. The obtained spectral power is then scaled to a normalized unit (0–100) based on the prior distribution of the spectral power of each IC candidate. Here, the prior distribution of each EEG feature may be assessed from the entire collected data for offline analysis. Alternatively, for a real-time workflow, the prior distribution may be constructed by collecting a certain duration of data as a calibration stage prior to running the above workflow. Finally, the coefficients for each axis are combined with the normalized EEG features as in formula (4). With the rescaled EEG features, the final values for valence ('Val.'), arousal ('Aro.') and expectation ('Exp.') are replaced by the values obtained from the EEG data as in formula (4).

Although we proposed a hypothesis-driven BEI model to quantify Waku-Waku with three putatively separable axes with corresponding neural markers, the model still requires validation with existing data samples. Although robust interpretation may be limited, we performed three additional analyses: 1) the spectral power of the ICs were compared for the contrast between 'predictive pleasant' and 'predictive unpleasant' to compare against the valence and expectation axes; 2) the spectral power of the ICs were compared for 'predictive' and 'unpredictive' to derive a potential predictive axis and compare it against the expectation axis; and 3) the estimated Waku-Waku was computed and compared against the subjective ratings for each condition.

First, we compared EEGs for the directly contrasting conditions in which Waku-Waku was rated the highest (predictive pleasant) and the lowest (predictive unpleasant). This contrast is unsuitable as a neural response for anticipation because it conceptually subtracts out the predictive component. However, this contrast might provide informative insight on a comparison with the valence ('seeing pleasant' vs. 'seeing unpleasant') or expectation ('predictive pleasant' vs. 'unpredictive') axis. Figure 6 below shows the results of the additional analyses: the θ-band of IC2, β-band of IC7, and θ and α-band of IC11 were significant (see supplementary table for the details).

Zoom In Zoom Out Reset image size

To further supplement the inspection, another comparison was performed on the two predictive conditions combined vs. the unpredictive conditions. In principle, this contrast supposedly reflects the anticipation of emotionally salient images regardless of their emotional value (either pleasant or unpleasant) with respect to the emotionally vague condition. Therefore, this contrast could have theoretically replaced the expectation axis. The θ-band of IC2, β-band of IC5, θ-band of IC6, and θ-band of IC11 were found to be significant.

Finally, the proposed BEI model was applied to the existing data by estimating Waku-Waku values based on the given spectral power distributions across the conditions. It was assumed that Waku-Waku can be quantified with the three dimensions, provided the existence of at least one target EEG feature for all axes. However, 22 out of 28 participants held all of the corresponding ICs in our proposed model, and 24 out of 28 held all corresponding ICs with 'prediction (IC cluster 5)' as the third axis.

The table below summarizes the estimated score for each axis and the final Waku-Waku score for each condition. As was expected, using the statistically significant EEG features selected in formula (4), the scores for each axial contrast straightforwardly reflected each pair of contrasts (i.e. the valence scores for the pleasant images were estimated higher than those for the unpleasant images, etc). More importantly, the main candidate of this study, Waku-Waku, seemed to well quantify the subjective ratings for each condition, as shown in figure 2. Waku-Waku was rated the highest for the predictive pleasant, followed by the unpredictive and predictive unpleasant conditions. Given that our proposed model ('W_e') is valid, anticipating a highly pleasant and arousing picture might drive excitement the most, while anticipating a picture with unpleasant and less arousal drove it the least. It should be noted that only one value per condition is reported and without its mean or standard deviation because this estimation was performed from the average of all EEG features for each axis.

As a counterpart model for the third axis, an alternative model of Waku-Waku ('W_p') was examined by merely replacing the third axis with 'prediction' regardless of emotional valence (the 'unpredictive' vs. 'predictive pleasant combined with predictive unpleasant' contrast). As an EEG feature for this axis, the θ-band of IC cluster 5 was selected as a candidate. The estimated score did not follow the trend achieved by the proposed model. The estimated Waku-Waku scores for the 'predictive pleasant' and 'predictive unpleasant' conditions were higher than those for the 'unpredictive' condition; however, it seemed that the score for the 'predictive unpleasant' condition was the highest, in which Waku-Waku was supposedly the lowest instead.

First, a psychological task was given to participants to visually trigger their emotions. The participants were required to engage in anticipation of upcoming stimuli and to report their subjective feelings as they were anticipating one of three conditions (predictive pleasant, predictive unpleasant, and unpredictive) on four factors: 'Waku-Waku,' valence, arousal, and expectation. As was expected, the 3-D psychological model of 'Waku-Waku' including an axis for 'expectation', achieved an adequate fitting accuracy. As quantified by the adjusted R² values, the fit of the 3-D model was better by 3% than that of the 2-D model. The improvement of the fit may be trivial to our aim, but the results indicate that the multiaxis model of emotion may be feasible to adapt.

Additionally, the 3-D model revealed a high loading on the third axis relative to that of the remaining two axes that the classical 2-D circumplex model of emotion would propose [1]. It assures that the subjective feeling of 'Waku-Waku' may be intimately linked to anticipation, followed by momentary emotions of pleasure and arousal. We also found that the two subjective ratings of valence and expectation moderately correlated with that of Waku-Waku, but they were not equal (the correlation coefficient (r²) was not very strong). These results suggest that the association between expectation alone or valence alone with Waku-Waku were not the same but both factors reflected a subset of Waku-Waku feelings to some extent. This result was plausible because 'Waku-Waku' is described as the state of one's heart being moved due to being pleased and expecting something pleasant. Therefore, both valence and expectation may be assumed to be highly loaded. Nevertheless, as was discussed earlier, Kansei—our instantaneous mental state of emotion—may be modeled by multifold human affects and cognition [6–8, 11]. Furthermore, the resultant variations in the coefficients for valence, arousal, and expectation suggest that the three aspects of psychological dimension differentially contribute to the feeling of Waku-Waku.

A psychological model should not be restricted by these proposed axes (valence, arousal, and expectation). In particular, the concept of the third axis in our case was a conceptualization of the time domain; however, any other dimensions associated with the human senses may be adapted. According to another 3-D emotional space model (pleasure, activation, and dominance, also known as the PAD model), dominance [37] could have been the third axis, and one could also build a nonlinear model as well. As discussed below, a sense of prediction that may putatively reflect a likelihood of expectation may also be a key factor because it is rooted in the concept of predictive coding. Furthermore, a psychological constructionist approach assumes emotion to be the interplay of the multiple brain networks that may require potentially more than three dimensions or other sophisticated methods [6]. Nevertheless, it is plausible to propose that the modeling our putatively complex nature of awareness would benefit from a multidimensional model.

To determine the neural correlates for each axis of the BEI, the spectral power of the ICs of the EEG data were analyzed. The valence and arousal axes were quantified from the duration over which the participants visually observed the emotion-triggering picture. The expectation axis was determined from the delay period in which the participants anticipated the upcoming picture according to the played auditory cue, which was tied to valence type. A conventional spectral power analysis was performed for each IC, and dissociable neural markers were identified for each axis. This dissociation is one of the essential aspects of the validation of our model. Because the multidimensional model assumes the independence of the psychological axes and their corresponding neural markers, it was necessary to confirm that the EEG features selected for each axis were also dissociable. For example, a previous study suggested that multiple ICs could be loaded onto one axis and that one IC might also contribute to the other axes to some extent [38]. In addition, some studies support the view that the functions of a brain region are ubiquitous and not limited to unitary and discrete functions [6]. Therefore, we are not necessarily stringent with regard to this overlap; however, a complete overlap between any one pair of axes would be another story. If the same EEG features entirely correspond to a pair of axes, it may suggest that the pair would represent the same neurophysiological system. As a result, this would violate the assumption of the multidimensional model, making the addition of a new dimension meaningless. However, our results minimized the possibility of this dimension redundancy. There was no complete overlap across the three axes, supporting our proof of concept. The three-dimensional model could plausibly reflect putatively dissociable neural mechanisms. Below, we discuss the outcomes for each axis.

4.2.1. IC markers of valence.

4.2.2. IC markers for arousal.

4.2.3. IC markers for expectation

In this study, the subjective ratings of Waku-Waku and the other axial factors were not collected for every trial. However, as an attempt to validate the neural markers found in the three axes, we directly contrasted the conditions in which Waku-Waku ratings were the highest ('Predictive pleasant') and lowest ('Predictive unpleasant'). Conceptually speaking, the valence axis and this contrast might have shared the same EEG features because both contrasts putatively compare emotionally pleasant and unpleasant images. The only difference between the two was whether participants were seeing an actual image on the screen or forming a mental image. We found that no EEG features overlapped between the two contrasts. This may imply that the neural mechanisms associated with expecting a future with uncertainty and those associated with exogenously confronting a reality at present may be dissociable. To reiterate, the inclusion of anticipation was fundamental to our model to estimate Waku-Waku. Therefore, it was presumed that the EEG features selected here would not represent the expectation axis because they would eliminate this valuable perspective of anticipatory neural function by contrasting the two predictive conditions.

Another potential contrast was made on 'the two predictive conditions combined' vs. the 'unpredictive' condition. Several neural signatures emerged on this axis, and they could become a counterpart for the third axis of expectation. The selected EEG feature for predictability successfully quantified the anticipatory conditions to some extent. However, the estimation of Waku-Waku with this axis (W_p) failed to follow the subjective ratings. See table that indicates that the predictive axis rated the highest ('86') when the cue was predictive unpleasant, whereas the estimated prediction value for the predictive pleasant condition was only 41. It might have been true that participants might 'predict' an occurrence of unwanted images; however, this contrast was not sufficient to represent the 'expectation' axis. Although the fit was not great for Waku-Waku, it does not necessarily mean that this axis of prediction could not be a part of a multiaxis model of emotion. It may be possible that the neural responses for the predictive unpleasant condition (somewhat associated with an emotion of worry) might be stronger than those for the predictive pleasant condition. If the experimental design of the way participants performed the task was more closely related to the prevalence, or entropy, of expectation regardless of emotional value, such a prediction or certainty axis with its selected components might have been a proper candidate for the third axis. Altogether, this axis was not suitable for our proposed model for Waku-Waku. Finally, it would be plausible to believe that the EEG feature we selected for the contrast between the 'predictive pleasant' and 'unpredictive' conditions was appropriate for our model.