Table of contents

Brain–computer interfaces (BCIs), also known as brain–machine interfaces (BMIs), translate brain activity into new outputs that replace, restore, enhance, supplement or improve natural brain outputs. BCI research and development has grown rapidly for the past two decades. It is beginning to provide useful communication and control capacities to people with severe neuromuscular disabilities; and it is expanding into new areas such as neurorehabilitation that may greatly increase its clinical impact. At the same time, significant challenges remain, particularly in regard to translating laboratory advances into clinical use.

The papers in this special section report some of the work presented at the Fifth International BCI Meeting held on 3–7 June 2013 at the Asilomar Conference Center in Pacific Grove, California, USA. Like its predecessors over the past 15 years, this meeting was supported by the National Institutes of Health, the National Science Foundation, and a variety of other governmental and private sponsors [1]. This fifth meeting was organized and managed by a program committee of BCI researchers from throughout the world [2]. It retained the distinctive retreat-style format developed by the Wadsworth Center researchers who organized and managed the first four meetings.

The 301 attendees came from 165 research groups in 29 countries; 37% were students or postdoctoral fellows. Of more than 200 extended abstracts submitted for peer review, 25 were selected for oral presentation [3], and 181 were presented as posters [4] and published in the open-access conference proceedings [5]. The meeting featured 19 highly interactive workshops [6] covering the broad spectrum of BCI research and development, as well as many demonstrations of BCI systems and associated technology.

Like the first four meetings, this one included attendees and embraced topics from across the broad spectrum of disciplines essential to effective BCI research and development, including neuroscience, engineering, applied mathematics, computer science, psychology and rehabilitation. In addition, this fifth meeting extended the spectrum in two very important ways. For the first time, presentations were given by several people who could potentially benefit from current BCI technology–people with severe disabilities who need assistive technology for communication. One presented in person and one remotely. A Virtual BCI User's Forum allowed these presenters and other potential BCI users to speak directly to the BCI research community about the advantages and disadvantages of current BCIs and important directions for future study (see [7]). Their personal experiences and desires can help guide BCI research and development. Their active participation, particularly in regard to the selection of goals and the evaluation and optimization of new methods and systems, is essential if BCIs are to become clinically valuable and widely used technology.

The second major innovation in this meeting was the strong emphasis on ethical issues related to BCI development and use. The meeting opened with a keynote presentation entitled 'Neuroethics, BCIs and the Cyborg Myth' by Dr Joseph Fins, a noted authority on neuroethics from the Weill Cornell Medical College and the Rockefeller University. He focused on the ability of BCIs to relieve suffering and restore function, while cautioning against applications that take intentional control away from the user. Ethical issues were also addressed in several of the workshops, and arose on multiple occasions and in multiple contexts over the course of the meeting. Their prominence reflected the growing importance and difficulty of ethical issues as BCI capacities and applications grow and extend to potentially enhancing or supplementing normal nervous system function.

The 16 articles in this special section reflect the breadth, depth, growing maturity and future directions of BCI research. The first paper presents a tutorial on best practices in BCI performance measurement [8]. The following eight papers focus on specific BCI applications and on methods for increasing their usefulness for people with severe disabilities. The next two examine how brain activity and BCI use affect each other. The final five studies investigate brain signals and evaluate new signal processing algorithms in order to improve BCI performance and broaden its possible applications in some of the newest areas of BCI research, including the direct interpretation of speech from electrocorticographic (ECoG) activity [9]. Together, these papers span many aspects of BCI research, including different recording modalities (i.e. electroencephalogram (EEG), ECoG, functional magnetic resonance imaging (fMRI)) and signal types (e.g. P300 event-related potentials (ERPs), sensorimotor rhythms, steady-state visual evoked potentials (SSVEPs)). Furthermore, additional clinically related studies that were presented at the meeting but were considered to be outside the scope of the Journal of Neural Engineering will appear in a special issue of the Archives of Physical Medicine and Rehabilitation .

With a theme of 'Defining the Future' the Fifth International BCI Meeting tackled the issues of a rapidly growing multidisciplinary research and development enterprise that is now entering clinical use. Important new areas that received attention included the need for active involvement of the people with severe disabilities who are the primary initial users of BCI technology and the growing importance and difficulty of the multiple ethical questions raised by BCIs and their potential applications. The meeting also marked the launching of the new journal Brain--Computer Interfaces , dedicated to BCI research and development, and initiated the establishment of the Brain--Computer Interface Society, which will organize and manage the Sixth International BCI Meeting to be held in 2016.

References

[1] http://bcimeeting.org/2013/sponsors.html

[2] http://bcimeeting.org/2013/meetinginfo.html

[3] http://bcimeeting.org/2013/researchsessions.html (indexes individual abstracts)

[4] http://bcimeeting.org/2013/posters.html (indexes individual abstracts)

[5] http://castor.tugraz.at/doku/BCIMeeting2013/BCIMeeting2013_all.pdf

[6] Huggins J E et al 2014 Workshops of the Fifth International Brain--Computer Interface Meeting: Defining the Future Brain--Computer Interface J.1 27–49

[7] Peters B, Bieker G, Heckman S M, Huggins J E, Wolf C, Zeitlin D and Fried-Oken M 2014 Brain--computer interface users speak up: the Virtual Users' Forum at the 2013 International BCI Meeting Archives of Physical Medicine and Rehabilitation vol 95 fall supplement at press

[8] Thompson D E et al 2014 Performance measurement for brain–computer or brain–machine interfaces: a tutorial J. Neural Eng.11 035001

[9] Mugler E, Patton J, Flint R, Wright Z, Schuele S, Rosenow J, Shih J, Krusienski D and Slutzky M 2014 Direct classification of all American English phonemes using signals from functional speech motor cortex J. Neural Eng.11 035015

Objective. Brain–computer interfaces (BCIs) have the potential to be valuable clinical tools. However, the varied nature of BCIs, combined with the large number of laboratories participating in BCI research, makes uniform performance reporting difficult. To address this situation, we present a tutorial on performance measurement in BCI research. Approach. A workshop on this topic was held at the 2013 International BCI Meeting at Asilomar Conference Center in Pacific Grove, California. This paper contains the consensus opinion of the workshop members, refined through discussion in the following months and the input of authors who were unable to attend the workshop. Main results. Checklists for methods reporting were developed for both discrete and continuous BCIs. Relevant metrics are reviewed for different types of BCI research, with notes on their use to encourage uniform application between laboratories. Significance. Graduate students and other researchers new to BCI research may find this tutorial a helpful introduction to performance measurement in the field.

Objective. Steady-state visually evoked potential (SSVEP)-based brain–computer interfaces (BCIs) allow healthy subjects to communicate. However, their dependence on gaze control prevents their use with severely disabled patients. Gaze-independent SSVEP-BCIs have been designed but have shown a drop in accuracy and have not been tested in brain-injured patients. In the present paper, we propose a novel independent SSVEP-BCI based on covert attention with an improved classification rate. We study the influence of feature extraction algorithms and the number of harmonics. Finally, we test online communication on healthy volunteers and patients with locked-in syndrome (LIS). Approach. Twenty-four healthy subjects and six LIS patients participated in this study. An independent covert two-class SSVEP paradigm was used with a newly developed portable light emitting diode-based 'interlaced squares' stimulation pattern. Main results. Mean offline and online accuracies on healthy subjects were respectively 85 ± 2% and 74 ± 13%, with eight out of twelve subjects succeeding to communicate efficiently with 80 ± 9% accuracy. Two out of six LIS patients reached an offline accuracy above the chance level, illustrating a response to a command. One out of four LIS patients could communicate online. Significance. We have demonstrated the feasibility of online communication with a covert SSVEP paradigm that is truly independent of all neuromuscular functions. The potential clinical use of the presented BCI system as a diagnostic (i.e., detecting command-following) and communication tool for severely brain-injured patients will need to be further explored.

Objective. Previous work has shown that it is possible to build an EEG-based binary brain–computer interface system (BCI) driven purely by shifts of attention to auditory stimuli. However, previous studies used abrupt, abstract stimuli that are often perceived as harsh and unpleasant, and whose lack of inherent meaning may make the interface unintuitive and difficult for beginners. We aimed to establish whether we could transition to a system based on more natural, intuitive stimuli (spoken words 'yes' and 'no') without loss of performance, and whether the system could be used by people in the locked-in state. Approach. We performed a counterbalanced, interleaved within-subject comparison between an auditory streaming BCI that used beep stimuli, and one that used word stimuli. Fourteen healthy volunteers performed two sessions each, on separate days. We also collected preliminary data from two subjects with advanced amyotrophic lateral sclerosis (ALS), who used the word-based system to answer a set of simple yes–no questions. Main results. The N1, N2 and P3 event-related potentials elicited by words varied more between subjects than those elicited by beeps. However, the difference between responses to attended and unattended stimuli was more consistent with words than beeps. Healthy subjects' performance with word stimuli (mean 77% ± 3.3 s.e.) was slightly but not significantly better than their performance with beep stimuli (mean 73% ± 2.8 s.e.). The two subjects with ALS used the word-based BCI to answer questions with a level of accuracy similar to that of the healthy subjects. Significance. Since performance using word stimuli was at least as good as performance using beeps, we recommend that auditory streaming BCI systems be built with word stimuli to make the system more pleasant and intuitive. Our preliminary data show that word-based streaming BCI is a promising tool for communication by people who are locked in.

Objective. Reliability is a desirable characteristic of brain–computer interface (BCI) systems when they are intended to be used under non-experimental operating conditions. In addition, their overall usability is influenced by the complex and frequent procedures that are required for configuration and calibration. Earlier studies examined the issue of asynchronous control in P300-based BCIs, introducing dynamic stopping and automatic control suspension features. This report proposes and evaluates an algorithm for the automatic recalibration of the classifier's parameters using unsupervised data. Approach. Ten healthy subjects participated in five P300-based BCI sessions throughout a single day. First, we examined whether continuous adaptation of control parameters improved the accuracy of the asynchronous system over time. Then, we assessed the performance of the self-calibration algorithm with respect to the no-recalibration and supervised calibration conditions with regard to system accuracy and communication efficiency. Main results. Offline tests demonstrated that continuous adaptation of the control parameters significantly increased the communication efficiency of asynchronous P300-based BCIs. The self-calibration algorithm correctly assigned labels to unsupervised data with 95% accuracy, effecting communication efficiency that was comparable with that of supervised repeated calibration. Significance. Although additional online tests that involve end-users under non-experimental conditions are needed, these preliminary results are encouraging, from which we conclude that the self-calibration algorithm is a promising solution to improve P300-based BCI usability and reliability.

Objective. Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping. Approach. A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)–(d) are investigated. Main results. Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance—competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation. Significance. A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI.

Objective. We have developed a brain–computer interface (BCI) system based on real-time functional magnetic resonance imaging (fMRI) with virtual reality feedback. The advantage of fMRI is the relatively high spatial resolution and the coverage of the whole brain; thus we expect that it may be used to explore novel BCI strategies, based on new types of mental activities. However, fMRI suffers from a low temporal resolution and an inherent delay, since it is based on a hemodynamic response rather than electrical signals. Thus, our objective in this paper was to explore whether subjects could perform a BCI task in a virtual environment using our system, and how their performance was affected by the delay. Approach. The subjects controlled an avatar by left-hand, right-hand and leg motion or imagery. The BCI classification is based on locating the regions of interest (ROIs) related with each of the motor classes, and selecting the ROI with maximum average values online. The subjects performed a cue-based task and a free-choice task, and the analysis includes evaluation of the performance as well as subjective reports. Main results. Six subjects performed the task with high accuracy when allowed to move their fingers and toes, and three subjects achieved high accuracy using imagery alone. In the cue-based task the accuracy was highest 8–12 s after the trigger, whereas in the free-choice task the subjects performed best when the feedback was provided 6 s after the trigger. Significance. We show that subjects are able to perform a navigation task in a virtual environment using an fMRI-based BCI, despite the hemodynamic delay. The same approach can be extended to other mental tasks and other brain areas.

Objective. In recent years, brain–computer interfaces (BCIs) have become mature enough to immensely benefit from the expertise and tools established in the field of human–computer interaction (HCI). One of the core objectives in HCI research is the design of systems that provide a pleasurable user experience (UX). While the majority of BCI studies exclusively evaluate common efficiency measures such as classification accuracy and speed, single research groups have begun to look at further usability aspects such as ease of use, workload and learnability. However, these evaluation metrics only cover pragmatic aspects of UX while still not considering the hedonic quality of UX. In order to gain a holistic perspective on UX, hedonic quality aspects such as motivation and frustration were also taken into account for our evaluation of three BCI-driven interfaces, which were proposed to be used as a two-stage neuroprosthetic control within the EU project MUNDUS. Approach. At the first stage, one of six possible actions was selected and either confirmed or cancelled at the second stage. For the experiment, a solely event-related-potential-based interface (ERP–ERP) and two hybrid solutions were tested that were controlled by ERP and motor imagery (MI)—resulting in the two possible combinations: ERP selection/MI confirmation (ERP–MI) or MI selection/ERP confirmation (MI–ERP). Behavioural, subjective and encephalographic (EEG) data of 12 healthy subjects were collected during an online experiment with the three graphical user interfaces (GUIs). Main results. Results showed a significantly greater pragmatic quality (in terms of accuracy, efficiency, workload, use quality and learnability) for the ERP–ERP and ERP–MI GUIs in contrast to the MI–ERP GUI. Consequently, the MI–ERP GUI is least suited for use as a neuroprosthetic control. With respect to the comparison of the ERP–ERP and ERP–MI GUIs, no significant differences in pragmatic and hedonic quality of UX were found. Since throughout better results were obtained for the conventional approach and it was most preferred by the subjects, the ERP–ERP GUI seems more suitable for its deployment in actual end-users. Nevertheless, for individuals with stable MI patterns, the hybrid interface can be provided as an additional option of choice within the MUNDUS framework. Significance. Although the paramount goal in BCI research still remains the improvement of classification accuracy and communication speed, it is of significance to note that it is equally important for end-users to keep up their motivation and prevent frustration. By including pragmatic as well as hedonic quality aspects, this study is the first effort to gain a holistic perspective of the UX while interacting with BCI-driven assistive technology aimed at actual end-users. The broad-scale methodology provided valuable insights into the underlying dynamics causing the users' experience to differ across the GUIs. The results will be used to refine a BCI-driven neuroprosthesis and test it with end-users.

Objective. Several ERP-based brain–computer interfaces (BCIs) that can be controlled even without eye movements (covert attention) have been recently proposed. However, when compared to similar systems based on overt attention, they displayed significantly lower accuracy. In the current interpretation, this is ascribed to the absence of the contribution of short-latency visual evoked potentials (VEPs) in the tasks performed in the covert attention modality. This study aims to investigate if this decrement (i) is fully explained by the lack of VEP contribution to the classification accuracy; (ii) correlates with lower temporal stability of the single-trial P300 potentials elicited in the covert attention modality. Approach. We evaluated the latency jitter of P300 evoked potentials in three BCI interfaces exploiting either overt or covert attention modalities in 20 healthy subjects. The effect of attention modality on the P300 jitter, and the relative contribution of VEPs and P300 jitter to the classification accuracy have been analyzed. Main results. The P300 jitter is higher when the BCI is controlled in covert attention. Classification accuracy negatively correlates with jitter. Even disregarding short-latency VEPs, overt-attention BCI yields better accuracy than covert. When the latency jitter is compensated offline, the difference between accuracies is not significant. Significance. The lower temporal stability of the P300 evoked potential generated during the tasks performed in covert attention modality should be regarded as the main contributing explanation of lower accuracy of covert-attention ERP-based BCIs.

Objective. The present study addressed the question whether neurophysiological signals exhibit characteristic modulations preceding a miss in a covert vigilant attention task which mimics a natural environment in which critical stimuli may appear in the periphery of the visual field. Approach. Subjective, behavioural and encephalographic (EEG) data of 12 participants performing a modified Mackworth Clock task were obtained and analysed offline. The stimulus consisted of a pointer performing regular ticks in a clockwise sequence across 42 dots arranged in a circle. Participants were requested to covertly attend to the pointer and press a response button as quickly as possible in the event of a jump, a rare and random event. Main results. Significant increases in response latencies and decreases in the detection rates were found as a function of time-on-task, a characteristic effect of sustained attention tasks known as the vigilance decrement. Subjective sleepiness showed a significant increase over the duration of the experiment. Increased activity in the α-frequency range (8–14 Hz) was observed emerging and gradually accumulating 10 s before a missed target. Additionally, a significant gradual attenuation of the P3 event-related component was found to antecede misses by 5 s. Significance. The results corroborate recent findings that behavioural errors are presaged by specific neurophysiological activity and demonstrate that lapses of attention can be predicted in a covert setting up to 10 s in advance reinforcing the prospective use of brain–computer interface (BCI) technology for the detection of waning vigilance in real-world scenarios. Combining these findings with real-time single-trial analysis from BCI may pave the way for cognitive states monitoring systems able to determine the current, and predict the near-future development of the brain's attentional processes.

Objective. It is well known that to acquire sensorimotor (SMR)-based brain–computer interface (BCI) control requires a training period before users can achieve their best possible performances. Nevertheless, the effect of this training procedure on the cortical activity related to the mental imagery ability still requires investigation to be fully elucidated. The aim of this study was to gain insights into the effects of SMR-based BCI training on the cortical spectral activity associated with the performance of different mental imagery tasks. Approach. Linear cortical estimation and statistical brain mapping techniques were applied on high-density EEG data acquired from 18 healthy participants performing three different mental imagery tasks. Subjects were divided in two groups, one of BCI trained subjects, according to their previous exposure (at least six months before this study) to motor imagery-based BCI training, and one of subjects who were naive to any BCI paradigms. Main results. Cortical activation maps obtained for trained and naive subjects indicated different spectral and spatial activity patterns in response to the mental imagery tasks. Long-term effects of the previous SMR-based BCI training were observed on the motor cortical spectral activity specific to the BCI trained motor imagery task (simple hand movements) and partially generalized to more complex motor imagery task (playing tennis). Differently, mental imagery with spatial attention and memory content could elicit recognizable cortical spectral activity even in subjects completely naive to (BCI) training. Significance. The present findings contribute to our understanding of BCI technology usage and might be of relevance in those clinical conditions when training to master a BCI application is challenging or even not possible.

Objective. Assimilating the diagnosis complete spinal cord injury (SCI) takes time and is not easy, as patients know that there is no 'cure' at the present time. Brain–computer interfaces (BCIs) can facilitate daily living. However, inter-subject variability demands measurements with potential user groups and an understanding of how they differ to healthy users BCIs are more commonly tested with. Thus, a three-class motor imagery (MI) screening (left hand, right hand, feet) was performed with a group of 10 able-bodied and 16 complete spinal-cord-injured people (paraplegics, tetraplegics) with the objective of determining what differences were present between the user groups and how they would impact upon the ability of these user groups to interact with a BCI. Approach. Electrophysiological differences between patient groups and healthy users are measured in terms of sensorimotor rhythm deflections from baseline during MI, electroencephalogram microstate scalp maps and strengths of inter-channel phase synchronization. Additionally, using a common spatial pattern algorithm and a linear discriminant analysis classifier, the classification accuracy was calculated and compared between groups. Main results. It is seen that both patient groups (tetraplegic and paraplegic) have some significant differences in event-related desynchronization strengths, exhibit significant increases in synchronization and reach significantly lower accuracies (mean (M) = 66.1%) than the group of healthy subjects (M = 85.1%). Significance. The results demonstrate significant differences in electrophysiological correlates of motor control between healthy individuals and those individuals who stand to benefit most from BCI technology (individuals with SCI). They highlight the difficulty in directly translating results from healthy subjects to participants with SCI and the challenges that, therefore, arise in providing BCIs to such individuals.

Objective. This study presents inter-subject models of scalp-recorded electroencephalographic (sEEG) event-related potentials (ERPs) using intracranially recorded ERPs from electrocorticography and stereotactic depth electrodes in the hippocampus, generally termed as intracranial EEG (iEEG). Approach. The participants were six patients with medically-intractable epilepsy that underwent temporary placement of intracranial electrode arrays to localize seizure foci. Participants performed one experimental session using a brain–computer interface matrix spelling paradigm controlled by sEEG prior to the iEEG electrode implantation, and one or more identical sessions controlled by iEEG after implantation. All participants were able to achieve excellent spelling accuracy using sEEG, four of the participants achieved roughly equivalent performance in the iEEG sessions, and all participants were significantly above chance accuracy for the iEEG sessions. The sERPs were modeled using a linear combination of iERPs using two different optimization criteria. Main results. The results indicate that sERPs can be accurately estimated from the iERPs for the patients that exhibited stable ERPs over the respective sessions, and that the transformed iERPs can be accurately classified with an sERP-derived classifier. Significance. The resulting models provide a new empirical representation of the formation and distribution of sERPs from underlying composite iERPs. These new insights provide a better understanding of ERP relationships and can potentially lead to the development of more robust signal processing methods for noninvasive EEG applications.

Objective. EEG artifacts of non-neural origin can be separated from neural signals by independent component analysis (ICA). It is unclear (1) how robustly recently proposed artifact classifiers transfer to novel users, novel paradigms or changed electrode setups, and (2) how artifact cleaning by a machine learning classifier impacts the performance of brain–computer interfaces (BCIs). Approach. Addressing (1), the robustness of different strategies with respect to the transfer between paradigms and electrode setups of a recently proposed classifier is investigated on offline data from 35 users and 3 EEG paradigms, which contain 6303 expert-labeled components from two ICA and preprocessing variants. Addressing (2), the effect of artifact removal on single-trial BCI classification is estimated on BCI trials from 101 users and 3 paradigms. Main results. We show that (1) the proposed artifact classifier generalizes to completely different EEG paradigms. To obtain similar results under massively reduced electrode setups, a proposed novel strategy improves artifact classification. Addressing (2), ICA artifact cleaning has little influence on average BCI performance when analyzed by state-of-the-art BCI methods. When slow motor-related features are exploited, performance varies strongly between individuals, as artifacts may obstruct relevant neural activity or are inadvertently used for BCI control. Significance. Robustness of the proposed strategies can be reproduced by EEG practitioners as the method is made available as an EEGLAB plug-in.

Objective. Brain–computer interfaces (BCIs) based on electroencephalography (EEG) have been shown to accurately detect mental activities, but the acquisition of high levels of control require extensive user training. Furthermore, EEG has low signal-to-noise ratio and low spatial resolution. The objective of the present study was to compare the accuracy between two types of BCIs during the first recording session. EEG and tripolar concentric ring electrode (TCRE) EEG (tEEG) brain signals were recorded and used to control one-dimensional cursor movements. Approach. Eight human subjects were asked to imagine either 'left' or 'right' hand movement during one recording session to control the computer cursor using TCRE and disc electrodes. Main results. The obtained results show a significant improvement in accuracies using TCREs (44%–100%) compared to disc electrodes (30%–86%). Significance. This study developed the first tEEG-based BCI system for real-time one-dimensional cursor movements and showed high accuracies with little training.

Objective. Although brain–computer interfaces (BCIs) can be used in several different ways to restore communication, communicative BCI has not approached the rate or efficiency of natural human speech. Electrocorticography (ECoG) has precise spatiotemporal resolution that enables recording of brain activity distributed over a wide area of cortex, such as during speech production. In this study, we sought to decode elements of speech production using ECoG. Approach. We investigated words that contain the entire set of phonemes in the general American accent using ECoG with four subjects. Using a linear classifier, we evaluated the degree to which individual phonemes within each word could be correctly identified from cortical signal. Main results. We classified phonemes with up to 36% accuracy when classifying all phonemes and up to 63% accuracy for a single phoneme. Further, misclassified phonemes follow articulation organization described in phonology literature, aiding classification of whole words. Precise temporal alignment to phoneme onset was crucial for classification success. Significance. We identified specific spatiotemporal features that aid classification, which could guide future applications. Word identification was equivalent to information transfer rates as high as 3.0 bits s⁻¹ (33.6 words min⁻¹), supporting pursuit of speech articulation for BCI control.

Objective. Detection of motor imagery of hand/arm has been extensively studied for stroke rehabilitation. This paper firstly investigates the detection of motor imagery of swallow (MI-SW) and motor imagery of tongue protrusion (MI-Ton) in an attempt to find a novel solution for post-stroke dysphagia rehabilitation. Detection of MI-SW from a simple yet relevant modality such as MI-Ton is then investigated, motivated by the similarity in activation patterns between tongue movements and swallowing and there being fewer movement artifacts in performing tongue movements compared to swallowing. Approach. Novel features were extracted based on the coefficients of the dual-tree complex wavelet transform to build multiple training models for detecting MI-SW. The session-to-session classification accuracy was boosted by adaptively selecting the training model to maximize the ratio of between-classes distances versus within-class distances, using features of training and evaluation data. Main results. Our proposed method yielded averaged cross-validation (CV) classification accuracies of 70.89% and 73.79% for MI-SW and MI-Ton for ten healthy subjects, which are significantly better than the results from existing methods. In addition, averaged CV accuracies of 66.40% and 70.24% for MI-SW and MI-Ton were obtained for one stroke patient, demonstrating the detectability of MI-SW and MI-Ton from the idle state. Furthermore, averaged session-to-session classification accuracies of 72.08% and 70% were achieved for ten healthy subjects and one stroke patient using the MI-Ton model. Significance. These results and the subjectwise strong correlations in classification accuracies between MI-SW and MI-Ton demonstrated the feasibility of detecting MI-SW from MI-Ton models.

Objective. Effective motor imagery performance, seen as strong suppression of the sensorimotor rhythm, is the key element in motor imagery therapy. Therefore, optimization of methods to classify whether the subject is performing the imagery task is a prerequisite. An optimal classification method should have high performance accuracy and use a small number of channels. We investigated the additional benefit of the common spatial pattern filtering (CSP) to a linear discriminant analysis (LDA) classifier, for different channel configurations. Methods. Ten hemispheric acute stroke patients and 11 healthy subjects were included. EEGs were recorded using 60 channels. The classifier was trained with a motor execution task. For both healthy controls and patients, analysis of recordings was initially limited to 3 and 11 electrodes recording from the motor cortex area, and later repeated using 45 electrodes. Results. No significant improvement on the addition of CSP to LDA was found (in both cases, the area under the receiving operating characteristic (AU-ROC) ≈0.70 (acceptable)). We then repeated the LDA+CSP method on recordings of 45 electrodes, since the use of imagery neuronal circuits may well extend beyond the motor area. AU-ROC rose to 0.90, but no virtual 'most responsible' electrode was observed. Finally, in mild-to-moderate stroke patients we could successfully use the EEG data recorded from the healthy hemisphere to train the classifier (AU-ROC ≈ 0.70). Significance. Including only the channels on the unaffected motor cortex is sufficient to train a classifier.

Objectives. Computational methods are increasingly used to optimize transcranial direct current stimulation (tDCS) dose strategies and yet complexities of existing approaches limit their clinical access. Since predictive modelling indicates the relevance of subject/pathology based data and hence the need for subject specific modelling, the incremental clinical value of increasingly complex modelling methods must be balanced against the computational and clinical time and costs. For example, the incorporation of multiple tissue layers and measured diffusion tensor (DTI) based conductivity estimates increase model precision but at the cost of clinical and computational resources. Costs related to such complexities aggregate when considering individual optimization and the myriad of potential montages. Here, rather than considering if additional details change current-flow prediction, we consider when added complexities influence clinical decisions. Approach. Towards developing quantitative and qualitative metrics of value/cost associated with computational model complexity, we considered field distributions generated by two 4 × 1 high-definition montages (m1 = 4 × 1 HD montage with anode at C3 and m2 = 4 × 1 HD montage with anode at C1) and a single conventional (m3 = C3-Fp2) tDCS electrode montage. We evaluated statistical methods, including residual error (RE) and relative difference measure (RDM), to consider the clinical impact and utility of increased complexities, namely the influence of skull, muscle and brain anisotropic conductivities in a volume conductor model. Main results. Anisotropy modulated current-flow in a montage and region dependent manner. However, significant statistical changes, produced within montage by anisotropy, did not change qualitative peak and topographic comparisons across montages. Thus for the examples analysed, clinical decision on which dose to select would not be altered by the omission of anisotropic brain conductivity. Significance. Results illustrate the need to rationally balance the role of model complexity, such as anisotropy in detailed current flow analysis versus value in clinical dose design. However, when extending our analysis to include axonal polarization, the results provide presumably clinically meaningful information. Hence the importance of model complexity may be more relevant with cellular level predictions of neuromodulation.

Objective. While brain–computer interfaces (BCIs) for communication have reached considerable technical maturity, there is still a great need for state-of-the-art evaluation by the end-users outside laboratory environments. To achieve this primary objective, it is necessary to augment a BCI with a series of components that allow end-users to type text effectively. Approach. This work presents the clinical evaluation of a motor imagery (MI) BCI text-speller, called BrainTree, by six severely disabled end-users and ten able-bodied users. Additionally, we define a generic model of code-based BCI applications, which serves as an analytical tool for evaluation and design. Main results. We show that all users achieved remarkable usability and efficiency outcomes in spelling. Furthermore, our model-based analysis highlights the added value of human–computer interaction techniques and hybrid BCI error-handling mechanisms, and reveals the effects of BCI performances on usability and efficiency in code-based applications. Significance. This study demonstrates the usability potential of code-based MI spellers, with BrainTree being the first to be evaluated by a substantial number of end-users, establishing them as a viable, competitive alternative to other popular BCI spellers. Another major outcome of our model-based analysis is the derivation of a 80% minimum command accuracy requirement for successful code-based application control, revising upwards previous estimates attempted in the literature.

Objective. Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain–computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. Approach. Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. Main results. The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). Significance. The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

Objective. A fundamental issue in EEG event-related potentials (ERPs) studies is the amount of data required to have an accurate ERP model. This also impacts the time required to train a classifier for a brain–computer interface (BCI). This issue is mainly due to the poor signal-to-noise ratio and the large fluctuations of the EEG caused by several sources of variability. One of these sources is directly related to the experimental protocol or application designed, and may affect the amplitude or latency of ERPs. This usually prevents BCI classifiers from generalizing among different experimental protocols. In this paper, we analyze the effect of the amplitude and the latency variations among different experimental protocols based on the same type of ERP. Approach. We present a method to analyze and compensate for the latency variations in BCI applications. The algorithm has been tested on two widely used ERPs (P300 and observation error potentials), in three experimental protocols in each case. We report the ERP analysis and single-trial classification. Main results. The results obtained show that the designed experimental protocols significantly affect the latency of the recorded potentials but not the amplitudes. Significance. These results show how the use of latency-corrected data can be used to generalize the BCIs, reducing the calibration time when facing a new experimental protocol.

Objective. The initiation of an action potential by extracellular stimulation occurs after local depolarization of the neuronal membrane above threshold. Although the technique shows remarkable clinical success, the site of action and the relevant stimulation parameters are not completely understood. Approach. Here we identify the site of action potential initiation in rabbit retinal ganglion cells (RGCs) interfaced to an array of extracellular capacitive stimulation electrodes. We determine which feature of the extracellular potential governs action potential initiation by simultaneous stimulation and recording RGCs interfaced in epiretinal configuration. Stimulation electrodes were combined to areas of different size and were presented at different positions with respect to the RGC. Main results. Based on stimulation by electrodes beneath the RGC soma and simultaneous sub-millisecond latency measurement we infer axonal initiation at the site of maximal curvature of the extracellular potential. Stimulation by electrodes at different positions along the axon reveals a nearly constant threshold current density except for a narrow region close to the cell soma. These findings are explained by the concept of the activating function modified to consider a region of lower excitability close to the cell soma. Significance. We present a framework how to estimate the site of action potential initiation and the stimulus required to cross threshold in neurons tightly interfaced to capacitive stimulation electrodes. Our results underscore the necessity of rigorous electrical characterization of the stimulation electrodes and of the interfaced neural tissue.

Objective. This study describes results of primary afferent neural microstimulation experiments using microelectrode arrays implanted chronically in the lumbar dorsal root ganglia (DRG) of four cats. The goal was to test the stability and selectivity of these microelectrode arrays as a potential interface for restoration of somatosensory feedback after damage to the nervous system such as amputation. Approach. A five-contact nerve-cuff electrode implanted on the sciatic nerve was used to record the antidromic compound action potential response to DRG microstimulation (2–15 µA biphasic pulses, 200 µs cathodal pulse width), and the threshold for eliciting a response was tracked over time. Recorded responses were segregated based on conduction velocity to determine thresholds for recruiting Group I and Group II/Aβ primary afferent fibers. Main results. Thresholds were initially low (5.1 ± 2.3 µA for Group I and 6.3 ± 2.0 µA for Group II/Aβ) and increased over time. Additionally the number of electrodes with thresholds less than or equal to 15 µA decreased over time. Approximately 12% of tested electrodes continued to elicit responses at 15 µA up to 26 weeks after implantation. Higher stimulation intensities (up to 30 µA) were tested in one cat at 23 weeks post-implantation yielding responses on over 20 additional electrodes. Within the first six weeks after implantation, approximately equal numbers of electrodes elicited only Group I or Group II/Aβ responses at threshold, but the relative proportion of Group II/Aβ responses decreased over time. Significance. These results suggest that it is possible to activate Group I or Group II/Aβ primary afferent fibers in isolation with penetrating microelectrode arrays implanted in the DRG, and that those responses can be elicited up to 26 weeks after implantation, although it may be difficult to achieve a consistent response day-to-day with currently available electrode technology. The DRG are compelling targets for sensory neuroprostheses with potential to achieve recruitment of a range of sensory fiber types over multiple months after implantation.

Objective. In a previous study, we presented a low-cost, small and wireless EEG system enabling the recording of single-trial P300 amplitudes in a truly mobile, outdoor walking condition (Debener et al (2012 Psychophysiology49 1449–53)). Small and wireless mobile EEG systems have substantial practical advantages as they allow for brain activity recordings in natural environments, but these systems may compromise the EEG signal quality. In this study, we aim to evaluate the EEG signal quality that can be obtained with the mobile system. Approach. We compared our mobile 14-channel EEG system with a state-of-the-art wired laboratory EEG system in a popular brain–computer interface (BCI) application. N = 13 individuals repeatedly performed a 6 × 6 matrix P300 spelling task. Between conditions, only the amplifier was changed, while electrode placement and electrode preparation, recording conditions, experimental stimulation and signal processing were identical. Main results. Analysis of training and testing accuracies and information transfer rate (ITR) revealed that the wireless mobile EEG amplifier performed as good as the wired laboratory EEG system. A very high correlation for testing ITR between both amplifiers was evident (r = 0.92). Moreover the P300 topographies and amplitudes were very similar for both devices, as reflected by high degrees of association (r > = 0.77). Significance. We conclude that efficient P300 spelling with a small, lightweight and quick to set up mobile EEG amplifier is possible. This technology facilitates the transfer of BCI applications from the laboratory to natural daily life environments, one of the key challenges in current BCI research.

Objective. Many serious concerns exist in the long-term stability of brain–machine interfaces (BMIs) based on spike signals (single unit activity, SUA; multi unit activity, MUA). Some studies showed local field potentials (LFPs) could offer a stable decoding performance. However, the decoding stability of LFPs was examined only when high quality spike signals were recorded. Here we aim to examine the long-term decoding stability of LFPs over a larger time scale when the quality of spike signals was from good to poor or even no spike was recorded. Approach. Neural signals were collected from motor cortex of three monkeys via silicon arrays over 230, 290 and 690 days post-implantation when they performed 2D center out task. To compare long-term stability between LFPs and spike signals, we examined them in neural signals characteristics, directional tuning properties and offline decoding performance, respectively. Main results. We observed slow decreasing trends in the number of LFP channels recorded and mean LFP power in different frequency bands when spike signals quality decayed over time. The number of significantly directional tuning LFP channels decreased more slowly than that of tuning SUA and MUA. The variable preferred directions for the same signal features across sessions indicated non-stationarity of neural activity. We also found that LFPs achieved better decoding performance than SUA and MUA in retrained decoder when the quality of spike signals seriously decayed. Especially, when no spike was recorded in one monkey after 671 days post-implantation, LFPs still provided some kinematic information. In addition, LFPs outperformed MUA in long-term decoding stability in a static decoder. Significance. Our results suggested that LFPs were more durable and could provide better decoding performance when spike signals quality seriously decayed. It might be due to their resistance to recording degradation and their high redundancy among channels.

Objective. The objective of this study is to find components that might be related to phoneme representation in the brain and to discriminate EEG responses for each speech sound on a trial basis. Approach. We used multivariate empirical mode decomposition (MEMD) and common spatial pattern for feature extraction. We chose three vowel stimuli, /a/, /i/ and /u/, based on previous findings, such that the brain can detect change in formant frequency (F2) of vowels. EEG activity was recorded from seven native Korean speakers at Gwangju Institute of Science and Technology. We applied MEMD over EEG channels to extract speech-related brain signal sources, and looked for the intrinsic mode functions which were dominant in the alpha bands. After the MEMD procedure, we applied the common spatial pattern algorithm for enhancing the classification performance, and used linear discriminant analysis (LDA) as a classifier. Main results. The brain responses to the three vowels could be classified as one of the learned phonemes on a single-trial basis with our approach. Significance. The results of our study show that brain responses to vowels can be classified for single trials using MEMD and LDA. This approach may not only become a useful tool for the brain–computer interface but it could also be used for discriminating the neural correlates of categorical speech perception.

Objective. Hypertension is the largest threat to patient health and a burden to health care systems. Despite various options, 30% of patients do not respond sufficiently to medical treatment. Mechanoreceptors in the aortic arch relay blood pressure (BP) levels through vagal nerve (VN) fibers to the brainstem and trigger the baroreflex, lowering the BP. Selective electrical stimulation of these nerve fibers reduced BP in rats. However, there is no technique described to localize and stimulate these fibers inside the VN without inadvertent stimulation of non-baroreceptive fibers causing side effects like bradycardia and bradypnea. Approach. We present a novel method for selective VN stimulation to reduce BP without the aforementioned side effects. Baroreceptor compound activity of rat VN (n = 5) was localized using a multichannel cuff electrode, true tripolar recording and a coherent averaging algorithm triggered by BP or electrocardiogram. Main results. Tripolar stimulation over electrodes near the barofibers reduced the BP without triggering significant bradycardia and bradypnea. The BP drop was adjusted to 60% of the initial value by varying the stimulation pulse width and duration, and lasted up to five times longer than the stimulation. Significance. The presented method is robust to impedance changes, independent of the electrode's relative position, does not compromise the nerve and can run on implantable, ultra-low power signal processors.

Objective. The objective of our current study was to look for the EEG correlates that can reveal the engaged state of the brain while undertaking cognitive tasks. Specifically, we aimed to identify EEG features that could detect audio distraction during simulated driving. Approach. Time varying autoregressive (TVAR) analysis using Kalman smoother was carried out on short time epochs of EEG data collected from participants as they undertook two simulated driving tasks. TVAR coefficients were then used to construct all pole model enabling the identification of EEG features that could differentiate normal driving from audio distracted driving. Main results. Pole analysis of the TVAR model led to the visualization of event related synchronization/desynchronization (ERS/ERD) patterns in the form of pole displacements in pole plots of the temporal EEG channels in the z plane enabling the differentiation of the two driving conditions. ERS in the EEG data has been demonstrated during audio distraction as an associated phenomenon. Significance. Visualizing the ERD/ERS phenomenon in terms of pole displacement is a novel approach. Although ERS/ERD has previously been demonstrated as reliable when applied to motor related tasks, it is believed to be the first time that it has been applied to investigate human cognitive phenomena such as attention and distraction. Results confirmed that distracted/non-distracted driving states can be identified using this approach supporting its applicability to cognition research.

Objective. Carbon nanotubes (CNTs) are attractive for use in peripheral nerve interfaces because of their unique combination of strength, flexibility, electrical conductivity and nanoscale surface texture. Here we investigated the growth of motor neurons on thin films of horizontally aligned CNTs (HACNTs). Approach. We cultured primary embryonic rat motor neurons on HACNTs and performed statistical analysis of the length and orientation of neurites. We next presented motor neurons with substrates of alternating stripes of HACNTs and SiO₂. Main results. The neurons survived on HACNT substrates for up to eight days, which was the full duration of our experiments. Statistical analysis of the length and orientation of neurites indicated that the longest neurites on HACNTs tended to align with the CNT direction, although the average neurite length was similar between HACNTs and glass control substrates. We observed that when motor neurons were presented with alternating stripes of HACNTs and SiO₂, the proportion of neurons on HACNTs increases over time, suggesting that neurons selectively migrate toward and adhere to the HACNT surface. Significance. The behavior of motor neurons on CNTs has not been previously investigated, and we show that aligned CNTs could provide a viable interface material to motor neurons. Combined with emerging techniques to build complex hierarchical structures of CNTs, our results suggest that organised CNTs could be incorporated into nerve grafts that use physical and electrical cues to guide regenerating axons.