Journal of Neural Engineering

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective. In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain–computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results. Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About of the studies used convolutional neural networks (CNNs), while used recurrent neural networks (RNNs), most often with a total of 3–10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

Objective. Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain–computer interfaces, BCI’s), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? Approach. A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. Main results. For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. Significance. This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

Objective. Most current electroencephalography (EEG)-based brain–computer interfaces (BCIs) are based on machine learning algorithms. There is a large diversity of classifier types that are used in this field, as described in our 2007 review paper. Now, approximately ten years after this review publication, many new algorithms have been developed and tested to classify EEG signals in BCIs. The time is therefore ripe for an updated review of EEG classification algorithms for BCIs. Approach. We surveyed the BCI and machine learning literature from 2007 to 2017 to identify the new classification approaches that have been investigated to design BCIs. We synthesize these studies in order to present such algorithms, to report how they were used for BCIs, what were the outcomes, and to identify their pros and cons. Main results. We found that the recently designed classification algorithms for EEG-based BCIs can be divided into four main categories: adaptive classifiers, matrix and tensor classifiers, transfer learning and deep learning, plus a few other miscellaneous classifiers. Among these, adaptive classifiers were demonstrated to be generally superior to static ones, even with unsupervised adaptation. Transfer learning can also prove useful although the benefits of transfer learning remain unpredictable. Riemannian geometry-based methods have reached state-of-the-art performances on multiple BCI problems and deserve to be explored more thoroughly, along with tensor-based methods. Shrinkage linear discriminant analysis and random forests also appear particularly useful for small training samples settings. On the other hand, deep learning methods have not yet shown convincing improvement over state-of-the-art BCI methods. Significance. This paper provides a comprehensive overview of the modern classification algorithms used in EEG-based BCIs, presents the principles of these methods and guidelines on when and how to use them. It also identifies a number of challenges to further advance EEG classification in BCI.

Objective. Advanced robotic lower limb prostheses are mainly controlled autonomously. Although the existing control can assist cyclic movements during locomotion of amputee users, the function of these modern devices is still limited due to the lack of neuromuscular control (i.e. control based on human efferent neural signals from the central nervous system to peripheral muscles for movement production). Neuromuscular control signals can be recorded from muscles, called electromyographic (EMG) or myoelectric signals. In fact, using EMG signals for robotic lower limb prostheses control has been an emerging research topic in the field for the past decade to address novel prosthesis functionality and adaptability to different environments and task contexts. The objective of this paper is to review robotic lower limb Prosthesis control via EMG signals recorded from residual muscles in individuals with lower limb amputations. Approach. We performed a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses. Main results. This review highlights the promise of EMG control for enabling new functionalities in robotic lower limb prostheses, as well as the existing challenges, knowledge gaps, and opportunities on this research topic from human motor control and clinical practice perspectives. Significance. This review may guide the future collaborations among researchers in neuromechanics, neural engineering, assistive technologies, and amputee clinics in order to build and translate true bionic lower limbs to individuals with lower limb amputations for improved motor function.

Objective. Powered exoskeletons have been used to help persons with gait impairment regain some walking ability. However, little is known about its impact on neuromuscular coordination in persons with stroke. The objective of this study is to investigate how a powered exoskeleton could affect the neuromuscular coordination of persons with post-stroke hemiparesis.

Approach. Eleven able-bodied subjects and ten stroke subjects participated in a single-visit treadmill walking assessment, in which their motion and lower-limb muscle activities were captured. By comparing spatiotemporal parameters, kinematics, and muscle synergy pattern between two groups, we characterized the normal gait pattern and the post-stroke motor deficits. Five eligible stroke subjects received exoskeleton-assisted gait trainings and walking assessments were conducted pre-intervention (Pre) and post-intervention (Post), without (WO) and with (WT) the exoskeleton. We compared their gait performance between (a) Pre and Post to investigate the effect of exoskeleton-assisted gait training and, (b) WO and WT the exoskeleton to investigate the effect of exoskeleton wearing on stroke subjects.

Main results. While four distinct motor modules were needed to describe lower-extremity activities during stead-speed walking among able-bodied subjects, three modules were sufficient for the paretic leg from the stroke subjects. Muscle coordination complexity, module composition and activation timing were preserved after the training, indicating the intervention did not significantly change the neuromuscular coordination. In contrast, walking WT the exoskeleton altered the stroke subjects’ synergy pattern, especially on the paretic side. The changes were dominated by the activation profile modulation towards the normal pattern observed from the able-bodied group.

Significance. This study gave us some critical insight into how a powered exoskeleton affects the stroke subjects’ neuromuscular coordination during gait and demonstrated the potential to use muscle synergy as a method to evaluate the effect of the exoskeleton training.

This study was registered at ClinicalTrials.gov (identifier: NCT03057652).

Objective. Intracortical microelectrode arrays (MEA) can be used as part of a brain–machine interface system to provide sensory feedback control of an artificial limb to assist persons with tetraplegia. Variability in functionality of electrodes has been reported but few studies in humans have examined the impact of chronic brain tissue responses revealed postmortem on electrode performance in vivo. Approach. In a tetraplegic man, recording MEAs were implanted into the anterior intraparietal area and Brodmann’s area 5 (BA5) of the posterior parietal cortex and a recording and stimulation array was implanted in BA1 of the primary somatosensory cortex (S1). The participant expired from unrelated causes seven months after MEA implantation. The underlying tissue of two of the three devices was processed for histology and electrophysiological recordings were assessed. Main results. Recordings of neuronal activity were obtained from all three MEAs despite meningeal encapsulation. However, the S1 array had a greater encapsulation, yielded lower signal quality than the other arrays and failed to elicit somatosensory percepts with electrical stimulation. Histological examination of tissues underlying S1 and BA5 implant sites revealed localized leptomeningeal proliferation and fibrosis, lymphocytic infiltrates, astrogliosis, and foreign body reaction around the electrodes. The BA5 recording site showed focal cerebral microhemorrhages and leptomeningeal vascular ectasia. The S1 site showed focal tissue damage including vascular recanalization, neuronal loss, and extensive subcortical white matter necrosis. The tissue response at the S1 site included hemorrhagic-induced injury suggesting a likely mechanism for reduced function of the S1 implant. Significance. Our findings are similar to those from animal studies with chronic intracortical implants and suggest that vascular disruption and microhemorrhage during device implantation are important contributors to overall array and individual electrode performance and should be a topic for future device development to mitigate tissue responses. Neurosurgical considerations are also discussed.

Stroke is one of the leading causes of long-term disability among adults and contributes to major socio-economic burden globally. Stroke frequently results in multifaceted impairments including motor, cognitive and emotion deficits. In recent years, brain–computer interface (BCI)-based therapy has shown promising results for post-stroke motor rehabilitation. In spite of the success received by BCI-based interventions in the motor domain, non-motor impairments are yet to receive similar attention in research and clinical settings. Some preliminary encouraging results in post-stroke cognitive rehabilitation using BCI seem to suggest that it may also hold potential for treating non-motor deficits such as cognitive and emotion impairments. Moreover, past studies have shown an intricate relationship between motor, cognitive and emotion functions which might influence the overall post-stroke rehabilitation outcome. A number of studies highlight the inability of current treatment protocols to account for the implicit interplay between motor, cognitive and emotion functions. This indicates the necessity to explore an all-inclusive treatment plan targeting the synergistic influence of these standalone interventions. This approach may lead to better overall recovery than treating the individual deficits in isolation. In this paper, we review the recent advances in BCI-based post-stroke motor rehabilitation and highlight the potential for the use of BCI systems beyond the motor domain, in particular, in improving cognition and emotion of stroke patients. Building on the current results and findings of studies in individual domains, we next discuss the possibility of a holistic BCI system for motor, cognitive and affect rehabilitation which may synergistically promote restorative neuroplasticity. Such a system would provide an all-encompassing rehabilitation platform, leading to overarching clinical outcomes and transfer of these outcomes to a better quality of living. This is one of the first works to analyse the possibility of targeting cross-domain influence of post-stroke functional recovery enabled by BCI-based rehabilitation.

Bioelectronic medicine (BM) is an emerging new approach for developing novel neuromodulation therapies for pathologies that have been previously treated with pharmacological approaches. In this review, we will focus on the neuromodulation of autonomic nervous system (ANS) activity with implantable devices, a field of BM that has already demonstrated the ability to treat a variety of conditions, from inflammation to metabolic and cognitive disorders. Recent discoveries about immune responses to ANS stimulation are the laying foundation for a new field holding great potential for medical advancement and therapies and involving an increasing number of research groups around the world, with funding from international public agencies and private investors. Here, we summarize the current achievements and future perspectives for clinical applications of neural decoding and stimulation of the ANS. First, we present the main clinical results achieved so far by different BM approaches and discuss the challenges encountered in fully exploiting the potential of neuromodulatory strategies. Then, we present current preclinical studies aimed at overcoming the present limitations by looking for optimal anatomical targets, developing novel neural interface technology, and conceiving more efficient signal processing strategies. Finally, we explore the prospects for translating these advancements into clinical practice.

Objective. Due to low spatial resolution and poor signal-to-noise ratio of electroencephalogram (EEG), high accuracy classifications still suffer from lots of obstacles in the context of motor imagery (MI)-based brain-machine interface (BMI) systems. Particularly, it is extremely challenging to decode multiclass MI EEG from the same upper limb. This research proposes a novel feature learning approach to address the classification problem of 6-class MI tasks, including imaginary elbow flexion/extension, wrist supination/pronation, and hand close/open within the unilateral upper limb. Approach. Instead of the traditional common spatial pattern (CSP) or filter-bank CSP (FBCSP) manner, the Riemannian geometry (RG) framework involving Riemannian distance and Riemannian mean was directly adopted to extract tangent space (TS) features from spatial covariance matrices of the MI EEG trials. Subsequently, to reduce the dimensionality of the TS features, the algorithm of partial least squares regression was applied to obtain more separable and compact feature representations. Main results. The performance of the learned RG feature representations was validated by a linear discriminative analysis and support vector machine classifier, with an average accuracy of 80.50% and 79.70% on EEG dataset collected from 12 participants, respectively. Significance. These results demonstrate that compared with CSP and FBCSP features, the proposed approach can significantly increase the decoding accuracy for multiclass MI tasks from the same upper limb. This approach is promising and could potentially be applied in the context of MI-based BMI control of a robotic arm or a neural prosthesis for motor disabled patients with highly impaired upper limb.

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective. In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain–computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results. Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About of the studies used convolutional neural networks (CNNs), while used recurrent neural networks (RNNs), most often with a total of 3–10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

Objective. Research in the area of transcranial electrical stimulation (TES) often relies on computational models of current flow in the brain. Models are built based on magnetic resonance images (MRI) of the human head to capture detailed individual anatomy. To simulate current flow on an individual, the subject’s MRI is segmented, virtual electrodes are placed on this anatomical model, the volume is tessellated into a mesh, and a finite element model (FEM) is solved numerically to estimate the current flow. Various software tools are available for each of these steps, as well as processing pipelines that connect these tools for automated or semi-automated processing. The goal of the present tool—realistic volumetric-approach to simulate transcranial electric simulation (ROAST)—is to provide an end-to-end pipeline that can automatically process individual heads with realistic volumetric anatomy leveraging open-source software and custom scripts to improve segmentation and execute electrode placement. Approach. ROAST combines the segmentation algorithm of SPM12, a Matlab script for touch-up and automatic electrode placement, the finite element mesher iso2mesh and the solver getDP. We compared its performance with commercial FEM software, and SimNIBS, a well-established open-source modeling pipeline. Main results. The electric fields estimated with ROAST differ little from the results obtained with commercial meshing and FEM solving software. We also do not find large differences between the various automated segmentation methods used by ROAST and SimNIBS. We do find bigger differences when volumetric segmentation are converted into surfaces in SimNIBS. However, evaluation on intracranial recordings from human subjects suggests that ROAST and SimNIBS are not significantly different in predicting field distribution, provided that users have detailed knowledge of SimNIBS. Significance. We hope that the detailed comparisons presented here of various choices in this modeling pipeline can provide guidance for future tool development. We released ROAST as an open-source, easy-to-install and fully-automated pipeline for individualized TES modeling.

Objective. Transcranial magnetic stimulation (TMS) and transcranial electric stimulation (TES) modulate brain activity non-invasively by generating electric fields either by electromagnetic induction or by injecting currents via skin electrodes. Numerical simulations based on anatomically detailed head models of the TMS and TES electric fields can help us to understand and optimize the spatial stimulation pattern in the brain. However, most realistic simulations are still slow, and the role of anatomical fidelity on simulation accuracy has not been evaluated in detail so far. Approach. We present and validate a new implementation of the finite element method (FEM) for TMS and TES that is based on modern algorithms and libraries. We also evaluate the convergence of the simulations and estimate errors stemming from numerical and modelling aspects. Main results. Comparisons with analytical solutions for spherical phantoms validate our new FEM implementation, which is three to six times faster than previous implementations. The convergence results suggest that accurately capturing the tissue geometry in addition to choosing a sufficiently accurate numerical method is of fundamental importance for accurate simulations. Significance. The new implementation allows for a substantial increase in computational efficiency of FEM TMS and TES simulations. This is especially relevant for applications such as the systematic assessment of model uncertainty and the optimization of multi-electrode TES montages. The results of our systematic error analysis allow the user to select the best tradeoff between model resolution and simulation speed for a specific application. The new FEM code is openly available as a part of our open-source software SimNIBS 3.0.

Objective. In recent years, brain–computer interface (BCI) systems based on electroencephalography (EEG) have developed rapidly. However, the decoding of voluntary finger pre-movements from EEG is still a challenge for BCIs. This study aimed to analyze the pre-movement EEG features in time and frequency domains and design an efficient method to decode the movement-related patterns. Approach. In this study, we first investigated the EEG features induced by the intention of left and right finger movements. Specifically, the movement-related cortical potential (MRCP) and event-related desynchronization (ERD) features were extracted using discriminative canonical pattern matching (DCPM) and common spatial patterns (CSP), respectively. Then, the two types of features were classified by two fisher discriminant analysis (FDA) classifiers, respectively. Their decision values were further assembled to facilitate the classification. To verify the validity of the proposed method, a private dataset containing 12 subjects and a public dataset from BCI competition II were used for estimating the classification accuracy. Main results. As a result, for the private dataset, the combination of DCPM and CSP achieved an average accuracy of 80.96%, which was 5.08% higher than the single DCPM method ( p   <  0.01) and 10.23% higher than the single CSP method ( p   <  0.01). Notably, the highest accuracy could achieve 91.5% for the combination method. The test accuracy of dataset IV of BCI competition II was 90%, which was equal to the best result in the existing literature. Significance. The results demonstrate the MRCP and ERD features of pre-movements contain significantly discriminative information, which are complementary to each other, and thereby could be well recognized by the proposed combination method of DCPM and CSP. Therefore, this study provides a promising approach for the decoding of pre-movement EEG patterns, which is significant for the development of BCIs.

Objective. Electroencephalography (EEG) motor imagery classification has been widely used in healthcare applications such as mobile assistive robots and post-stroke rehabilitation. Recently, EEG motor imagery classification methods based on convolutional neural networks (CNNs) have been proposed and have achieved relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when training data is limited. Approach. To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification. Main results. Compared with several state-of-the-art methods, the proposed method achieves an average classification accuracy of 91.57% and 87.6% on two commonly used datasets, which outperforms several state-of-the-art EEG motor imagery classification methods. Significance. The proposed method effectively addresses the issues of existing CNN-based EEG motor imagery classification methods and improves the classification accuracy.

Objective. Electroencephalography (EEG) cleaning has been a longstanding issue in the research community. In recent times, huge leaps have been made in the field, resulting in very promising techniques to address the issue. The most widespread ones rely on a family of mathematical methods known as blind source separation (BSS), ideally capable of separating artefactual signals from the brain originated ones. However, corruption of EEG data still remains a problem, especially in real life scenario where a mixture of artefact components affects the signal and thus correctly choosing the correct cleaning procedure can be non trivial. Our aim is here to evaluate and score the plethora of available BSS-based cleaning methods, providing an overview of their advantages and downsides and of their best field of application. Approach. To address this, we here first characterized and modeled different types of artefact, i.e. arising from muscular or blinking activity as well as from transcranial alternate current stimulation. We then tested and scored several BSS-based cleaning procedures on semi-synthetic datasets corrupted by the previously modeled noise sources. Finally, we built a lifelike dataset affected by many artefactual components. We tested an iterative multistep approach combining different BSS steps, aimed at sequentially removing each specific artefactual component. Main results. We did not find an overall best method, as different scenarios require different approaches. We therefore provided an overview of the performance in terms of both reconstruction accuracy and computational burden of each method in different use cases. Significance. Our work provides insightful guidelines for signal cleaning procedures in the EEG related field.

Objective. Advanced robotic lower limb prostheses are mainly controlled autonomously. Although the existing control can assist cyclic movements during locomotion of amputee users, the function of these modern devices is still limited due to the lack of neuromuscular control (i.e. control based on human efferent neural signals from the central nervous system to peripheral muscles for movement production). Neuromuscular control signals can be recorded from muscles, called electromyographic (EMG) or myoelectric signals. In fact, using EMG signals for robotic lower limb prostheses control has been an emerging research topic in the field for the past decade to address novel prosthesis functionality and adaptability to different environments and task contexts. The objective of this paper is to review robotic lower limb Prosthesis control via EMG signals recorded from residual muscles in individuals with lower limb amputations. Approach. We performed a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses. Main results. This review highlights the promise of EMG control for enabling new functionalities in robotic lower limb prostheses, as well as the existing challenges, knowledge gaps, and opportunities on this research topic from human motor control and clinical practice perspectives. Significance. This review may guide the future collaborations among researchers in neuromechanics, neural engineering, assistive technologies, and amputee clinics in order to build and translate true bionic lower limbs to individuals with lower limb amputations for improved motor function.

Objective. Intracortical microelectrode arrays (MEA) can be used as part of a brain–machine interface system to provide sensory feedback control of an artificial limb to assist persons with tetraplegia. Variability in functionality of electrodes has been reported but few studies in humans have examined the impact of chronic brain tissue responses revealed postmortem on electrode performance in vivo. Approach. In a tetraplegic man, recording MEAs were implanted into the anterior intraparietal area and Brodmann's area 5 (BA5) of the posterior parietal cortex and a recording and stimulation array was implanted in BA1 of the primary somatosensory cortex (S1). The participant expired from unrelated causes seven months after MEA implantation. The underlying tissue of two of the three devices was processed for histology and electrophysiological recordings were assessed. Main results. Recordings of neuronal activity were obtained from all three MEAs despite meningeal encapsulation. However, the S1 array had a greater encapsulation, yielded lower signal quality than the other arrays and failed to elicit somatosensory percepts with electrical stimulation. Histological examination of tissues underlying S1 and BA5 implant sites revealed localized leptomeningeal proliferation and fibrosis, lymphocytic infiltrates, astrogliosis, and foreign body reaction around the electrodes. The BA5 recording site showed focal cerebral microhemorrhages and leptomeningeal vascular ectasia. The S1 site showed focal tissue damage including vascular recanalization, neuronal loss, and extensive subcortical white matter necrosis. The tissue response at the S1 site included hemorrhagic-induced injury suggesting a likely mechanism for reduced function of the S1 implant. Significance. Our findings are similar to those from animal studies with chronic intracortical implants and suggest that vascular disruption and microhemorrhage during device implantation are important contributors to overall array and individual electrode performance and should be a topic for future device development to mitigate tissue responses. Neurosurgical considerations are also discussed.

Objective. Simultaneous electroencephalography-functional magnetic resonance imaging (EEG-fMRI) recordings offer a high spatiotemporal resolution approach to study human brain and understand the underlying mechanisms mediating cognitive and behavioral processes. However, the high susceptibility of EEG to MRI-induced artifacts hinders a broad adaptation of this approach. More specifically, EEG data collected during fMRI acquisition are contaminated with MRI gradients and ballistocardiogram artifacts, in addition to artifacts of physiological origin. There have been several attempts for reducing these artifacts with manual and time-consuming pre-processing, which may result in biasing EEG data due to variations in selecting steps order, parameters, and classification of artifactual independent components. Thus, there is a strong urge to develop a fully automatic and comprehensive pipeline for reducing all major EEG artifacts. In this work, we introduced an open-access toolbox with a fully automatic pipeline for reducing artifacts from EEG data collected simultaneously with fMRI (refer to APPEAR). Approach. The pipeline integrates average template subtraction and independent component analysis to suppress both MRI-related and physiological artifacts. To validate our results, we tested APPEAR on EEG data recorded from healthy control subjects during resting-state (n= 48) and task-based (i.e. event-related-potentials (ERPs); n= 8) paradigms. The chosen gold standard is an expert manual review of the EEG database. Main results. We compared manually and automated corrected EEG data during resting-state using frequency analysis and continuous wavelet transformation and found no significant differences between the two corrections. A comparison between ERP data recorded during a so-called stop-signal task (e.g. amplitude measures and signal-to-noise ratio) also showed no differences between the manually and fully automatic fMRI-EEG-corrected data. Significance. APPEAR offers the first comprehensive open-source toolbox that can speed up advancement of EEG analysis and enhance replication by avoiding experimenters' preferences while allowing for processing large EEG-fMRI cohorts composed of hundreds of subjects with manageable researcher time and effort.

Objective. Optogenetics has emerged as a promising technique for neural prosthetics, especially retinal prostheses, with unprecedented spatiotemporal resolution. Newly discovered opsins with high light sensitivity and fast temporal kinetics can provide sufficient temporal resolution at safe light powers and overcome the limitations of presently used opsins. It is also important to formulate accurate mathematical models for optogenetic retinal prostheses, which can facilitate optimization of photostimulation factors to improve the performance. Approach. A detailed theoretical analysis of optogenetic excitation of model retinal ganglion neurons (RGNs) and hippocampal neurons expressed with already tested opsins for retinal prostheses, namely, ChR2, ReaChR and ChrimsonR, and also with recently discovered potent opsins CsChrimson, bReaChES and ChRmine, was carried out. Main results. Under continuous illumination, ChRmine-expressing RGNs begin to respond at very low irradiances ~10⁻⁴ mW mm⁻², and evoke firing upto ~280 Hz, highest among other opsin-expressing RGNs, at 10⁻² mW mm⁻². Under pulsed illumination at randomized photon fluxes, ChRmine-expressing RGNs respond to changes in pulse to pulse irradiances upto four logs, although very bright pulses >10¹⁴ photons mm⁻² s⁻¹ block firing in these neurons. The minimum irradiance threshold for ChRmine-expressing RGNs is lower by two orders of magnitude, whereas, the first spike latency in ChRmine-expressing RGNs is shorter by an order of magnitude, alongwith stable latency of subsequest spikes compared to others. Further, a good set of photostimulation parameters were determined to achieve high-frequency control with single spike resolution at minimal power. Although ChrimsonR enables spiking upto 100 Hz in RGNs, it requires very high irradiances. ChRmine provides control at light powers that are two orders of magnitude smaller than that required with experimentally studied opsins, while maintaining single spike temporal resolution upto 40 Hz. Significance. The present study highlights the importance of ChRmine as a potential opsin for optogenetic retinal prostheses.

Visual prosthesis devices designed to restore sight to the blind have been under development in the laboratory for several decades. Clinical translation continues to be challenging, due in part to gaps in our understanding of critical parameters such as how phosphenes, the electrically-generated pixels of artificial vision, can be combined to form images. In this review we explore the effects that synchronous and asynchronous electrical stimulation across multiple electrodes have in evoking phosphenes. Understanding how electrical patterns influence phosphene generation to control object binding and perception of visual form is fundamental to creation of a clinically successful prosthesis.

Progress in understanding neuronal interaction and circuit behavior of the central and peripheral nervous system (PNS) strongly relies on the advancement of tools that record and stimulate with high fidelity and specificity. Currently, devices used in exploratory research predominantly utilize cables or tethers to provide pathways for power supply, data communication, stimulus delivery and recording, which constrains the scope and use of such devices. In particular, the tethered connection, mechanical mismatch to surrounding soft tissues and bones frustrate the interface leading to irritation and limitation of motion of the subject, which in the case of fundamental and preclinical studies, impacts naturalistic behaviors of animals and precludes the use in experiments involving social interaction and ethologically relevant three-dimensional environments, limiting the use of current tools to mostly rodents and exclude species such as birds and fish. This review explores the current state-of-the-art in wireless, subdermally implantable tools that quantitively expand capabilities in analysis and perturbation of the central and PNS by removing tethers and externalized features of implantable neuromodulation and recording tools. Specifically, the review explores power harvesting strategies, wireless communication schemes, and soft materials and mechanics that enable the creation of such devices and discuss their capabilities in the context of freely-behaving subjects. Highlights of this class of devices includes wireless battery-free and fully implantable operation with capabilities in cell specific recording, multimodal neural stimulation and electrical, optogenetic and pharmacological neuromodulation capabilities. We conclude with a discussion on translation of such technologies, which promises routes towards broad dissemination.

Bioelectronic medicine (BM) is an emerging new approach for developing novel neuromodulation therapies for pathologies that have been previously treated with pharmacological approaches. In this review, we will focus on the neuromodulation of autonomic nervous system (ANS) activity with implantable devices, a field of BM that has already demonstrated the ability to treat a variety of conditions, from inflammation to metabolic and cognitive disorders. Recent discoveries about immune responses to ANS stimulation are the laying foundation for a new field holding great potential for medical advancement and therapies and involving an increasing number of research groups around the world, with funding from international public agencies and private investors. Here, we summarize the current achievements and future perspectives for clinical applications of neural decoding and stimulation of the ANS. First, we present the main clinical results achieved so far by different BM approaches and discuss the challenges encountered in fully exploiting the potential of neuromodulatory strategies. Then, we present current preclinical studies aimed at overcoming the present limitations by looking for optimal anatomical targets, developing novel neural interface technology, and conceiving more efficient signal processing strategies. Finally, we explore the prospects for translating these advancements into clinical practice.

Many implantable electrode arrays exist for the purpose of stimulating or recording electrical activity in brain, spinal, or peripheral nerve tissue, however most of these devices are constructed from materials that are mechanically rigid. A growing body of evidence suggests that the chronic presence of these rigid probes in the neural tissue causes a significant immune response and glial encapsulation of the probes, which in turn leads to gradual increase in distance between the electrodes and surrounding neurons. In recording electrodes, the consequence is the loss of signal quality and, therefore, the inability to collect electrophysiological recordings long term. In stimulation electrodes, higher current injection is required to achieve a comparable response which can lead to tissue and electrode damage. To minimize the impact of the immune response, flexible neural probes constructed with softer materials have been developed. These flexible probes, however, are often not strong enough to be inserted on their own into the tissue, and instead fail via mechanical buckling of the shank under the force of insertion. Several strategies have been developed to allow the insertion of flexible probes while minimizing tissue damage. It is critical to keep these strategies in mind during probe design in order to ensure successful surgical placement. In this review, existing insertion strategies will be presented and evaluated with respect to surgical difficulty, immune response, ability to reach the target tissue, and overall limitations of the technique. Overall, the majority of these insertion techniques have only been evaluated for the insertion of a single probe and do not quantify the accuracy of probe placement. More work needs to be performed to evaluate and optimize insertion methods for accurate placement of devices and for devices with multiple probes.

Objective. Advanced robotic lower limb prostheses are mainly controlled autonomously. Although the existing control can assist cyclic movements during locomotion of amputee users, the function of these modern devices is still limited due to the lack of neuromuscular control (i.e. control based on human efferent neural signals from the central nervous system to peripheral muscles for movement production). Neuromuscular control signals can be recorded from muscles, called electromyographic (EMG) or myoelectric signals. In fact, using EMG signals for robotic lower limb prostheses control has been an emerging research topic in the field for the past decade to address novel prosthesis functionality and adaptability to different environments and task contexts. The objective of this paper is to review robotic lower limb Prosthesis control via EMG signals recorded from residual muscles in individuals with lower limb amputations. Approach. We performed a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses. Main results. This review highlights the promise of EMG control for enabling new functionalities in robotic lower limb prostheses, as well as the existing challenges, knowledge gaps, and opportunities on this research topic from human motor control and clinical practice perspectives. Significance. This review may guide the future collaborations among researchers in neuromechanics, neural engineering, assistive technologies, and amputee clinics in order to build and translate true bionic lower limbs to individuals with lower limb amputations for improved motor function.

Objective. Intracellular patch-clamp electrophysiology, one of the most ubiquitous, high-fidelity techniques in biophysics, remains laborious and low-throughput. While previous efforts have succeeded at automating some steps of the technique, here we demonstrate a robotic ‘PatcherBot’ system that can perform many patch-clamp recordings sequentially, fully unattended. Approach. Comprehensive automation is accomplished by outfitting the robot with machine vision, and cleaning pipettes instead of manually exchanging them. Main results. the PatcherBot can obtain data at a rate of 16 cells per hour and work with no human intervention for up to 3 h. We demonstrate the broad applicability and scalability of this system by performing hundreds of recordings in tissue culture cells and mouse brain slices with no human supervision. Using the PatcherBot, we also discovered that pipette cleaning can be improved by a factor of three. Significance. The system is potentially transformative for applications that depend on many high-quality measurements of single cells, such as drug screening, protein functional characterization, and multimodal cell type investigations.

Objective. Most brain–computer interfaces (BCIs) based on functional near-infrared spectroscopy (fNIRS) require that users perform mental tasks such as motor imagery, mental arithmetic, or music imagery to convey a message or to answer simple yes or no questions. These cognitive tasks usually have no direct association with the communicative intent, which makes them difficult for users to perform. Approach. In this paper, a 3-class intuitive BCI is presented which enables users to directly answer yes or no questions by covertly rehearsing the word ‘yes’ or ‘no’ for 15 s. The BCI also admits an equivalent duration of unconstrained rest which constitutes the third discernable task. Twelve participants each completed one offline block and six online blocks over the course of two sessions. The mean value of the change in oxygenated hemoglobin concentration during a trial was calculated for each channel and used to train a regularized linear discriminant analysis (RLDA) classifier. Main results. By the final online block, nine out of 12 participants were performing above chance ( p  <  0.001 using the binomial cumulative distribution), with a 3-class accuracy of 83.8%  ±  9.4%. Even when considering all participants, the average online 3-class accuracy over the last three blocks was 64.1 %  ±  20.6%, with only three participants scoring below chance ( p  <  0.001). For most participants, channels in the left temporal and temporoparietal cortex provided the most discriminative information. Significance. To our knowledge, this is the first report of an online 3-class imagined speech BCI. Our findings suggest that imagined speech can be used as a reliable activation task for selected users for development of more intuitive BCIs for communication.

Objective. Optical techniques for recording and manipulating neural activity have traditionally been constrained to superficial brain regions due to light scattering. New techniques are needed to extend optical access to large 3D volumes in deep brain areas, while retaining local connectivity. Approach. We have developed a method to implant bundles of hundreds or thousands of optical microfibers, each with a diameter of 8 μm. During insertion, each fiber moves independently, following a path of least resistance. The fibers achieve near total internal reflection, enabling optically interfacing with the tissue near each fiber aperture. Main results. At a depth of 3 mm, histology shows fibers consistently splay over 1 mm in diameter throughout the target region. Immunohistochemical staining after chronic implants reveals neurons in close proximity to the fiber tips. Models of photon fluence indicate that fibers can be used as a stimulation light source to precisely activate distinct patterns of neurons by illuminating a subset of fibers in the bundle. By recording fluorescent beads diffusing in water, we demonstrate the recording capability of the fibers. Significance. Our histology, modeling and fluorescent bead recordings suggest that the optical microfibers may provide a minimally invasive, stable, bidirectional interface for recording or stimulating genetic probes in deep brain regions—a hyper-localized form of fiber photometry.

Objective. Stereotactic frame systems are the gold-standard for stereotactic surgeries, such as implantation of deep brain stimulation (DBS) devices for treatment of medically resistant neurologic and psychiatric disorders. However, frame-based systems require that the patient is awake with a stereotactic frame affixed to their head for the duration of the surgical planning and implantation of the DBS electrodes. While frameless systems are increasingly available, a reusable re-attachable frame system provides unique benefits. As such, we created a novel reusable MRI-compatible stereotactic frame system that maintains clinical accuracy through the detachment and reattachment of its stereotactic devices used for MRI-guided neuronavigation. Approach. We designed a reusable arc-centered frame system that includes MRI-compatible anchoring skull screws for detachment and re-attachment of its stereotactic devices. We validated the stability and accuracy of our system through phantom, in vivo mock-human porcine DBS-model and human cadaver testing. Main results. Phantom testing achieved a root mean square error (RMSE) of 0.94  ±  0.23 mm between the ground truth and the frame-targeted coordinates; and achieved an RMSE of 1.11  ±  0.40 mm and 1.33  ±  0.38 mm between the ground truth and the CT- and MRI-targeted coordinates, respectively. In vivo and cadaver testing achieved a combined 3D Euclidean localization error of 1.85  ±  0.36 mm ( p  <  0.03) between the pre-operative MRI-guided placement and the post-operative CT-guided confirmation of the DBS electrode. Significance. Our system demonstrated consistent clinical accuracy that is comparable to conventional frame and frameless stereotactic systems. Our frame system is the first to demonstrate accurate relocation of stereotactic frame devices during in vivo MRI-guided DBS surgical procedures. As such, this reusable and re-attachable MRI-compatible system is expected to enable more complex, chronic neuromodulation experiments, and lead to a clinically available re-attachable frame that is expected to decrease patient discomfort and costs of DBS surgery.

Objective. Cochlear implants, while providing significant benefits to recipients, remain limited due to broad neural activation. Focussed multipolar stimulation (FMP) is an advanced stimulation strategy that uses multiple current sources to produce highly focussed patterns of neural excitation in order to overcome these shortcomings. Approach. This report presents single-source multipolar stimulation (SSMPS), a novel form of stimulation based on a single current source and a passive current divider. Compared to conventional FMP with multiple current sources, SSMPS can be implemented as a modular addition to conventional (i.e. single) current source stimulation systems facilitating charge balance within the cochlea. As with FMP, SSMPS requires the determination of a transimpedance matrix to allow for focusing of the stimulation. The first part of this study therefore investigated the effects of varying the probe stimulus (e.g. current level and pulse width) on the measurement of the transimpedance matrix. SSMPS was then studied using in vitro based measurements of voltages at non-stimulated electrodes along an electrode array in normal saline. The voltage reduction with reference to monopolar stimulation was compared to tripolar and common ground stimulation, two clinically established stimulation modes. Finally, a proof of principle in vivo test of SSMPS in a feline model was performed. Main results. A probe stimulus of at least 40 nC is required to reliably measure the transimpedance matrix. In vitro stimulation using SSMPS resulted in a significantly greater voltage reduction compared to monopolar, tripolar and common ground stimulation. Interestingly, matching measurement and stimulation parameters did not lead to an improved focussing performance. Compared to monopolar stimulation, SSMPS resulted in reduced spread of neural activity in the inferior colliculus, albeit with increased thresholds. Significance. The present study demonstrates that SSMPS successfully limits the broadening of the excitatory field along the electrode array and a subsequent reduction in the spread of neural excitation.

Objective. The development of experimental methodology utilizing graphene micro-transistor arrays to facilitate and advance translational research into cortical spreading depression (CSD) in the awake brain. Approach. CSDs were reliably induced in awake nontransgenic mice using optogenetic methods. High-fidelity DC-coupled electrophysiological mapping of propagating CSDs was obtained using flexible arrays of graphene soultion-gated field-effect transistors (gSGFETs). Main results. Viral vectors targetted channelrhopsin expression in neurons of the motor cortex resulting in a transduction volume ⩾1 mm ³. 5–10 s of continous blue light stimulation induced CSD that propagated across the cortex at a velocity of 3.0 ± 0.1 mm min ⁻¹. Graphene micro-transistor arrays enabled high-density mapping of infraslow activity correlated with neuronal activity suppression across multiple frequency bands during both CSD initiation and propagation. Localized differences in the CSD waveform could be detected and categorized into distinct clusters demonstrating the spatial resolution advantages of DC-coupled recordings. We exploited the reliable and repeatable induction of CSDs using this preparation to perform proof-of-principle pharmacological interrogation studies using NMDA antagonists. MK801 (3 mg kg ⁻¹) suppressed CSD induction and propagation, an effect mirrored, albeit transiently, by ketamine (15 mg kg ⁻¹), thus demonstrating this models’ applicability as a preclinical drug screening platform. Finally, we report that CSDs could be detected through the skull using graphene micro-transistors, highlighting additional advantages and future applications of this technology. Significance. CSD is thought to contribute to the pathophysiology of several neurological diseases. CSD research will benefit from technological advances that permit high density electrophysiological mapping of the CSD waveform and propagation across the cortex. We report an in vivo assay that permits minimally invasive optogenetic induction, combined with multichannel DC-coupled recordings enabled by gSGFETs in the awake brain. Adoption of this technological approach could facilitate and transform preclinical investigations of CSD in disease relevant models.

Objective. Ensuring the longevity of implantable devices is critical for their clinical usefulness. This is commonly achieved by hermetically sealing the sensitive electronics in a water impermeable housing, however, this method limits miniaturisation. Alternatively, silicone encapsulation has demonstrated long-term protection of implanted thick-film electronic devices. However, much of the current conformal packaging research is focused on more rigid coatings, such as parylene, liquid crystal polymers and novel inorganic layers. Here, we consider the potential of silicone to protect implants using thin-film technology with features 33 times smaller than thick-film counterparts. Approach. Aluminium interdigitated comb structures under plasma-enhanced chemical vapour deposited passivation (SiO _x , SiO _x N _y , SiO _x N _y + SiC) were encapsulated in medical grade silicones, with a total of six passivation/silicone combinations. Samples were aged in phosphate-buffered saline at 67  ^∘C for up to 694 days under a continuous ±5 V biphasic waveform. Periodic electrochemical impedance spectroscopy measurements monitored for leakage currents and degradation of the metal traces. Fourier-transform infrared spectroscopy, x-ray photoelectron spectroscopy, focused-ion-beam and scanning-electron- microscopy were employed to determine any encapsulation material changes. Main results. No silicone delamination, passivation dissolution, or metal corrosion was observed during ageing. Impedances greater than 100 GΩ were maintained between the aluminium tracks for silicone encapsulation over SiO _x N _y and SiC passivations. For these samples the only observed failure mode was open-circuit wire bonds. In contrast, progressive hydration of the SiO _x caused its resistance to decrease by an order of magnitude. Significance. These results demonstrate silicone encapsulation offers excellent protection to thin-film conducting tracks when combined with appropriate inorganic thin films. This conclusion corresponds to previous reliability studies of silicone encapsulation in aqueous environments, but with a larger sample size. Therefore, we believe silicone encapsulation to be a realistic means of providing long-term protection for the circuits of implanted electronic medical devices.

Objective. Lack of sensation from a hand or prosthesis can result in substantial functional deficits. Surface electrical stimulation of the peripheral nerves is a promising non-invasive approach to restore lost sensory function. However, the utility of standard surface stimulation methods has been hampered by localized discomfort caused by unintended activation of afferents near the electrodes and limited ability to specifically target underlying neural tissue. The objectives of this work were to develop and evaluate a novel channel-hopping interleaved pulse scheduling (CHIPS) strategy for surface stimulation that is designed to activate deep nerves while reducing activation of fibers near the electrodes. Approach. The median nerve of able-bodied subjects was activated by up to two surface stimulating electrode pairs placed around their right wrist. Subjects received biphasic current pulses either from one electrode pair at a time (single-channel), or interleaved between two electrode pairs (multi-channel). Percept thresholds were characterized for five pulse durations under each approach, and psychophysical questionnaires were used to interrogate the perceived modality, quality and location of evoked sensations. Main results. Stimulation with CHIPS elicited enhanced tactile percepts that were distally referred, while avoiding the distracting sensations and discomfort associated with localized charge densities. These effects were reduced after introduction of large delays between interleaved pulses. Significance. These findings demonstrate that our pulse scheduling strategy can selectively elicit referred sensations that are comfortable, thus overcoming the primary limitations of standard surface stimulation methods. Implementation of this strategy with an array of spatially distributed electrodes may allow for rapid and effective stimulation fitting. The ability to elicit comfortable and referred tactile percepts may enable the use of this neurostimulation strategy to provide meaningful and intuitive feedback from a prosthesis, enhance tactile feedback after sensory loss secondary to nerve damage, and deliver non-invasive stimulation therapies to treat various pain conditions.

Visual prosthesis devices designed to restore sight to the blind have been under development in the laboratory for several decades. Clinical translation continues to be challenging, due in part to gaps in our understanding of critical parameters such as how phosphenes, the electrically-generated pixels of artificial vision, can be combined to form images. In this review we explore the effects that synchronous and asynchronous electrical stimulation across multiple electrodes have in evoking phosphenes. Understanding how electrical patterns influence phosphene generation to control object binding and perception of visual form is fundamental to creation of a clinically successful prosthesis.

Objective. For patients with disorders of consciousness (DOC), such as vegetative state (VS) and minimally conscious state (MCS), communication is challenging. Currently, the communication methods of DOC patients are limited to behavioral responses. However, patients with DOC cannot provide sufficient behavioral responses due to motor impairments and limited attention. In this study, we proposed a hybrid asynchronous brain–computer interface (BCI) system that provides a new communication channel for patients with DOC. Approach. Seven patients with DOC (3 VS and 4 MCS) and eleven healthy subjects participated in our experiment. Each subject was instructed to focus on the square with the Chinese words ‘Yes’ and ‘No’. Then, the BCI system determined the target square with both P300 and steady-state visual evoked potential (SSVEP) detections. For the healthy group, we tested the performance of the hybrid system and the single-modality BCI system. Main results. All healthy subjects achieved significant accuracy (ranging from 72% to 100%) in both the hybrid system and the single modality system. The hybrid asynchronous BCI system outperformed the P300-only and SSVEP-only systems. Furthermore, we employed the asynchronous approach to dynamically collect the electroencephalography signal. Compared with the synchronous system, there was a 21% reduction in the average required rounds and a reduction of 105 s in the online experiment time. This asynchronous system was applied to detect the ‘yes/no’ communication function of seven patients with DOC, and the results showed that three of the patients (3 MCS) not only showed significant accuracies (67 ± 3%) in the online experiment, and their Coma Recovery Scale-Revised scores were also improved compared with the scores before the experiment. This result demonstrated that 3 of 7 patients were able to communicate using our hybrid asynchronous BCI system. Significance. This hybrid asynchronous BCI system can be used as a useful auxiliary bedside tool for simple communication with DOC patients.

Objectives. This paper aims to investigate the feasibility and the validity of applying deep convolutional neural networks (CNN) to identify motor unit (MU) spike trains and estimate the neural drive to muscles from high-density electromyography (HD-EMG) signals in real time. Two distinct deep CNNs are compared with the convolution kernel compensation (CKC) algorithm using simulated and experimentally recorded signals. The effects of window size and step size of the input HD-EMG signals are also investigated. Approach. The MU spike trains were first identified with the CKC algorithm. The HD-EMG signals and spike trains were used to train the deep CNN. Then, the deep CNN decomposed the HD-EMG signals into MU discharge times in real time. Two CNN approaches are compared with the CKC: (a) multiple single-output deep CNN (SO-DCNN) with one MU decomposed per network, and (b) one multiple-output deep CNN (MO-DCNN) to decompose all MUs (up to 23) with one network. Main results. The MO-DCNN outperformed the SO-DCNN in terms of training time (3.2–21.4 s epoch ⁻¹ vs 6.5–47.8 s epoch ⁻¹, respectively) and prediction time (0.04 vs 0.27 s sample ⁻¹, respectively). The optimal window size and step size for MO-DCNN were 120 and 20 data points, respectively. It results in sensitivity of 98% and 85% with simulated and experimentally recorded HD-EMG signals, respectively. There is a high cross-correlation coefficient between the neural drive estimated with CKC and that estimated with MO-DCNN (range of r-value across conditions: 0.88–0.95). Significance. We demonstrate the feasibility and the validity of using deep CNN to accurately identify MU activity from HD-EMG with a latency lower than 80 ms, which falls within the lower bound of the human electromechanical delay. This method opens many opportunities for using the neural drive to interface humans with assistive devices.

Objective. High frequency deep brain stimulation (DBS) of the subthalamic nucleus (STN) suppresses excessive beta band (∼13–30 Hz) activity of the motor cortex in Parkinson’s disease (PD). While the mechanisms of action of STN DBS are not well-understood, strong evidence supports a role for cortical network modulating effects elicited by antidromic activation of cortical axons via the hyperdirect pathway. Approach. A spiking model of the thalamo-cortical microcircuit was developed to examine modulation of cortical network activity by antidromic STN DBS, mediated by direct activation of deep pyramidal neurons (PNs) and subsequent indirect activation of other thalamo-cortical structures. Main results. Increasing synaptic coupling strength from cortical granular to superficial layers, from inhibitory neurons to deep PNs, and from thalamus reticular to relay cells, along with thalamocortical connection strength, accompanied by reduced coupling from cortical superficial to granular layers, from thalamus relay cells to reticular neurons, and corticothalamic connection strength, led to increased beta activity and neural synchrony, as observed in PD. High frequency DBS desynchronized correlated neural activity, resulting in clusters of both excited and inhibited deep cortical PNs. The emergence of additional frequency components in the local field potential (LFP), and increased power at subharmonics of the DBS frequency as observed in patients with dyskinesia during DBS, occurred under different stimulus amplitudes and frequencies. While high-frequency (>100 Hz) DBS suppressed the LFP beta power, low-frequency (<40 Hz) DBS increased beta power when more than 10% of PNs were activated, but reduced the total beta power at lower levels of neural activation. Significance. The results suggest a potential mechanism for experimentally observed alterations in cortical neural activity during DBS via the propagation of DBS stimuli throughout the cortical network, modulated by short-term synaptic plasticity, and the emergence of resonance due to interaction of DBS with existing M1 rhythms by engaging feedforward-feedback loops.

Objective. Typically, a brain–computer interface (BCI) is calibrated using user- and session-specific data because of the individual idiosyncrasies and the non-stationary signal properties of the electroencephalogram (EEG). Therefore, it is normal for BCIs to undergo a time-consuming passive training stage that prevents users from directly operating them. In this study, we systematically reduce the training data set in a stepwise fashion, to ultimately arrive at a calibration-free method for a code-modulated visually evoked potential (cVEP)-based BCI to fully eliminate the tedious training stage. Approach. In an extensive offline analysis, we compare our sophisticated encoding model with a traditional event-related potential (ERP) technique. We calibrate the encoding model in a standard way, with data limited to a single class while generalizing to all others and without any data. In addition, we investigate the feasibility of the zero-training cVEP BCI in an online setting. Main results. By adopting the encoding model, the training data can be reduced substantially, while maintaining both the classification performance as well as the explained variance of the ERP method. Moreover, with data from only one class or even no data at all, it still shows excellent performance. In addition, the zero-training cVEP BCI achieved high communication rates in an online spelling task, proving its feasibility for practical use. Significance. To date, this is the fastest zero-training cVEP BCI in the field, allowing high communication speeds without calibration while using only a few non-invasive water-based EEG electrodes. This allows us to skip the training stage altogether and spend all the valuable time on direct operation. This minimizes the session time and opens up new exciting directions for practical plug-and-play BCI. Fundamentally, these results validate that the adopted neural encoding model compresses data into event responses without the loss of explanatory power compared to using full ERPs as a template.

Objective. Growing evidence suggests that electroencephalography (EEG) electrode (sensor) potential time series (PTS) of slow cortical potentials (SCPs) hold motor neural correlates that can be used for motion trajectory prediction, commonly by multiple linear regression (mLR). It is not yet known whether arm-joint trajectories can be reliably decoded from current sources, computed from sensor data, from which brain areas they can be decoded and using which neural features. Approach. In this study, the PTS of 44 sensors were fed into sLORETA source localization software to compute current source activity in 30 regions of interest (ROIs) found in a recent meta-analysis to be engaged in action execution, motor imagery and motor preparation. The current sources PTS and band-power time series (BTS) in several frequency bands and time lags were used to predict actual and imagined trajectories in 3D space of the three velocity components of the hand, elbow and shoulder of nine subjects using an mLR model. Main results. For all arm joints and movement types, current source SCPs PTS contributed most to trajectory reconstruction with time lags 150, 116 and 84 ms providing the highest contribution, and current source BTS in any of the tested frequency bands was not informative. Person’s correlation coefficient ( r) averaged across movement types, arm joints and velocity components using source data was slightly lower than using sensor data ( r = 0.25 and r = 0.28, respectively). For each ROI, the three current source dipoles had different contribution to the reconstruction of each of the three velocity components. Significance. Overall, our results demonstrate the feasibility of predicting of actual and imagined 3D trajectories of all arm joints from current sources, computed from scalp EEG. These findings may be used by developers of a future BCI as a validated set of contributing ROIs.

Objective. Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying common spatial pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian spatial pattern (RSP) method, which is based on the backward channel selection procedure. Approach. The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers. Main results. Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results. Significance. Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.