Table of contents

Objective. Adaptive deep brain stimulation (aDBS) is a form of invasive stimulation that was conceived to overcome the technical limitations of traditional DBS, which delivers continuous stimulation of the target structure without considering patients' symptoms or status in real-time. Instead, aDBS delivers on-demand, contingency-based stimulation. So far, aDBS has been tested in several neurological conditions, and will be soon extensively studied to translate it into clinical practice. However, an exhaustive description of technical aspects is still missing. Approach. in this topical review, we summarize the knowledge about the current (and future) aDBS approach and control algorithms to deliver the stimulation, as reference for a deeper undestending of aDBS model. Main results. We discuss the conceptual and functional model of aDBS, which is based on the sensing module (that assesses the feedback variable), the control module (which interpretes the variable and elaborates the new stimulation parameters), and the stimulation module (that controls the delivery of stimulation), considering both the historical perspective and the state-of-the-art of available biomarkers. Significance. aDBS modulates neuronal circuits based on clinically relevant biofeedback signals in real-time. First developed in the mid-2000s, many groups have worked on improving closed-loop DBS technology. The field is now at a point in conducting large-scale randomized clinical trials to translate aDBS into clinical practice. As we move towards implanting brain-computer interfaces in patients, it will be important to understand the technical aspects of aDBS.

Objective. Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain–computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines. Approach. The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc. Main results. The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development. Significance. Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

Objective. Stroke is one of the most common neural disorders, which causes physical disabilities and motor impairments among its survivors. Several technologies have been developed for providing stroke rehabilitation and to assist the survivors in performing their daily life activities. Currently, the use of flexible technology (FT) for stroke rehabilitation systems is on a rise that allows the development of more compact and lightweight wearable systems, which stroke survivors can easily use for long-term activities. Approach. For stroke applications, FT mainly includes the 'flexible/stretchable electronics', 'e-textile (electronic textile)' and 'soft robotics'. Thus, a thorough literature review has been performed to report the practical implementation of FT for post-stroke application. Main results. In this review, the highlights of the advancement of FT in stroke rehabilitation systems are dealt with. Such systems mainly involve the 'biosignal acquisition unit', 'rehabilitation devices' and 'assistive systems'. In terms of biosignals acquisition, electroencephalography and electromyography are comprehensively described. For rehabilitation/assistive systems, the application of functional electrical stimulation and robotics units (exoskeleton, orthosis, etc) have been explained. Significance. This is the first review article that compiles the different studies regarding FT based post-stroke systems. Furthermore, the technological advantages, limitations, and possible future implications are also discussed to help improve and advance the flexible systems for the betterment of the stroke community.

Objective: Perinatal ischemic stroke is estimated to occur in 1/2300–1/5000 live births, but early differential diagnosis from global hypoxia-ischemia is often difficult. In this study, we tested the ability of a hand-held transcranial photoacoustic (PA) imaging probe to non-invasively detect a focal photothrombotic stroke (PTS) within 2 h of stroke onset in a gyrencephalic piglet brain. Approach: About 17 stroke lesions of approximately 1 cm² area were introduced randomly in anterior or posterior cortex via the light/dye PTS technique in anesthetized neonatal piglets (n = 11). The contralateral non-ischemic region served as control tissue for discrimination contrast for the PA hemoglobin metrics: oxygen saturation, total hemoglobin (tHb), and individual quantities of oxygenated and deoxygenated hemoglobin (HbO₂ and HbR). Main results: The PA-derived tissue oxygen saturation at 2 h yielded a significant separation between control and affected regions-of-interest (p < 0.0001), which were well matched with 24 h post-stroke cerebral infarction confirmed in the triphenyltetrazolium chloride-stained image. The quantity of HbO₂ also displayed a significant contrast (p = 0.021), whereas tHb and HbR did not. The analysis on receiver operating characteristic curves and multivariate data analysis also agreed with the results above. Significance: This study shows that a hand-held transcranial PA neuroimaging device can detect a regional thrombotic stroke in the cerebral cortex of a neonatal piglet. In particular, we conclude that the oxygen saturation metric can be used alone to identify regional stroke lesions. The lack of change in tHb may be related to arbitrary hand-held imaging configuration and/or entrapment of red blood cells within the thrombotic stroke.

Objective. We propose a novel method to estimate the instantaneous oscillatory phase to implement a real-time system for state-informed sensory stimulation in electroencephalography (EEG) experiments. Approach. The method uses Kalman filter-based prediction to estimate current and future EEG signals. We tested the performance of our method in a real-time situation. Main results. Our method showed higher accuracy in predicting the EEG phase than the conventional autoregressive (AR) model-based method. Significance. A Kalman filter allows us to easily estimate the instantaneous phase of EEG oscillations based on the automatically estimated AR model implemented in a real-time signal processing machine. The proposed method has a potential for versatile applications targeting the modulation of EEG phase dynamics and the plasticity of brain networks in relation to perceptual or cognitive functions.

Objective. Photothermal neural stimulation has been developed in a variety of interfaces as an alternative technology that can perturb neural activity. The demonstrations of these techniques have heavily relied on open-loop stimulation or complete suppression of neural activity. To extend the controllability of photothermal neural stimulation, combining it with a closed-loop system is required. In this work, we investigated whether photothermal suppression mechanism can be used in a closed-loop system to reliably modulate neural spike rate to non-zero setpoints. Approach. To incorporate the photothermal inhibition mechanism into the neural feedback system, we combined a thermoplasmonic stimulation platform based on gold nanorods (GNRs) and near-infrared illuminations (808 nm, spot size: 2 mm or 200 μm in diameter) with a proportional-integral (PI) controller. The closed-loop feedback control system was implemented to track predetermined target spike rates of hippocampal neuronal networks cultured on GNR-coated microelectrode arrays. Main results. The closed-loop system for neural spike rate control was successfully implemented using a PI controller and the thermoplasmonic neural suppression platform. Compared to the open-loop control, the target-channel spike rates were precisely modulated to remain constant or change in a sinusoidal form in the range below baseline spike rates. The spike rate response behaviors were affected by the choice of the controller gain. We also demonstrated that the functional connectivity of a synchronized bursting network could be altered by controlling the spike rate of one of the participating channels. Significance. The thermoplasmonic feedback controller proved that it can precisely modulate neural spike rate of neural activity in vitro. This technology can be used for studying neuronal network dynamics and might provide insights in developing new neuromodulation techniques in clinical applications.

Objective. Coils designed for transcranial magnetic stimulation (TMS) must incorporate trade-offs between the required electrical power or energy, focality and depth penetration of the induced electric field (E-field), coil size, and mechanical properties of the coil, as all of them cannot be optimally met at the same time. In multi-locus TMS (mTMS), a transducer consisting of several coils allows electronically targeted stimulation of the cortex without physically moving a coil. In this study, we aimed to investigate the relationship between the number of coils in an mTMS transducer, the focality of the induced E-field, and the extent of the cortical region within which the location and orientation of the maximum of the induced E-field can be controlled. Approach. We applied convex optimization to design planar and spherically curved mTMS transducers of different E-field focalities and analyzed their properties. We characterized the trade-off between the focality of the induced E-field and the extent of the cortical region that can be stimulated with an mTMS transducer with a given number of coils. Main results. At the expense of the E-field focality, one can, with the same number of coils, design an mTMS transducer that can control the location and orientation of the peak of the induced E-field within a wider cortical region. Significance. With E-fields of moderate focality, the problem of electronically targeted TMS becomes considerably easier compared with highly focal E-fields; this may speed up the development of mTMS and the emergence of new clinical and research applications.

Objective. Neural activity represents a functional readout of neurons that is increasingly important to monitor in a wide range of experiments. Extracellular recordings have emerged as a powerful technique for measuring neural activity because these methods do not lead to the destruction or degradation of the cells being measured. Current approaches to electrophysiology have a low throughput of experiments due to manual supervision and expensive equipment. This bottleneck limits broader inferences that can be achieved with numerous long-term recorded samples. Approach. We developed Piphys, an inexpensive open source neurophysiological recording platform that consists of both hardware and software. It is easily accessed and controlled via a standard web interface through Internet of Things (IoT) protocols. Main results. We used a Raspberry Pi as the primary processing device along with an Intan bioamplifier. We designed a hardware expansion circuit board and software to enable voltage sampling and user interaction. This standalone system was validated with primary human neurons, showing reliability in collecting neural activity in near real-time. Significance. The hardware modules and cloud software allow for remote control of neural recording experiments as well as horizontal scalability, enabling long-term observations of development, organization, and neural activity at scale.

Objective. Brain-controlled robotic arms have shown broad application prospects with the development of robotics, science and information decoding. However, disadvantages, such as poor flexibility restrict its wide application. Approach. In order to alleviate these drawbacks, this study proposed a robotic arm asynchronous control system based on steady-state visual evoked potential (SSVEP) in an augmented reality (AR) environment. In the AR environment, the participants were able to concurrently see the robot arm and visual stimulation interface through the AR device. Therefore, there was no need to switch attention frequently between the visual stimulation interface and the robotic arm. This study proposed a multi-template algorithm based on canonical correlation analysis and task-related component analysis to identify 12 targets. An optimization strategy based on dynamic window was adopted to adjust the duration of visual stimulation adaptively. Main results. Experimental results of this study found that the high-frequency SSVEP-based brain–computer interface (BCI) realized the switch of the system state, which controlled the robotic arm asynchronously. The average accuracy of the offline experiment was 94.97$\%$, whereas the average information translate rate was 67.37 ± 14.27 bits·min⁻¹. The online results from ten healthy subjects showed that the average selection time of a single online command was 2.04 s, which effectively reduced the visual fatigue of the subjects. Each subject could quickly complete the puzzle task. Significance. The experimental results demonstrated the feasibility and potential of this human-computer interaction strategy and provided new ideas for BCI-controlled robots.

Objective. Computational models have shown that directional electrical contacts placed within the epineurium, between the fascicles, and not penetrating the perineurium, can achieve selectivity levels similar to point source contacts placed within the fascicle. The objective of this study is to test, in a murine model, the hypothesis that directed interfascicular contacts are selective. Approach. Multiple interfascicular electrodes with directional contacts, exposed on a single face, were implanted in the sciatic nerves of 32 rabbits. Fine-wire intramuscular wire electrodes were implanted to measure electromyographic (EMG) activity from medial and lateral gastrocnemius, soleus, and tibialis anterior muscles. Main results. The recruitment data demonstrated that directed interfascicular interfaces, which do not penetrate the perineurium, selectively activate different axon populations. Significance. Interfascicular interfaces that are inside the nerve, but do not penetrate the perineurium are an alternative to intrafascicular interfaces and may offer additional selectivity compared to extraneural approaches.

Objective. Epiretinal prostheses are designed to restore vision to people blinded by photoreceptor degenerative diseases by stimulating surviving retinal ganglion cells (RGCs), which carry visual signals to the brain. However, inadvertent stimulation of RGCs at their axons can result in non-focal visual percepts, limiting the quality of artificial vision. Theoretical work has suggested that axon activation can be avoided with current stimulation designed to minimize the second spatial derivative of the induced extracellular voltage along the axon. However, this approach has not been verified experimentally at the resolution of single cells. Approach. In this work, a custom multi-electrode array (512 electrodes, 10 μm diameter, 60 μm pitch) was used to stimulate and record RGCs in macaque retina ex vivo at single-cell, single-spike resolution. RGC activation thresholds resulting from bi-electrode stimulation, which consisted of bipolar currents simultaneously delivered through two electrodes straddling an axon, were compared to activation thresholds from traditional single-electrode stimulation. Main results. On average, across three retinal preparations, the bi-electrode stimulation strategy reduced somatic activation thresholds (∼21%) while increasing axonal activation thresholds (∼14%), thus favoring selective somatic activation. Furthermore, individual examples revealed rescued selective activation of somas that was not possible with any individual electrode. Significance. This work suggests that a bi-electrode epiretinal stimulation strategy can reduce inadvertent axonal activation at cellular resolution, for high-fidelity artificial vision.

Objective. Currently, only behavioral speech understanding tests are available, which require active participation of the person being tested. As this is infeasible for certain populations, an objective measure of speech intelligibility is required. Recently, brain imaging data has been used to establish a relationship between stimulus and brain response. Linear models have been successfully linked to speech intelligibility but require per-subject training. We present a deep-learning-based model incorporating dilated convolutions that operates in a match/mismatch paradigm. The accuracy of the model's match/mismatch predictions can be used as a proxy for speech intelligibility without subject-specific (re)training. Approach. We evaluated the performance of the model as a function of input segment length, electroencephalography (EEG) frequency band and receptive field size while comparing it to multiple baseline models. Next, we evaluated performance on held-out data and finetuning. Finally, we established a link between the accuracy of our model and the state-of-the-art behavioral MATRIX test. Main results. The dilated convolutional model significantly outperformed the baseline models for every input segment length, for all EEG frequency bands except the delta and theta band, and receptive field sizes between 250 and 500 ms. Additionally, finetuning significantly increased the accuracy on a held-out dataset. Finally, a significant correlation (r = 0.59, p = 0.0154) was found between the speech reception threshold (SRT) estimated using the behavioral MATRIX test and our objective method. Significance. Our method is the first to predict the SRT from EEG for unseen subjects, contributing to objective measures of speech intelligibility.

Objective. Electrical recordings of neural activity from brain surface have been widely employed in basic neuroscience research and clinical practice for investigations of neural circuit functions, brain–computer interfaces, and treatments for neurological disorders. Traditionally, these surface potentials have been believed to mainly reflect local neural activity. It is not known how informative the locally recorded surface potentials are for the neural activities across multiple cortical regions. Approach. To investigate that, we perform simultaneous local electrical recording and wide-field calcium imaging in awake head-fixed mice. Using a recurrent neural network model, we try to decode the calcium fluorescence activity of multiple cortical regions from local electrical recordings. Main results. The mean activity of different cortical regions could be decoded from locally recorded surface potentials. Also, each frequency band of surface potentials differentially encodes activities from multiple cortical regions so that including all the frequency bands in the decoding model gives the highest decoding performance. Despite the close spacing between recording channels, surface potentials from different channels provide complementary information about the large-scale cortical activity and the decoding performance continues to improve as more channels are included. Finally, we demonstrate the successful decoding of whole dorsal cortex activity at pixel-level using locally recorded surface potentials. Significance. These results show that the locally recorded surface potentials indeed contain rich information of the large-scale neural activities, which could be further demixed to recover the neural activity across individual cortical regions. In the future, our cross-modality inference approach could be adapted to virtually reconstruct cortex-wide brain activity, greatly expanding the spatial reach of surface electrical recordings without increasing invasiveness. Furthermore, it could be used to facilitate imaging neural activity across the whole cortex in freely moving animals, without requirement of head-fixed microscopy configurations.

Objective. Hyperscanning is an emerging technology that concurrently scans the neural dynamics of multiple individuals to study interpersonal interactions. In particular, hyperscanning with electroencephalography (EEG) is increasingly popular owing to its mobility and its ability to allow studying social interactions in naturalistic settings at the millisecond scale. Approach. To align multiple EEG time series with sophisticated event markers in a single time domain, a precise and unified timestamp is required for stream synchronization. This study proposes a clock-synchronized method that uses a custom-made RJ45 cable to coordinate the sampling between wireless EEG amplifiers to prevent incorrect estimation of interbrain connectivity due to asynchronous sampling. In this method, analog-to-digital converters are driven by the same sampling clock. Additionally, two clock-synchronized amplifiers leverage additional radio frequency channels to keep the counter of their receiving dongles updated, which guarantees that binding event markers received by the dongle with the EEG time series have the correct timestamp. Main results. The results of two simulation experiments and one video gaming experiment reveal that the proposed method ensures synchronous sampling in a system with multiple EEG devices, achieving near-zero phase lag and negligible amplitude difference between the signals. Significance. According to all of the signal-similarity metrics, the suggested method is a promising option for wireless EEG hyperscanning and can be utilized to precisely assess the interbrain couplings underlying social-interaction behaviors.

Objective. Accumulating evidence has revealed that emotions can be provided with the modulatory effect on working memory (WM) and WM load is an important factor for the interaction between emotion and WM. However, it remains controversial whether emotions inhibit or facilitate WM and the interaction between cognitive task, processing load and emotional processing remains unclear. Approach. In this study, we used a change detection paradigm wherein memory items have four different load sizes and emotion videos to induce three emotions (negative, neutral, and positive). We performed an event-related spectral perturbation (ERSP) analysis and a spatiotemporal pattern similarity (STPS) analysis on the electroencephalography data. Main results. The ERSP results indicated that alpha and beta oscillations can reflect the difference among WM load sizes and also can reflect the difference among emotions under middle high WM load over posterior brain region in the maintenance stage. Moreover, the STPS results demonstrated a significant interaction between emotion and WM load size in the posterior region and found significantly higher similarity indexes for the negative emotion to the neutral emotion under the middle high WM load during WM maintenance. In addition, The STPS results also revealed that both positive emotion and negative emotion could interfere with the distinction of load sizes. Significance. The consistence of the behavioral, ERSP and STPS results suggested that when the memory load approaches the limit of WM capacity, negative emotion could facilitate WM through the top–down attention modulation promoting the most relevant information storage during WM maintenance.

Objective. Hippocampal ripples are transient neuronal features observed in high-frequency oscillatory bands of local field potentials (LFPs), and they occur primarily during periods of behavioral immobility and slow-wave sleep. Ripples have been defined based on mathematically engineered features, such as magnitudes, durations, and cycles per event. However, the 'ripple' could vary from laboratory to laboratory because their definition is subject to human bias, including the arbitrary choice of parameters and thresholds. In addition, LFPs are often influenced by myoelectric noise arising from animal movement, making it difficult to distinguish ripples from high-frequency noises. These problems have to be overcome. Approach. We extracted ripple candidates under few constraints and labeled them as binary or stochastic 'true' or 'false' ripples using Gaussian mixed model clustering and a deep convolutional neural network (CNN) in a weakly supervised fashion. Main results. Our automatic method separated ripples and myoelectric noise and was able to detect ripples even when the animals were moving. Moreover, we confirmed that a CNN detected ripples defined by our method. Leave-one-animal-out cross-validation estimated the accuracy, the area under the precision-recall curve, the receiver operating characteristic curve to be 0.88, 0.99 and 0.96, respectively. Significance. Our automatic ripple detection method will reduce time spent on performing laborious experiments and analyses.

Objective. Optogenetics involves delivery of light-sensitive opsins to the target brain region, as well as introduction of optical and electrical devices to manipulate and record neural activity, respectively, from the targeted neural population. Combining these functionalities in a single implantable device is of great importance for a precise investigation of neural networks while minimizing tissue damage. Approach. We report on the development, characterization, and in vivo validation of a multifunctional optrode that combines a silicon-based neural probe with an integrated microfluidic channel, and an optical glass fiber in a compact assembly. The silicon probe comprises an 11-µm-wide fluidic channel and 32 recording electrodes (diameter 30 µm) on a tapered probe shank with a length, thickness, and maximum width of 7.5 mm, 50 µm, and 150 µm, respectively. The size and position of fluidic channels, electrodes, and optical fiber can be precisely tuned according to the in vivo application. Main results. With a total system weight of 0.97 g, our multifunctional optrode is suitable for chronic in vivo experiments requiring simultaneous drug delivery, optical stimulation, and neural recording. We demonstrate the utility of our device in optogenetics by injecting a viral vector carrying a ChR2-construct in the prefrontal cortex and subsequent photostimulation of the transduced neurons while recording neural activity from both the target and adjacent regions in a freely moving rat for up to 9 weeks post-implantation. Additionally, we demonstrate a pharmacological application of our device by injecting GABA antagonist bicuculline in an anesthetized rat brain and simultaneously recording the electrophysiological response. Significance. Our triple-modality device enables a single-step optogenetic surgery. In comparison to conventional multi-step surgeries, our approach achieves higher spatial specificity while minimizing tissue damage.

Objective. We introduce Sparse exact low resolution electromagnetic tomography (eLORETA), a novel method for estimating a nonparametric solution to the source localization problem. Its goal is to generate a sparser solution compared to other source localization methods including eLORETA while benefitting from the latter's superior source localization accuracy. Approach. Sparse eLORETA starts by reducing the source space of the Lead Field Matrix using structured sparse Bayesian learning from which a Reduced Lead Field Matrix is constructed, which is used as input to eLORETA. Main results. With Sparse eLORETA, source sparsity can be traded against signal fidelity; the proposed optimum is shown to yield a much sparser solution than eLORETA's with only a slight loss in signal fidelity. Significance. When pursuing a data-driven approach, for cases where it is difficult to choose specific regions of interest, or when subsequently a connectivity analysis is performed, source space reduction could prove beneficial.

Objective. In this paper, a new approach of extracting and measuring the variability in electroencephalogram (EEG) was proposed to assess the depth of anesthesia (DOA) under general anesthesia. Approach. The EEG variability (EEGV) was extracted as a fluctuation in time interval that occurs between two local maxima of EEG. Eight parameters related to EEGV were measured in time and frequency domains, and compared with state-of-the-art DOA estimation parameters, including sample entropy, permutation entropy, median frequency and spectral edge frequency of EEG. The area under the receiver-operator characteristics curve (AUC) and Pearson correlation coefficient were used to validate its performance on 56 patients. Main results. Our proposed EEGV-derived parameters yield significant difference for discriminating between awake and anesthesia stages at a significance level of 0.05, as well as improvement in AUC and correlation coefficient on average, which surpasses the conventional features of EEG in detection accuracy of unconscious state and tracking the level of consciousness. Significance. To sum up, EEGV analysis provides a new perspective in quantifying EEG and corresponding parameters are powerful and promising for monitoring DOA under clinical situations.

Objective. For decades electrical stimulation has been used in neuroscience to investigate brain networks and been deployed clinically as a mode of therapy. Classically, all methods of electrical stimulation require implanted electrodes to be connected in some manner to an apparatus which provides power for the stimulation itself. Approach. We show the use of novel organic electronic devices, specifically organic electrolytic photocapacitors (OEPCs), which can be activated when illuminated with deep-red wavelengths of light and correspondingly do not require connections with external wires or power supplies when implanted at various depths in vivo. Main results. We stimulated cortical brain tissue of mice with devices implanted subcutaneously, as well as beneath both the skin and skull to demonstrate a wireless stimulation of the whisker motor cortex. Devices induced both a behavior response (whisker movement) and a sensory response in the corresponding sensory cortex. Additionally, we showed that coating OEPCs with a thin layer of a conducting polymer formulation (PEDOT:PSS) significantly increases their charge storage capacity, and can be used to further optimize the applied photoelectrical stimulation. Significance. Overall, this new technology can provide an on-demand electrical stimulation by simply using an OEPC and a deep-red wavelength illumination. Wires and interconnects to provide power to implanted neurostimulation electrodes are often problematic in freely-moving animal research and with implanted electrodes for long-term therapy in patients. Our wireless brain stimulation opens new perspectives for wireless electrical stimulation for applications in fundamental neurostimulation and in chronic therapy.

Objective. Understanding how the retina converts a natural image or an electrically stimulated one into neural firing patterns is the focus of on-going research activities. Ex vivo, the retina can be readily investigated using multi electrode arrays (MEAs). However, MEA recording and stimulation from an intact retina (in the eye) has been so far insufficient. Approach. In the present study, we report new soft carbon electrode arrays suitable for recording and stimulating neural activity in an intact retina. Screen-printing of carbon ink on 20 µm polyurethane (PU) film was used to realize electrode arrays with electrodes as small as 40 µm in diameter. Passivation was achieved with a holey membrane, realized using laser drilling in a thin (50 µm) PU film. Plasma polymerized 3.4-ethylenedioxythiophene was used to coat the electrode array to improve the electrode specific capacitance. Chick retinas, embryonic stage day 13, both explanted and intact inside an enucleated eye, were used. Main results. A novel fabrication process based on printed carbon electrodes was developed and yielded high capacitance electrodes on a soft substrate. Ex vivo electrical recording of retina activity with carbon electrodes is demonstrated. With the addition of organic photo-capacitors, simultaneous photo-electrical stimulation and electrical recording was achieved. Finally, electrical activity recordings from an intact chick retina (inside enucleated eyes) were demonstrated. Both photosensitive retinal ganglion cell responses and spontaneous retina waves were recorded and their features analyzed. Significance. Results of this study demonstrated soft electrode arrays with unique properties, suitable for simultaneous recording and photo-electrical stimulation of the retina at high fidelity. This novel electrode technology opens up new frontiers in the study of neural tissue in vivo.

Objective. Neural interfaces are an essential tool to enable the human body to directly communicate with machines such as computers or prosthetic robotic arms. Since invasive electrodes can be located closer to target neurons, they have advantages such as precision in stimulation and high signal-to-noise ratio (SNR) in recording, while they often exhibit unstable performance in long-term in-vivo implantation because of the tissue damage caused by the electrodes insertion. In the present study, we investigated the electrical functionality of flexible penetrating microelectrode arrays (FPMAs) up to 3 months in in-vivo conditions. Approach. The in-vivo experiment was performed by implanting FPMAs in five rats. The in-vivo impedance as well as the action potential (AP) amplitude and SNR were analyzed over weeks. Additionally, APs were tracked over time to investigate the possibility of single neuron recording. Main results. It was observed that the FPMAs exhibited dramatic increases in impedance for the first 4 weeks after implantation, accompanied by decreases in AP amplitude. However, the increase/decrease in AP amplitude was always accompanied by the increase/decrease in background noise, resulting in quite consistently maintained SNRs. After 4 weeks of implantation, we observed two distinctive issues regarding long-term implantation, each caused by chronic tissue responses or by the delamination of insulation layer. The results demonstrate that the FPMAs successfully recorded neuronal signals up to 12 weeks, with very stably maintained SNRs, reduced by only 16.1% on average compared to the first recordings, although biological tissue reactions or physical degradation of the FPMA were present. Significance. The fabricated FPMAs successfully recorded intracortical signals for 3 months. The SNR was maintained up to 3 months and the chronic function of FPMA was comparable with other silicon based implantable electrodes.

Objective. Brain–machine interfaces (BMIs) have the potential to augment human functions and restore independence in people with disabilities, yet a compromise between non-invasiveness and performance limits their relevance. Approach. Here, we hypothesized that a non-invasive neuromuscular–machine interface providing real-time neurofeedback of individual motor units within a muscle could enable independent motor unit control to an extent suitable for high-performance BMI applications. Main results. Over 6 days of training, eight participants progressively learned to skillfully and independently control three biceps brachii motor units to complete a 2D center-out task. We show that neurofeedback enabled motor unit activity that largely violated recruitment constraints observed during ramp-and-hold isometric contractions thought to limit individual motor unit controllability. Finally, participants demonstrated the suitability of individual motor units for powering general applications through a spelling task. Significance. These results illustrate the flexibility of the sensorimotor system and highlight individual motor units as a promising source of control for BMI applications.

Objective. Neuromodulation of visceral nerves is being intensively studied for treating a wide range of conditions, but effective translation requires increasing the efficacy and predictability of neural interface performance. Here we use computational models of rat visceral nerve to predict how neuroanatomical variability could affect both electrical stimulation and recording with an experimental planar neural interface. Approach. We developed a hybrid computational pipeline, Visceral Nerve Ensemble Recording and Stimulation (ViNERS), to couple finite-element modelling of extracellular electrical fields with biophysical simulations of individual axons. Anatomical properties of fascicles and axons in rat pelvic and vagus nerves were measured or obtained from public datasets. To validate ViNERS, we simulated pelvic nerve stimulation and recording with an experimental four-electrode planar array. Main results. Axon diameters measured from pelvic nerve were used to model a population of myelinated and unmyelinated axons and simulate recordings of electrically evoked single-unit field potentials (SUFPs). Across visceral nerve fascicles of increasing size, our simulations predicted an increase in stimulation threshold and a decrease in SUFP amplitude. Simulated threshold changes were dominated by changes in perineurium thickness, which correlates with fascicle diameter. We also demonstrated that ViNERS could simulate recordings of electrically-evoked compound action potentials (ECAPs) that were qualitatively similar to pelvic nerve recording made with the array used for simulation. Significance. We introduce ViNERS as a new open-source computational tool for modelling large-scale stimulation and recording from visceral nerves. ViNERS predicts how neuroanatomical variation in rat pelvic nerve affects stimulation and recording with an experimental planar electrode array. We show ViNERS can simulate ECAPS that capture features of our recordings, but our results suggest the underlying NEURON models need to be further refined and specifically adapted to accurately simulate visceral nerve axons.

Objective. The goal of this study is to decode the electrical activity of single neurons in the human subthalamic nucleus (STN) to infer the speech features that a person articulated, heard or imagined. We also aim to evaluate the amount of subthalamic neurons required for high accuracy decoding suitable for real-life speech brain-machine interfaces (BMI). Approach. We intraoperatively recorded single-neuron activity in the STN of 21 neurosurgical patients with Parkinson's disease undergoing implantation of deep brain stimulator while patients produced, perceived or imagined the five monophthongal vowel sounds. Our decoder is based on machine learning algorithms that dynamically learn specific features of the speech-related firing patterns. Main results. In an extensive comparison of algorithms, our sparse decoder ('SpaDe'), based on sparse decomposition of the high dimensional neuronal feature space, outperformed the other algorithms in all three conditions: production, perception and imagery. For speech production, our algorithm, Spade, predicted all vowels correctly (accuracy: 100%; chance level: 20%). For perception accuracy was 96%, and for imagery: 88%. The accuracy of Spade showed a linear behavior in the amount of neurons for the perception data, and even faster for production or imagery. Significance. Our study demonstrates that the information encoded by single neurons in the STN about the production, perception and imagery of speech is suitable for high-accuracy decoding. It is therefore an important step towards BMIs for restoration of speech faculties that bears an enormous potential to alleviate the suffering of completely paralyzed ('locked-in') patients and allow them to communicate again with their environment. Moreover, our research indicates how many subthalamic neurons may be necessary to achieve each level of decoding accuracy, which is of supreme importance for a neurosurgeon planning the implantation of a speech BMI.

Objective. With the development in the field of neural networks, explainable AI (XAI), is being studied to ensure that artificial intelligence models can be explained. There are some attempts to apply neural networks to neuroscientific studies to explain neurophysiological information with high machine learning performances. However, most of those studies have simply visualized features extracted from XAI and seem to lack an active neuroscientific interpretation of those features. In this study, we have tried to actively explain the high-dimensional learning features contained in the neurophysiological information extracted from XAI, compared with the previously reported neuroscientific results. Approach. We designed a deep neural network classifier using 3D information (3D DNN) and a 3D class activation map (3D CAM) to visualize high-dimensional classification features. We used those tools to classify monkey electrocorticogram (ECoG) data obtained from the unimanual and bimanual movement experiment. Main results. The 3D DNN showed better classification accuracy than other machine learning techniques, such as 2D DNN. Unexpectedly, the activation weight in the 3D CAM analysis was high in the ipsilateral motor and somatosensory cortex regions, whereas the gamma-band power was activated in the contralateral areas during unimanual movement, which suggests that the brain signal acquired from the motor cortex contains information about both contralateral movement and ipsilateral movement. Moreover, the hand-movement classification system used critical temporal information at movement onset and offset when classifying bimanual movements. Significance. As far as we know, this is the first study to use high-dimensional neurophysiological information (spatial, spectral, and temporal) with the deep learning method, reconstruct those features, and explain how the neural network works. We expect that our methods can be widely applied and used in neuroscience and electrophysiology research from the point of view of the explainability of XAI as well as its performance.

Objective. Brain-computer Interfaces (BCI) with functional electrical stimulation (FES) as a feedback device might promote neuroplasticity and hence improve motor function. Novel findings suggested that neuroplasticity could be possible in people with multiple sclerosis (pwMS). This preliminary study explores the effects of using a BCI-FES in therapeutic intervention, as an emerging methodology for gait rehabilitation in pwMS. Approach. People with relapsing-remitting, primary progressive or secondary progressive MS were evaluated with the inclusion criteria to enroll the nine participants required by the statistically computed sample size. Each patient trained with a BCI-FES during 24 sessions distributed in eight weeks. The effects were evaluated on gait speed (Timed 25 Foot Walk), walking ability (12-item Multiple Sclerosis Walking Scale), quality of life measures, the true positive rate as the BCI-FES performance metric and the event-related desynchronization (ERD) onset latency of the sensorimotor rhythms. Main results. Seven patients completed the therapeutic intervention. A statistically and clinically significant post-treatment improvement was observed in gait speed, as a result of a reduction in the time to walk 25 feet (−1.99 s, p = 0.018), and walking ability (−31.25 score points, p = 0.028). The true positive rate showed a statistically significant improvement (+15.87 score points, p = 0.018). An earlier ERD onset latency (−180 ms) after treatment was found. Significance. This is the first study that explored gait rehabilitation using BCI-FES in pwMS. The results showed improvement in gait which might have been promoted by changes in functional brain connections involved in sensorimotor rhythm modulation. Although more studies with a larger sample size and control group are required to validate the efficacy of this approach, these results suggest that BCI-FES technology could have a positive effect on MS gait rehabilitation.

Objective. Present methods for assessing color vision require the person's active participation. Here we describe a brain-computer interface-based method for assessing color vision that does not require the person's participation. Approach. This method uses steady-state visual evoked potentials to identify metamers—two light sources that have different spectral distributions but appear to the person to be the same color. Main results. We demonstrate that: minimization of the visual evoked potential elicited by two flickering light sources identifies the metamer; this approach can distinguish people with color-vision deficits from those with normal color vision; and this metamer-identification process can be automated. Significance. This new method has numerous potential clinical, scientific, and industrial applications.

Objective. Heterogeneous clinical responses to treatment with non-invasive brain stimulation are commonly observed, making it necessary to determine personally optimized stimulation parameters. We investigated neuroimaging markers of effective brain targets of treatment with continuous theta burst stimulation (cTBS) in mal de débarquement syndrome (MdDS), a balance disorder of persistent oscillating vertigo previously shown to exhibit abnormal intrinsic functional connectivity. Approach. Twenty-four right-handed, cTBS-naive individuals with MdDS received single administrations of cTBS over one of three stimulation targets in randomized order. The optimal target was determined based on the assessment of acute changes after the administration of cTBS over each target. Repetitive cTBS sessions were delivered on three consecutive days with the optimal target chosen by the participant. Electroencephalography (EEG) was recorded at single-administration test sessions of cTBS. Simultaneous EEG and functional MRI data were acquired at baseline and after completion of 10–12 sessions. Network connectivity changes after single and repetitive stimulations of cTBS were analyzed. Main results. Using electrophysiological source imaging and a data-driven method, we identified network-level connectivity changes in EEG that correlated with symptom responses after completion of multiple sessions of cTBS. We further determined that connectivity changes demonstrated by EEG during test sessions of single administrations of cTBS were signatures that could predict optimal targets. Significance. Our findings demonstrate the effect of cTBS on resting state brain networks and suggest an imaging-based, closed-loop stimulation paradigm that can identify optimal targets during short-term test sessions of stimulation.

ClinicalTrials.gov Identifier: NCT02470377.

Objective. Recording and stimulating neuronal activity across different brain regions requires interfacing at multiple sites using dedicated tools while tissue reactions at the recording sites often prevent their successful long-term application. This implies the technological challenge of developing complex probe geometries while keeping the overall footprint minimal, and of selecting materials compatible with neural tissue. While the potential of soft materials in reducing tissue response is uncontested, the implantation of these materials is often limited to reliably target neuronal structures across large brain volumes. Approach. We report on the development of a new multi-electrode array exploiting the advantages of soft and stiff materials by combining 7-µm-thin polyimide wings carrying platinum electrodes with a silicon backbone enabling a safe probe implantation. The probe fabrication applies microsystems technologies in combination with a temporal wafer fixation method for rear side processing, i.e. grinding and deep reactive ion etching, of slender probe shanks and electrode wings. The wing-type neural probes are chronically implanted into the entorhinal-hippocampal formation in the mouse for in vivo recordings of freely behaving animals. Main results. Probes comprising the novel wing-type electrodes have been realized and characterized in view of their electrical performance and insertion capability. Chronic electrophysiological in vivo recordings of the entorhinal-hippocampal network in the mouse of up to 104 days demonstrated a stable yield of channels containing identifiable multi-unit and single-unit activity outperforming probes with electrodes residing on a Si backbone. Significance. The innovative fabrication process using a process compatible, temporary wafer bonding allowed to realize new Michigan-style probe arrays. The wing-type probe design enables a precise probe insertion into brain tissue and long-term stable recordings of unit activity due to the application of a stable backbone and 7-µm-thin probe wings provoking locally a minimal tissue response and protruding from the glial scare of the backbone.

Objective. Biomimetic protein-based artificial retinas offer a new paradigm for restoring vision for patients blinded by retinal degeneration. Artificial retinas, comprised of an ion-permeable membrane and alternating layers of bacteriorhodopsin (BR) and a polycation binder, are assembled using layer-by-layer electrostatic adsorption. Upon light absorption, the oriented BR layers generate a unidirectional proton gradient. The main objective of this investigation is to demonstrate the ability of the ion-mediated subretinal artificial retina to activate retinal ganglion cells (RGCs) of degenerated retinal tissue. Approach. Ex vivo extracellular recording experiments with P23H line 1 rats are used to measure the response of RGCs following selective stimulation of our artificial retina using a pulsed light source. Single-unit recording is used to evaluate the efficiency and latency of activation, while a multielectrode array (MEA) is used to assess the spatial sensitivity of the artificial retina films. Main results. The activation efficiency of the artificial retina increases with increased incident light intensity and demonstrates an activation latency of ∼150 ms. The results suggest that the implant is most efficient with 200 BR layers and can stimulate the retina using light intensities comparable to indoor ambient light. Results from using an MEA show that activation is limited to the targeted receptive field. Significance. The results of this study establish potential effectiveness of using an ion-mediated artificial retina to restore vision for those with degenerative retinal diseases, including retinitis pigmentosa.

Objective. The efficacy of an adopted feature extraction method directly affects the classification of the electromyographic (EMG) signals in myoelectric control applications. Most methods attempt to extract the dynamics of the multi-channel EMG signals in the time domain and on a channel-by-channel, or at best pairs of channels, basis. However, considering multi-channel information to build a similarity matrix has not been taken into account. Approach. Combining methods of long and short-term memory (LSTM) and dynamic temporal warping, we developed a new feature, called spatio-temporal warping (STW), for myoelectric signals. This method captures the spatio-temporal relationships of multi-channels EMG signals. Main results. Across four online databases, we show that in terms of average classification error and standard deviation values, the STW feature outperforms traditional features by 5%–17%. In comparison to the more recent deep learning models, e.g. convolutional neural networks (CNNs), STW outperformed by 5%–18%. Also, STW showed enhanced performance when compared to the CNN + LSTM model by 2%–14%. All differences were statistically significant with a large effect size. Significance. This feasibility study provides evidence supporting the hypothesis that the STW feature of the EMG signals can enhance the classification accuracy in an explainable way when compared to recent deep learning methods. Future work includes real-time implementation of the method and testing for prosthesis control.

Objective. Proprioceptive information plays an important role for recognizing and coordinating our limb's static and dynamic states relative to our body or the environment. In this study, we determined how artificially evoked proprioceptive feedback affected the continuous control of a prosthetic finger. Approach. We elicited proprioceptive information regarding the joint static position and dynamic movement of a prosthetic finger via a vibrotactor array placed around the subject's upper arm. Myoelectric signals of the finger flexor and extensor muscles were used to control the prosthesis, with or without the evoked proprioceptive feedback. Two control modes were evaluated: the myoelectric signal amplitudes were continuously mapped to either the position or the velocity of the prosthetic joint. Main results. Our results showed that the evoked proprioceptive information improved the control accuracy of the joint angle, with comparable performance in the position- and velocity-control conditions. However, greater angle variability was prominent during position-control than velocity-control. Without the proprioceptive feedback, the position-control tended to show a smaller angle error than the velocity-control condition. Significance. Our findings suggest that closed-loop control of a prosthetic device can potentially be achieved using non-invasive evoked proprioceptive feedback delivered to intact participants. Moreover, the evoked sensory information was integrated during myoelectric control effectively for both control strategies. The outcomes can facilitate our understanding of the sensorimotor integration process during human-machine interactions, which can potentially promote fine control of prosthetic hands.

Objective. In this work we adapted a protocol for the fast generation of human neurons to build 3D neuronal networks with controlled structure and cell composition suitable for systematic electrophysiological investigations. Approach. We used biocompatible chitosan microbeads as scaffold to build 3D networks and to ensure nutrients-medium exchange from the core of the structure to the external environment. We used excitatory neurons derived from human-induced pluripotent stem cells (hiPSCs) co-cultured with astrocytes. By adapting the well-established NgN2 differentiation protocol, we obtained 3D engineered networks with good control over cell density, volume and cell composition. We coupled the 3D neuronal networks to 60-channel micro electrode arrays (MEAs) to monitor and characterize their electrophysiological development. In parallel, we generated two-dimensional neuronal networks cultured on chitosan to compare the results of the two models. Main results. We sustained samples until 60 d in vitro (DIV) and 3D cultures were healthy and functional. From the structural point of view, the hiPSC derived neurons were able to adhere to chitosan microbeads and to form a stable 3D assembly thanks to the connections among cells. From a functional point of view, neuronal networks showed spontaneous activity after a couple of weeks. Significance. We presented a particular method to generate 3D engineered cultures for the first time with human-derived neurons coupled to MEAs, overcoming some of the limitations related to 2D and 3D neuronal networks and thus increasing the therapeutic target potential of these models for biomedical applications.

Objective. Closed-loop deep brain stimulation (DBS) with neural feedback has shown great potential in improving the therapeutic effect and reducing side effects. However, the amplitude of stimulation artifacts is much larger than the local field potentials, which remains a bottleneck in developing a closed-loop stimulation strategy with varied parameters. Approach. We proposed an irregular sampling method for the real-time removal of stimulation artifacts. The artifact peaks were detected by applying a threshold to the raw recordings, and the samples within the contaminated period of the stimulation pulses were excluded and replaced with the interpolation of the samples prior to and after the stimulation artifact duration. This method was evaluated with both simulation signals and in vivo closed-loop DBS applications in Parkinsonian animal models. Main results. The irregular sampling method was able to remove the stimulation artifacts effectively with the simulation signals. The relative errors between the power spectral density of the recovered and true signals within a wide frequency band (2–150 Hz) were 2.14%, 3.93%, 7.22%, 7.97% and 6.25% for stimulation at 20 Hz, 60 Hz, 130 Hz, 180 Hz, and stimulation with variable low and high frequencies, respectively. This stimulation artifact removal method was verified in real-time closed-loop DBS applications in vivo, and the artifacts were effectively removed during stimulation with frequency continuously changing from 130 Hz to 1 Hz and stimulation adaptive to beta oscillations. Significance. The proposed method provides an approach for real-time removal in closed-loop DBS applications, which is effective in stimulation with low frequency, high frequency, and variable frequency. This method can facilitate the development of more advanced closed-loop DBS strategies.

Objective. Alzheimer's disease (AD), a common disease of the elderly with unknown etiology, has been adversely affecting many people, especially with the aging of the population and the younger trend of this disease. Current artificial intelligence (AI) methods based on individual information or magnetic resonance imaging (MRI) can solve the problem of diagnostic sensitivity and specificity, but still face the challenges of interpretability and clinical feasibility. In this study, we propose an interpretable multimodal deep reinforcement learning model for inferring pathological features and the diagnosis of AD. Approach. First, for better clinical feasibility, the compressed-sensing MRI image is reconstructed using an interpretable deep reinforcement learning model. Then, the reconstructed MRI is input into the full convolution neural network to generate a pixel-level disease probability risk map (DPM) of the whole brain for AD. The DPM of important brain regions and individual information are then input into the attention-based fully deep neural network to obtain the diagnosis results and analyze the biomarkers. We used 1349 multi-center samples to construct and test the model. Main results. Finally, the model obtained 99.6% ± 0.2%, 97.9% ± 0.2%, and 96.1% ± 0.3% area under curve in ADNI, AIBL and NACC, respectively. The model also provides an effective analysis of multimodal pathology, predicts the imaging biomarkers in MRI and the weight of each individual item of information. In this study, a deep reinforcement learning model was designed, which can not only accurately diagnose AD, but analyze potential biomarkers. Significance. In this study, a deep reinforcement learning model was designed. The model builds a bridge between clinical practice and AI diagnosis and provides a viewpoint for the interpretability of AI technology.

Objective. To understand neural circuit dynamics, it is critical to manipulate and record many individual neurons. Traditional recording methods, such as glass microelectrodes, can only control a small number of neurons. More recently, devices with high electrode density have been developed, but few of them can be used for intracellular recording or stimulation in intact nervous systems. Carbon fiber electrodes (CFEs) are 8 µm-diameter electrodes that can be assembled into dense arrays (pitches ⩾ 80 µm). They have good signal-to-noise ratios (SNRs) and provide stable extracellular recordings both acutely and chronically in neural tissue in vivo (e.g. rat motor cortex). The small fiber size suggests that arrays could be used for intracellular stimulation. Approach. We tested CFEs for intracellular stimulation using the large identified and electrically compact neurons of the marine mollusk Aplysia californica. Neuron cell bodies in Aplysia range from 30 µm to over 250 µm. We compared the efficacy of CFEs to glass microelectrodes by impaling the same neuron's cell body with both electrodes and connecting them to a DC coupled amplifier. Main results. We observed that intracellular waveforms were essentially identical, but the amplitude and SNR in the CFE were lower than in the glass microelectrode. CFE arrays could record from 3 to 8 neurons simultaneously for many hours, and many of these recordings were intracellular, as shown by simultaneous glass microelectrode recordings. CFEs coated with platinum-iridium could stimulate and had stable impedances over many hours. CFEs not within neurons could record local extracellular activity. Despite the lower SNR, the CFEs could record synaptic potentials. CFEs were less sensitive to mechanical perturbations than glass microelectrodes. Significance. The ability to do stable multi-channel recording while stimulating and recording intracellularly make CFEs a powerful new technology for studying neural circuit dynamics.

Objective. Retinal implants have been developed to electrically stimulate healthy retinal neurons in the progressively degenerated retina. Several stimulation approaches have been proposed to improve the visual percept induced in patients with retinal prostheses. We introduce a computational model capable of simulating the effects of electrical stimulation on retinal neurons. Leveraging this computational platform, we delve into the underlying mechanisms influencing the sensitivity of retinal neurons' response to various stimulus waveforms. Approach. We implemented a model of spiking bipolar cells (BCs) in the magnocellular pathway of the primate retina, diffuse BC subtypes (DB4), and utilized our multiscale admittance method (AM)-NEURON computational platform to characterize the response of BCs to epiretinal electrical stimulation with monophasic, symmetric, and asymmetric biphasic pulses. Main results. Our investigations yielded four notable results: (a) the latency of BCs increases as stimulation pulse duration lengthens; conversely, this latency decreases as the current amplitude increases. (b) Stimulation with a long anodic-first symmetric biphasic pulse (duration > 8 ms) results in a significant decrease in spiking threshold compared to stimulation with similar cathodic-first pulses (from 98.2 to 57.5 µA). (c) The hyperpolarization-activated cyclic nucleotide-gated channel was a prominent contributor to the reduced threshold of BCs in response to long anodic-first stimulus pulses. (d) Finally, extending the study to asymmetric waveforms, our results predict a lower BCs threshold using asymmetric long anodic-first pulses compared to that of asymmetric short cathodic-first stimulation. Significance. This study predicts the effects of several stimulation parameters on spiking BCs response to electrical stimulation. Of importance, our findings shed light on mechanisms underlying the experimental observations from the literature, thus highlighting the capability of the methodology to predict and guide the development of electrical stimulation protocols to generate a desired biological response, thereby constituting an ideal testbed for the development of electroceutical devices.

Objective. Brain–machine interfaces are key components for the development of hands-free, brain-controlled devices. Electroencephalogram (EEG) electrodes are particularly attractive for harvesting the neural signals in a non-invasive fashion. Approach. Here, we explore the use of epitaxial graphene (EG) grown on silicon carbide on silicon for detecting the EEG signals with high sensitivity. Main results and significance. This dry and non-invasive approach exhibits a markedly improved skin contact impedance when benchmarked to commercial dry electrodes, as well as superior robustness, allowing prolonged and repeated use also in a highly saline environment. In addition, we report the newly observed phenomenon of surface conditioning of the EG electrodes. The prolonged contact of the EG with the skin electrolytes functionalize the grain boundaries of the graphene, leading to the formation of a thin surface film of water through physisorption and consequently reducing its contact impedance more than three-fold. This effect is primed in highly saline environments, and could be also further tailored as pre-conditioning to enhance the performance and reliability of the EG sensors.

Objective. Parkinson's disease (PD) is one of the most common neurodegenerative diseases, and early diagnosis is crucial to delay disease progression. The diagnosis of early PD has always been a difficult clinical problem due to the lack of reliable biomarkers. Electroencephalogram (EEG) is the most common clinical detection method, and studies have attempted to discover the EEG spectrum characteristics of early PD, but the reported conclusions are not uniform due to the heterogeneity of early PD patients. There is an urgent need for a more advanced algorithm to extract spectrum characteristics from EEG to satisfy the personalized requirements. Approach. The structured power spectral density with spatial distribution was used as the input of convolutional neural network (CNN). A visualization technique called gradient-weighted class activation mapping was used to extract the optimal frequency bands for identifying early PD. Based on the model visualization, we proposed a novel quantitative index of spectral characteristics, spatial-mapping relative power (SRP), to detect personalized abnormalities in the spatial spectral characteristics of EEG in early PD. Main results. We demonstrated the feasibility of applying CNN to identify the patients with early PD with an accuracy of 99.87% ± 0.03%. The models indicated the characteristic frequency bands (high-delta (3.5–4.5 Hz) and low-alpha (7.5–11 Hz) frequency bands) that are used to identify the early PD. The SRP of these two characteristic bands in early PD patients was significantly higher than that in the control group, and the abnormalities were consistent at the group and individual levels. Significance. This study provides a novel personalized detection algorithm based on deep learning to reveal the optimal frequency bands for identifying early PD and obtain the spatial frequency characteristics of early PD. The findings of this study will provide an effective reference for the auxiliary diagnosis of early PD in clinical practice.

Objective. Previous studies have demonstrated that ultrasound thalamic stimulation (UTS) can treat disorders of consciousness. However, it is still unclear how UTS modulates neural activity in the thalamus and cortex. Approach. In this study, we performed UTS in mice and recorded the neural activities including spike and local field potential (LFP) of the thalamus and motor cortex (M1). We analyzed the firing rate of spikes and the power spectrum of LFPs and evaluated the coupling relationship between LFPs from the thalamus and M1 with Granger causality. Main results. Our results clearly indicate that UTS can directly induce neural activity in the thalamus and indirectly induce neural activity in the M1. We also found that there is a strong connection relationship of neural activity between thalamus and M1 under UTS. Significance. These results demonstrate that UTS can modulate the neural activity of the thalamus and M1 in mice. It has the potential to provide guidance for the ultrasound treatment of thalamus-related diseases.

Objective. The blue light-activated inhibitory opsin, stGtACR2, is gaining prominence as a neuromodulatory tool due its ability to shunt-inhibit neurons and is being frequently used in in vivo experimentation. However, experiments involving stGtACR2 use longer durations of blue light pulses, which inadvertently heat up the local brain tissue and confound experimental results. Therefore, the heating effects of illumination parameters used for in vivo optogenetic inhibition must be evaluated. Approach. To assess blue light (473 nm)-induced heating of the brain, we used a computational model as well as direct temperature measurements using a fiber Bragg grating (FBG). The effects of different light power densities (LPDs) and pulse durations on evoked potentials (EP) recorded from dentate gyrus were assessed. For opsin-negative rats, LPDs between 127 and 636 mW mm⁻² and pulse durations between 20 and 5120 ms were tested while for stGtACR2 expressing rats, LPD of 127 mW mm⁻² and pulse durations between 20 and 640 ms were tested. Main results. Increasing LPDs and pulse durations logarithmically increased the peak temperature and significantly decreased the population spike (PS) amplitude and latencies of EPs. For a pulse duration of 5120 ms, the tissue temperature increased by 0.6 °C–3.4 °C. All tested LPDs decreased the PS amplitude in opsin-negative rats, but 127 mW mm⁻² had comparatively minimal effects and a significant effect of increasing light pulse duration was seen from 320 ms and beyond. This corresponded with an average temperature increase of 0.2 °C–1.1 °C at the recorded site. Compared to opsin-negative rats, illumination in stGtACR2-expressing rats resulted in much greater inhibition of EPs. Significance. Our study demonstrates that light-induced heating of the brain can be accurately measured in vivo using FBG sensors. Such light-induced heating alone can affect neuronal excitability. Useful neuromodulation by the activation of stGtACR2 is still possible while minimizing thermal effects.

Objective. Interictal epileptiform discharges (IEDs) occur between two seizures onsets. IEDs are mainly captured by intracranial recordings and are often invisible over the scalp. This study proposes a model based on tensor factorization to map the time-frequency (TF) features of scalp EEG (sEEG) to the TF features of intracranial EEG (iEEG) in order to detect IEDs from over the scalp with high sensitivity. Approach. Continuous wavelet transform is employed to extract the TF features. Time, frequency, and channel modes of IED segments from iEEG recordings are concatenated into a four-way tensor. Tucker and CANDECOMP/PARAFAC decomposition techniques are employed to decompose the tensor into temporal, spectral, spatial, and segmental factors. Finally, TF features of both IED and non-IED segments from scalp recordings are projected onto the temporal components for classification. Main results. The model performance is obtained in two different approaches: within- and between-subject classification approaches. Our proposed method is compared with four other methods, namely a tensor-based spatial component analysis method, TF-based method, linear regression mapping model, and asymmetric–symmetric autoencoder mapping model followed by convolutional neural networks. Our proposed method outperforms all these methods in both within- and between-subject classification approaches by respectively achieving 84.2% and 72.6% accuracy values. Significance. The findings show that mapping sEEG to iEEG improves the performance of the scalp-based IED detection model. Furthermore, the tensor-based mapping model outperforms the autoencoder- and regression-based mapping models.

Objective. Steady-state visual evoked potential (SSVEP) based brain–computer interface (BCI) has the characteristics of fast communication speed, high stability, and wide applicability, thus it has been widely studied. With the rapid development in paradigm, algorithm, and system design, SSVEP-BCI is gradually applied in clinical and real-life scenarios. In order to improve the ease of use of the SSVEP-BCI system, many studies have been focusing on developing it on the hairless area, but due to the lack of redesigning the stimulation paradigm to better adapt to the new area, the electroencephalography response in the hairless area is worse than occipital region. Approach. This study first proposed a phase difference estimation method based on stimulating the left and right visual field separately, then developed and optimized a left and right visual field biphasic stimulation paradigm for SSVEP-based BCIs with hairless region behind the ear. Main results. In the 12-target online experiment, after a short model estimation training, all 16 subjects used their best stimulus condition. The paradigm designed in this study can increase the proportion of applicable subjects for the behind-ear SSVEP-BCI system from 58.3% to 75% and increase the accuracy from 74.6 ± 20.0% (the existing best SSVEP stimulus with hairless region behind the ear) to 84.2$\pm$ 14.7%, and the information transfer rate from 14.2$\pm$6.4 bits min⁻¹ to 17.8$\pm$5.7 bits min⁻¹. Significance. These results demonstrated that the proposed paradigm can effectively improve the BCI performance using the signal from the hairless region behind the ear, compared with the standard SSVEP stimulation paradigm.

Objective. Electroencephalography (EEG) is a widely used technique to address research questions about brain functioning, from controlled laboratorial conditions to naturalistic environments. However, EEG data are affected by biological (e.g. ocular, myogenic) and non-biological (e.g. movement-related) artifacts, which—depending on their extent—may limit the interpretability of the study results. Blind source separation (BSS) approaches have demonstrated to be particularly promising for the attenuation of artifacts in high-density EEG (hdEEG) data. Previous EEG artifact removal studies suggested that it may not be optimal to use the same BSS method for different kinds of artifacts. Approach. In this study, we developed a novel multi-step BSS approach to optimize the attenuation of ocular, movement-related and myogenic artifacts from hdEEG data. For validation purposes, we used hdEEG data collected in a group of healthy participants in standing, slow-walking and fast-walking conditions. During part of the experiment, a series of tone bursts were used to evoke auditory responses. We quantified event-related potentials (ERPs) using hdEEG signals collected during an auditory stimulation, as well as the event-related desynchronization (ERD) by contrasting hdEEG signals collected in walking and standing conditions, without auditory stimulation. We compared the results obtained in terms of auditory ERP and motor-related ERD using the proposed multi-step BSS approach, with respect to two classically used single-step BSS approaches. Main results. The use of our approach yielded the lowest residual noise in the hdEEG data, and permitted to retrieve stronger and more reliable modulations of neural activity than alternative solutions. Overall, our study confirmed that the performance of BSS-based artifact removal can be improved by using specific BSS methods and parameters for different kinds of artifacts. Significance. Our technological solution supports a wider use of hdEEG-based source imaging in movement and rehabilitation studies, and contributes to the further development of mobile brain/body imaging applications.

Objective. There has become of increasing interest in transcranial alternating current stimulation (tACS) since its inception nearly a decade ago. tACS in modulating brain state is an active area of research and has been demonstrated effective in various neuropsychological and clinical domains. In the visual domain, much effort has been dedicated to brain rhythms and rhythmic stimulation, i.e. tACS. However, less is known about the interplay between the rhythmic stimulation and visual stimulation. Approach. Here, we used steady-state visual evoked potential (SSVEP), induced by flickering driving as a widely used technique for frequency-tagging, to investigate the aftereffect of tACS in healthy human subjects. Seven blocks of 64-channel electroencephalogram were recorded before and after the administration of 20min 10Hz tACS, while subjects performed several blocks of SSVEP tasks. We characterized the physiological properties of tACS aftereffect by comparing and validating the temporal, spatial, spatiotemporal and signal-to-noise ratio (SNR) patterns between and within blocks in real tACS and sham tACS. Main results. Our result revealed that tACS boosted the 10Hz SSVEP significantly. Besides, the aftereffect on SSVEP was mitigated with time and lasted up to 5 min. Significance. Our results demonstrate the feasibility of facilitating the flickering driving by external rhythmic stimulation and open a new possibility to alter the brain state in a direction by noninvasive transcranial brain stimulation.

Objective. To investigate computationally the interaction of combined electrical and ultrasonic modulation of isolated neurons and of the parkinsonian cortex-basal ganglia-thalamus loop. Approach. Continuous-wave or pulsed electrical and ultrasonic neuromodulation is applied to isolated Otsuka plateau-potential generating subthalamic nucleus (STN) and Pospischil regular, fast and low-threshold spiking cortical cells in a temporally alternating or simultaneous manner. Similar combinations of electrical/ultrasonic waveforms are applied to a parkinsonian biophysical cortex-basal ganglia-thalamus neuronal network. Ultrasound-neuron interaction is modelled respectively for isolated neurons and the neuronal network with the NICE and SONIC implementations of the bilayer sonophore underlying mechanism. Reduction in $\alpha-\beta$ spectral energy is used as a proxy to express improvement in Parkinson's disease by insonication and electrostimulation. Main results. Simultaneous electro-acoustic stimulation achieves a given level of neuronal activity at lower intensities compared to the separate stimulation modalities. Conversely, temporally alternating stimulation with $50~\mathrm{Hz}$ electrical and ultrasound pulses is capable of eliciting $100~\mathrm{Hz}$ STN firing rates. Furthermore, combination of ultrasound with hyperpolarizing currents can alter cortical cell relative spiking regimes. In the parkinsonian neuronal network, continuous-wave and pulsed ultrasound reduce pathological oscillations by different mechanisms. High-frequency pulsed separated electrical and ultrasonic deep brain stimulation (DBS) reduce pathological $\alpha-\beta$ power by entraining STN-neurons. In contrast, continuous-wave ultrasound reduces pathological oscillations by silencing the STN. Compared to the separated stimulation modalities, temporally simultaneous or alternating electro-acoustic stimulation can achieve higher reductions in $\alpha-\beta$ power for the same safety contraints on electrical/ultrasonic intensity. Significance. Focused ultrasound has the potential of becoming a non-invasive alternative of conventional DBS for the treatment of Parkinson's disease. Here, we elaborate on proposed benefits of combined electro-acoustic stimulation in terms of improved dynamic range, efficiency, spatial resolution, and neuronal selectivity.

Objective. Microelectrode arrays are standard tools for conducting chronic electrophysiological experiments, allowing researchers to simultaneously record from large numbers of neurons. Specifically, Utah electrode arrays (UEAs) have been utilized by scientists in many species, including rodents, rhesus macaques, marmosets, and human participants. The field of clinical human brain-computer interfaces currently relies on the UEA as a number of research groups have clearance from the United States Federal Drug Administration (FDA) for this device through the investigational device exemption pathway. Despite its widespread usage in systems neuroscience, few studies have comprehensively evaluated the reliability and signal quality of the Utah array over long periods of time in a large dataset. Approach. We collected and analyzed over 6000 recorded datasets from various cortical areas spanning almost nine years of experiments, totaling 17 rhesus macaques (Macaca mulatta) and 2 human subjects, and 55 separate microelectrode Utah arrays. The scale of this dataset allowed us to evaluate the average life of these arrays, based primarily on the signal-to-noise ratio of each electrode over time. Main results. Using implants in primary motor, premotor, prefrontal, and somatosensory cortices, we found that the average lifespan of available recordings from UEAs was 622 days, although we provide several examples of these UEAs lasting over 1000 days and one up to 9 years; human implants were also shown to last longer than non-human primate implants. We also found that electrode length did not affect longevity and quality, but iridium oxide metallization on the electrode tip exhibited superior yield as compared to platinum metallization. Significance. Understanding longevity and reliability of microelectrode array recordings allows researchers to set expectations and plan experiments accordingly and maximize the amount of high-quality data gathered. Our results suggest that one can expect chronic unit recordings to last at least two years, with the possibility for arrays to last the better part of a decade.

Objective. Recent technological advances are revealing the complex physiology of the axon and challenging long-standing assumptions. Namely, while most action potential (AP) initiation occurs at the axon initial segment in central nervous system neurons, initiation in distal parts of the axon has been reported to occur in both physiological and pathological conditions. The functional role of these ectopic APs, if exists, is still not clear, nor its impact on network activity dynamics. Approach. Using an electrophysiology platform specifically designed for assessing axonal conduction we show here for the first time regular and effective bidirectional axonal conduction in hippocampal and dorsal root ganglia cultures. We investigate and characterize this bidirectional propagation both in physiological conditions and after distal axotomy. Main results. A significant fraction of APs are not coming from the canonical synapse-dendrite-soma signal flow, but instead from signals originating at the distal axon. Importantly, antidromic APs may carry information and can have a functional impact on the neuron, as they consistently depolarize the soma. Thus, plasticity or gene transduction mechanisms triggered by soma depolarization can also be affected by these antidromic APs. Conduction velocity is asymmetrical, with antidromic conduction being slower than orthodromic. Significance. Altogether these findings have important implications for the study of neuronal function in vitro, reshaping our understanding on how information flows in neuronal cultures.

Objective. Motor neuroprostheses require the identification of stimulation protocols that effectively produce desired movements. Manual search for these protocols can be very time-consuming and often leads to suboptimal solutions, as several stimulation parameters must be personalized for each subject for a variety of target motor functions. Here, we present an algorithm that efficiently tunes peripheral intraneural stimulation protocols to elicit functionally relevant distal limb movements. Approach. We developed the algorithm using Bayesian optimization (BO) with multi-output Gaussian Processes (GPs) and defined objective functions based on coordinated muscle recruitment. We applied the algorithm offline to data acquired in rats for walking control and in monkeys for hand grasping control and compared different GP models for these two systems. We then performed a preliminary online test in a monkey to experimentally validate the functionality of our method. Main results. Offline, optimal intraneural stimulation protocols for various target motor functions were rapidly identified in both experimental scenarios. Using the model that performed best, the algorithm converged to stimuli that evoked functionally consistent movements with an average number of actions equal to 20% of the search space size in both the rat and monkey animal models. Online, the algorithm quickly guided the observations to stimuli that elicited functional hand gestures, although more selective motor outputs could have been achieved by refining the objective function used. Significance. These results demonstrate that BO can reliably and efficiently automate the tuning of peripheral neurostimulation protocols, establishing a translational framework to configure peripheral motor neuroprostheses in clinical applications. The proposed method can also potentially be applied to optimize motor functions using other stimulation modalities.

Objective. A novel flexible hydrogel electrode with a strong moisturizing ability was prepared for long-term electroencephalography (EEG) monitoring. Approach. The hydrogel was synthesized by polymerizing the N-acryloyl glycinamide monomer. And a proper amount of glycerin was added to the hydrogel to increase the moisture retention ability of the electrodes. The hydrogel shows high mechanical properties, and the liquid in the hydrogel produces a hydrating effect on the skin stratum corneum, which could decrease the contact impedance between skin and electrode. In addition, the installation of hydrogel electrode is very convenient, and the skin of the subject does not need to be abraded. Main results. Scanning electron microscope images show that there are a large number of micropores in the hydrogel, which provide storage space for water molecules. The average potential drift of the hydrogel electrode is relatively low (1.974 ± 0.560 µV min⁻¹). The average contact impedance of hydrogel electrode in forehead region and hair region are 6.43 ± 0.84 kΩ cm² and 13.15 ± 3.72 kΩ cm², respectively. The result of open/closed paradigm, steady-state visual evoked potentials, and P300 visual evoked potential show that hydrogel electrode has excellent performance. Compared with the hydrogel without glycerin, the moisture retention ability of hydrogel containing glycerin was greatly improved. Significance. Compared with standard Ag/AgCl wet electrode, hydrogel electrode is more convenient to install and has strong moisture retention ability, which makes it have great potential in daily life for long-term EEG recording.

Objective. The use of motor imagery (MI) in motor rehabilitation protocols has been increasingly investigated as a potential technique for enhancing traditional treatments, yielding better clinical outcomes. However, since MI performance can be challenging, practice is usually required. This demands appropriate training, actively engaging the MI-related brain areas, consequently enabling the user to properly benefit from it. The role of feedback is central for MI practice. Yet, assessing which underlying neural changes are feedback-specific or purely due to MI practice is still a challenging effort, mainly due to the difficulty in isolating their contributions. In this work, we aimed to assess functional connectivity (FC) changes following MI practice that are either extrinsic or specific to feedback. Approach. To achieve this, we investigated FC, using graph theory, in electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data, during MI performance and at resting-state (rs), respectively. Thirty healthy subjects were divided into three groups, receiving no feedback (control), 'false' feedback (sham) or actual neurofeedback (active). Participants underwent 12–13 hands-MI EEG sessions and pre- and post-MI training fMRI exams. Main results. Following MI practice, control participants presented significant increases in degree and in eigenvector centrality for occipital nodes at rs-fMRI scans, whereas sham-feedback produced similar effects, but to a lesser extent. Therefore, MI practice, by itself, seems to stimulate visual information processing mechanisms that become apparent during basal brain activity. Additionally, only the active group displayed decreases in inter-subject FC patterns, both during MI performance and at rs-fMRI. Significance. Hence, actual neurofeedback impacted FC by disrupting common inter-subject patterns, suggesting that subject-specific neural plasticity mechanisms become important. Future studies should consider this when designing experimental NFBT protocols and analyses.

Objective. Soft-robotic-assisted training may improve motor function during post-stroke recovery, but the underlying physiological changes are not clearly understood. We applied a single-session of intensive proprioceptive stimulation to stroke survivors using a soft robotic glove to delineate its short-term influence on brain functional activity and connectivity. Approach. In this study, we utilized task-based and resting-state functional magnetic resonance imaging (fMRI) to characterize the changes in different brain networks following a soft robotic intervention. Nine stroke patients with hemiplegic upper limb engaged in resting-state and motor-task fMRI. The motor tasks comprised two conditions: active movement of fingers (active task) and glove-assisted active movement using a robotic glove (glove-assisted task), both with visual instruction. Each task was performed using bilateral hands simultaneously or the affected hand only. The same set of experiments was repeated following a 30 min treatment of continuous passive motion (CPM) using a robotic glove. Main results. On simultaneous bimanual movement, increased activation of supplementary motor area (SMA) and primary motor area (M1) were observed after CPM treatment compared to the pre-treatment condition, both in active and glove-assisted task. However, when performing the tasks solely using the affected hand, the phenomena of increased activity were not observed either in active or glove-assisted task. The comparison of the resting-state fMRI between before and after CPM showed the connectivity of the supramarginal gyrus and SMA was increased in the somatosensory network and salience network. Significance. This study demonstrates how passive motion exercise activates M1 and SMA in the post-stroke brain. The effective proprioceptive motor integration seen in bimanual exercise in contrast to the unilateral affected hand exercise suggests that the unaffected hemisphere might reconfigure connectivity to supplement damaged neural networks in the affected hemisphere. The somatosensory modulation rendered by the intense proprioceptive stimulation would affect the motor learning process in stroke survivors.

Objective. Brain–computer interface (BCI) systems decode electroencephalogram (EEG) signals to establish a channel for direct interaction between the human brain and the external world without the need for muscle or nerve control. The P300 speller, one of the most widely used BCI applications, presents a selection of characters to the user and performs character recognition by identifying P300 event-related potentials from the EEG. Such P300-based BCI systems can reach good levels of accuracy but are difficult to use in day-to-day life due to redundancy and noisy signal. A room for improvement should be considered. We propose a novel hybrid feature selection method for the P300-based BCI system to address the problem of feature redundancy, which combines the Menger curvature and linear discriminant analysis. Approach. First, selected strategies are applied separately to a given dataset to estimate the gain for application to each feature. Then, each generated value set is ranked in descending order and judged by a predefined criterion to be suitable in classification models. The intersection of the two approaches is then evaluated to identify an optimal feature subset. Main results. The proposed method is evaluated using three public datasets, i.e. BCI Competition III dataset II, brain/neural computer interaction Horizon dataset, and Lausanne Federal Institute of Technology dataset. Experimental results indicate that compared with other typical feature selection and classification methods, our proposed method has better or comparable performance. Significance. Additionally, our proposed method can achieve the best classification accuracy after all epochs in three datasets. In summary, our proposed method provides a new way to enhance the performance of the P300-based BCI speller.

Objective. Electroencephalography (EEG) microstates (MSs), which reflect a large topographical representation of coherent electrophysiological brain activity, are widely adopted to study cognitive processes mechanisms and aberrant alterations in brain disorders. MS topographies are quasi-stable lasting between 60–120 ms. Some evidence suggests that MS are the electrophysiological signature of resting-state networks (RSNs). However, the spatial and functional interpretation of MS and their association with functional magnetic resonance imaging (fMRI) remains unclear. Approach. In a cohort of healthy subjects (n = 52), we conducted several statistical and machine learning (ML) approaches analyses on the association among MS spatio-temporal dynamics and the blood-oxygenation-level dependent (BOLD) simultaneous EEG-fMRI data using statistical and ML approaches. Main results. Our results using a generalized linear model showed that MS transitions were largely and negatively associated with BOLD signals in the somatomotor, visual, dorsal attention, and ventral attention fMRI networks with limited association within the default mode network. Additionally, a novel recurrent neural network (RNN) confirmed the association between MS transitioning and fMRI signal while revealing that MS dynamics can model BOLD signals and vice versa. Significance. Results suggest that MS transitions may represent the deactivation of fMRI RSNs and provide evidence that both modalities measure common aspects of undergoing brain neuronal activities. These results may help to better understand the electrophysiological interpretation of MS.

Objective. Reorienting is central to how humans direct attention to different stimuli in their environment. Previous studies typically employ well-controlled paradigms with limited eye and head movements to study the neural and physiological processes underlying attention reorienting. Here, we aim to better understand the relationship between gaze and attention reorienting using a naturalistic virtual reality (VR)-based target detection paradigm. Approach. Subjects were navigated through a city and instructed to count the number of targets that appeared on the street. Subjects performed the task in a fixed condition with no head movement and in a free condition where head movements were allowed. Electroencephalography (EEG), gaze and pupil data were collected. To investigate how neural and physiological reorienting signals are distributed across different gaze events, we used hierarchical discriminant component analysis (HDCA) to identify EEG and pupil-based discriminating components. Mixed-effects general linear models (GLM) were used to determine the correlation between these discriminating components and the different gaze events time. HDCA was also used to combine EEG, pupil and dwell time signals to classify reorienting events. Main results. In both EEG and pupil, dwell time contributes most significantly to the reorienting signals. However, when dwell times were orthogonalized against other gaze events, the distributions of the reorienting signals were different across the two modalities, with EEG reorienting signals leading that of the pupil reorienting signals. We also found that the hybrid classifier that integrates EEG, pupil and dwell time features detects the reorienting signals in both the fixed (AUC = 0.79) and the free (AUC = 0.77) condition. Significance. We show that the neural and ocular reorienting signals are distributed differently across gaze events when a subject is immersed in VR, but nevertheless can be captured and integrated to classify target vs. distractor objects to which the human subject orients.

Objective. Brain-computer interface (BCI) aims to establish communication paths between the brain processes and external devices. Different methods have been used to extract human intentions from electroencephalography (EEG) recordings. Those based on motor imagery (MI) seem to have a great potential for future applications. These approaches rely on the extraction of EEG distinctive patterns during imagined movements. Techniques able to extract patterns from raw signals represent an important target for BCI as they do not need labor-intensive data pre-processing. Approach. We propose a new approach based on a 10-layer one-dimensional convolution neural network (1D-CNN) to classify five brain states (four MI classes plus a 'baseline' class) using a data augmentation algorithm and a limited number of EEG channels. In addition, we present a transfer learning method used to extract critical features from the EEG group dataset and then to customize the model to the single individual by training its late layers with only 12-min individual-related data. Main results. The model tested with the 'EEG Motor Movement/Imagery Dataset' outperforms the current state-of-the-art models by achieving a ${99.38\%}$ accuracy at the group level. In addition, the transfer learning approach we present achieves an average accuracy of ${99.46\%}$. Significance. The proposed methods could foster the development of future BCI applications relying on few-channel portable recording devices and individual-based training.

Objective. Neuro-steered assistive technologies have been suggested to offer a major advancement in future devices like neuro-steered hearing aids. Auditory attention decoding (AAD) methods would in that case allow for identification of an attended speaker within complex auditory environments, exclusively from neural data. Decoding the attended speaker using neural information has so far only been done in controlled laboratory settings. Yet, it is known that ever-present factors like distraction and movement are reflected in the neural signal parameters related to attention. Approach. Thus, in the current study we applied a two-competing speaker paradigm to investigate performance of a commonly applied electroencephalography-based AAD model outside of the laboratory during leisure walking and distraction. Unique environmental sounds were added to the auditory scene and served as distractor events. Main results. The current study shows, for the first time, that the attended speaker can be accurately decoded during natural movement. At a temporal resolution of as short as 5 s and without artifact attenuation, decoding was found to be significantly above chance level. Further, as hypothesized, we found a decrease in attention to the to-be-attended and the to-be-ignored speech stream after the occurrence of a salient event. Additionally, we demonstrate that it is possible to predict neural correlates of distraction with a computational model of auditory saliency based on acoustic features. Significance. Taken together, our study shows that auditory attention tracking outside of the laboratory in ecologically valid conditions is feasible and a step towards the development of future neural-steered hearing aids.