Restoring continuous finger function with temporarily paralyzed nonhuman primates using brain–machine interfaces

Objective. Brain–machine interfaces (BMIs) have shown promise in extracting upper extremity movement intention from the thoughts of nonhuman primates and people with tetraplegia. Attempts to restore a user’s own hand and arm function have employed functional electrical stimulation (FES), but most work has restored discrete grasps. Little is known about how well FES can control continuous finger movements. Here, we use a low-power brain-controlled functional electrical stimulation (BCFES) system to restore continuous volitional control of finger positions to a monkey with a temporarily paralyzed hand. Approach. We delivered a nerve block to the median, radial, and ulnar nerves just proximal to the elbow to simulate finger paralysis, then used a closed-loop BMI to predict finger movements the monkey was attempting to make in two tasks. The BCFES task was one-dimensional in which all fingers moved together, and we used the BMI’s predictions to control FES of the monkey’s finger muscles. The virtual two-finger task was two-dimensional in which the index finger moved simultaneously and independently from the middle, ring, and small fingers, and we used the BMI’s predictions to control movements of virtual fingers, with no FES. Main results. In the BCFES task, the monkey improved his success rate to 83% (1.5 s median acquisition time) when using the BCFES system during temporary paralysis from 8.8% (9.5 s median acquisition time, equal to the trial timeout) when attempting to use his temporarily paralyzed hand. In one monkey performing the virtual two-finger task with no FES, we found BMI performance (task success rate and completion time) could be completely recovered following temporary paralysis by executing recalibrated feedback-intention training one time. Significance. These results suggest that BCFES can restore continuous finger function during temporary paralysis using existing low-power technologies and brain-control may not be the limiting factor in a BCFES neuroprosthesis.


Introduction
Neural prostheses have the potential to restore function and independence to people with neurological disorders and injuries. By interfacing with the nervous system directly, they can extract a user's intention information from the native controlling circuits and use those signals to control prostheses, computers, or reanimate paralyzed limbs. Particularly in tetraplegia, it has been found that restoration of hand and arm function is of greatest importance [1]. Furthermore, for the purposes of upper extremity restoration, people with paralysis would prefer use of their natural arm and hand over external prostheses [2].
Functional electrical stimulation (FES) has shown promise in restoring function to paralyzed arms and hands for reaching and grasping [3][4][5][6]. By delivering electrical current to partially or completely non-functional musculature, commercial devices like the Freehand System [7,8] have already been used with people with upper extremity paralysis to restore movement of native arms and hands. More recently, systems like the Networked Neuroprosthesis (NNP) have shown promise in primarily research-focused studies, having been implanted in five people with spinal cord injury to date [9,10]. In cases of upper extremity paralysis, FES systems are often controlled by residual functional musculature to cycle between and activate grips. One study investigated how well three people with tetraplegia could use their shoulder position, wrist position, or wrist myoelectricity to control FES for opening and closing a grasp [11], establishing the control systems used in many followup studies [6,9,12,13]. While successfully restoring functional grasps to paralyzed hands in these studies, the controllers are not intuitive beyond a single degree of freedom and would require extensive training for efficient usage. Furthermore, they are not generalizable solutions and people with severe tetraplegia, such as high cervical spinal cord injury, may not have sufficient residual function for such electromyography or positional detectors.
For upper extremity restoration, brain-controlled functional electrical stimulation (BCFES) typically comes in discrete or continuous control forms. For example, Bouton et al classified the user's intentions with a support vector machine and delivered surface stimulation patterns to generate intended hand postures [34]. For continuous control, Moritz et al and Ethier et al predicted and restored muscular force outputs as a continuous function of the spiking activities of few (Moritz et al) or hundreds (Ethier et al) of units [30,31]. Badi et al used spiking activities of tens of units to continuously control stimulation to just the temporarily paralyzed radial nerve in an object grasping task [29]. Ajiboye et al used a similar control system [33]. They recorded units from hundreds of electrodes to continuously predict joint angles via optimal linear estimation. Then, those predicted joint angles were translated to stimulation patterns that generated the user's intended movements. These studies showed clear improvements in abilities during paralysis, however, restoration of hand function only came in the form of discrete grasps. Even for regression-based algorithms, the tasks involved swapping between a hand opened or closed state. While undeniably important for completing common activities of daily living, there is insufficient understanding of how precisely FES can continuously control movements of the hand.
In this work, we leveraged our high-performance finger BMI [35][36][37][38] to restore continuous movement to temporarily paralyzed prehensile fingers in an NHP using FES, the first of such a demonstration to our knowledge. When using the system, the monkey acquired virtual finger targets at a rate substantially higher than he could with his temporarily paralyzed hand. With two monkeys in an anesthetized state, we used target information to control FES directly and found that hysteresis, failure of muscular recruitment, and feedback latency substantially reduce target acquisition speed and create challenges for a BCFES neuroprosthesis user. Finally, we use a virtual task without FES to demonstrate, for the first time, that the simultaneous and independent movements of multiple finger groups can be controlled by a BMI even when the monkey's hand is temporarily paralyzed. Despite the performance reduction that was likely a result of the absence of sensory feedback, recalibrating the BMI using recalibrated feedback intention-training (ReFIT) restored performance to levels achieved by the BMI prior to temporary paralysis.

Methods
All procedures were approved by the University of Michigan Institutional Animal Care and Use Committee.

Implants
We implanted two adult male rhesus macaques with Utah microelectrode arrays (Blackrock Microsystems, Salt Lake City, UT, USA) in the hand area of preand postcentral gyri, as described previously [36][37][38]. Monkey N was first implanted with two 64-channel arrays in left hemisphere precentral gyrus and one 96channel array in left hemisphere postcentral gyrus. Data from the postcentral gyrus array was only used to investigate how well sensory information could predict one degree of freedom finger movements measured by a manipulandum. Monkey N was age 5 years and was 37 days post implant for this data collection. Later, connections to these arrays were cut and Monkey N was implanted with two new 64channel arrays in right hemisphere precentral gyrus and one 96-channel array in right hemisphere postcentral gyrus. Monkey N was age 9 and between 867 and 1071 days post cortical implant for all other data collected in this study, which resulted from these arrays. Monkey W was implanted with one 96channel array in each of left hemisphere precentral and postcentral gyri. Monkey W was age 6 years and was 590 days post implant at the time of data collection. All of Monkey N's right hemisphere arrays, only Monkey N's left hemisphere postcentral gyrus array, and only Monkey W's postcentral gyrus array were used in this study. Note that Monkey N's left and right hemisphere postcentral gyrus arrays and Monkey W's postcentral gyrus array were only used for the analysis predicting finger movements from postcentral gyrus activity, not with FES. Pictures of all implants can be found in supplementary figure 2.
In a separate surgery, we implanted Monkey N with 86 cm chronic bipolar intramuscular electromyography recording electrodes (similar to PermaLoc ™ electrodes, Synapse Biomedical, Inc., Oberlin, OH, USA). These electrodes were limited to 5 µC of charge per pulse. Electrodes were implanted as described previously [36]. Briefly, a single radialvolar incision was used to access flexor muscles and six electrodes were implanted. For this study, we only used the two electrodes implanted in flexor digitorum profundus-index (FDPi) and flexor digitorum profundus for the middle, ring, and small (MRS) fingers (FDPmrs). Electrodes were secured intramuscularly using non-absorbable monofilament suture. After closing the radial-volar incision, a single dorsal-ulnar incision was used to access the extensors and five electrodes were implanted. For this study, we only used the electrode implanted in extensor digitorum communis (EDC). Electrodes were tunneled proximally to an interscapular exit site and connected to the standard PermaLoc ™ connector. All incisions were closed in a layered fashion using absorbable sutures and leads were stitched to the exit site. Following implantation, the monkey persistently wore a Primate jacket (Lomir Biomedical, Inc., Malone, NY, USA). Monkey N was between 195 and 399 days post FES electrode implant for all data collected in this study.
To investigate continuously-controlled onedimensional (1D) FES in Monkey W, we acutely implanted bipolar fine wire electrodes (019-475400, Natus Medical Inc., Middleton, WI, USA) in his left forearm. After induction of anesthesia with propofol (see methods below), we targeted flexor digitorum profundus-MRS and extensor digitorum communis anatomically and observationally by manually flexing and extending the fingers.

Feature extraction
All processing was done in MATLAB versions 2012b or 2019b (Mathworks, Natick, MA, USA), except where noted.
Precentral gyrus (motor-related) spiking band power (SBP) from Monkey N was acquired in realtime during the experiments (see the subsequent section for a description of data flow). To do so, we configured the Cerebus Neural Signal Processor (Blackrock Microsystems) to band-pass filter the raw signals to 300-1000 Hz using the Digital Filter Editor feature included in the Central Software Suite version 6.5.4 (Blackrock Microsystems), then sampled at 2kSps for SBP. The continuous data was streamed to a computer running xPC Target version 2012b (Mathworks), which took the magnitude of the incoming data, summed all magnitudes acquired in each 1 ms iteration within each channel (not across channels), and stored the 1 ms sums for each channel as well as the quantity of samples received each 1 ms synchronized with all other real-time experimental information. This allowed offline and online binning of the neural activity to create larger bin sizes, such as the 32 ms used in this work, with 1 ms precision. Closedloop decoders only used channels that were not saturated with noise and had contained morphological spikes during the experiment or at some time in the past, as SBP could possibly extract firing rates of low signal-to-noise ratio units remaining represented on such channels [35]. This resulted in 63 or 64 channels used by the closed-loop decoders.
Postcentral gyrus (sensory-related) SBP was recorded to disk for later offline synchronization. The pedestals for the arrays implanted in postcentral gyrus were connected to a CerePlex Direct (Blackrock Microsystems) via a CerePlex E (Blackrock Microsystems), which either collected the raw data at 30kSps or band-pass filtered the incoming signals to 300-1000 Hz using the Digital Filter Editor feature, then sampled at 2kSps for SBP. To synchronize the postcentral gyrus SBP activity after the experiment, we used the Sync Pulse functionality included in Central. The unique Sync Pulses were recorded by both the Cerebus and the CerePlex direct, enabling synchronization of the recordings from both systems offline. Then, postcentral gyrus SBP was filtered to 300-1000 Hz if not previously done, absolute valued, and accumulated in equivalent windows as the precentral gyrus SBP used for comparison using MATLAB R2019b.

Experimental setup
The experimental apparatus used for these experiments is similar to what was described previously [36][37][38]. Briefly, the monkeys' Utah arrays were connected to the patient cable (Blackrock Microsystems) and neural data (as described previously) were streamed to the xPC Target computer in real-time via a User Datagram Protocol packet structure. The xPC Target computer coordinated several components of the experiments, including coordinating target presentation, acquiring measured finger group positions from one flex sensor per group (FS-L-0073-103-ST, Spectra Symbol, Salt Lake City, UT, USA), and transmitting finger positions along with target locations to an additional computer simulating movements of a virtual monkey hand (MusculoSkeletal Modeling Software) [39]. Task parameters, states, and neural features were stored in real-time for later offline analysis.
One additional functionality was implemented in the xPC Target computer to facilitate realtime FES control. We used an RS-232 interface to the NNP Access Point that comprised an MSP-EXP430F5529LP evaluation board (Texas Instruments Inc., Dallas, TX, USA), an Evaluation Module Adapter Board (Texas Instruments), a CC1101EMK433 evaluation kit (Texas Instruments), and a MAX3222E RS-232 level shifter (Maxim Integrated, San Jose, CA, USA). Stimulation commands for one pattern, which are rules governing simultaneous stimulation to multiple electrodes, were transmitted at 115 200 baud, which were sent wirelessly to the NNP power module to be configured into pulses delivered (see sections 2.7 and 2.8).

Behavioral task
We trained Monkeys N and W to acquire virtual targets with virtual fingers by moving their physical fingers in a one-or two-finger task, similar to what we have previously published [36,38]. During all sessions, the monkeys sat in a shielded chamber with their arms fixed at their sides flexed at 90 • at the elbow, resting on a table. The monkeys had their left or right hand (contralateral to cortical implants and ipsilateral to intramuscular implants) placed in the manipulandum described previously [36]. During manipulandum control (not FES), Monkey W had the flexion measuring sensors (FS-L-0073-103-ST, Spectra Symbol Corp., Salt Lake City, UT, USA) taped directly to his index finger and was trained to perform the task with his index finger. Movements of the other fingers were not directly measured and were not used in this study, though Monkey W often moved all four of his fingers together for this task even though he was free to move his other fingers independently. During historical data collection from Monkey N's left hemisphere (for the sensory analysis), the manipulandum degree of freedom for his MRS fingers was locked such that they were at full extension (which prevented movement), while the index finger was free to move across its full range of motion. During all other Monkey N's one degree of freedom finger tasks in this study (BCFES and target-controlled FES), the manipulandum's degrees of freedom were locked together so that Monkey N could only move his fingers together. During Monkey N's two degree of freedom finger task (brain-control of the virtual hand, with no FES), he was free to move both the index and MRS finger groups independently in their degrees of freedom within the manipulandum. The monkeys sat in front of a computer monitor displaying the virtual hand model and targets described previously.
Each trial began with one spherical target per finger degree of freedom appearing along the 1D movement arc. Each target occupied 15% of the full arc of motion of the virtual fingers with two exceptions. BCFES trials with Monkey N during his first usage of the system had a 16.5% target size (94 of 424 presented trials) and all historical trials used to estimate how well postcentral gyrus could predict one degree of freedom finger movements with Monkey W used a 14.25% target size. Targets were presented in a center-out pattern, every other target was presented at rest (halfway between full flexion and full extension or 50% as illustrated in figures), and the non-rest targets were randomly chosen between 20%, 30%, or 40% flexion or extension from rest. For a successful trial, the monkeys were required to move the virtual fingers into their respective targets and remain there for 750 ms continuously in ablebodied manipulandum control or 500 ms continuously in post-block manipulandum control, brain control, and BCFES modes. During historical trials with Monkey W estimating how well postcentral gyrus predicted one degree of freedom finger movements during able-bodied manipulandum control, hold time was 500 ms. If the monkeys could not acquire and hold the target within 10 s, the trial was deemed unsuccessful and the target was placed at center repeatedly until successfully acquired. Upon successful target acquisition, the monkeys received a juice reward, which was modulated to maintain motivation levels. Monkeys were water restricted to a level of 30 ml kg −1 or higher for all data used in this study. Monkey N's weight varied from 10.4 to 13.9 kg and Monkey W's weight varied from 9.4 to 12.7 kg throughout the time period of data collection.

Nerve block procedure
Temporary paralysis of the hand was achieved by delivering a solution of lidocaine (2%) and epinephrine (1:100 000) to the perineural space around three peripheral nerves (radial, median, ulnar) in the upper arm just proximal to the elbow. The solution was delivered so that it completely surrounded each nerve. Supplementary figure 3 presents some example snapshots of the median, radial, and ulnar nerves from Monkey N under ultrasound. The lidocaine/epinephrine solution was either purchased premixed (NDC 0409-3182-11) or compounded using stock solutions of lidocaine (2%) and epinephrine 1 mg ml −1 . Delivery was done under ultrasound (Lumify L12-4 broadband linear array transducer, Philips Healthcare, Best, NL, paired with Samsung Galaxy Tab S6, Samsung Electronics, Suwon, KOR) using a 20 gauge echogenic needle (B Braun #33642, B Braun, Melsungen, DE). The NHP was placed in a restraint chair and the arm was manually restrained. In some experiments, prior to delivering the block, pure lidocaine (2%) was injected subcutaneously to the target injection sites to ease with comfort during the injection, and approximately 6-7 mg kg −1 (4-6 ml) of lidocaine in the lidocaine/epinephrine solution (less than that was ineffective) was used cumulatively for each blocking procedure. Block onset typically occurred after 20-30 min and lasted up to 2 h. There were at least two days between blocking procedures on the same animal.
To guarantee the nerve block was still active at any time during an experiment, we would occasionally inject 'catch' trials and disable the BCFES system and ReFIT Kalman filter (RKF) control, allowing the monkey to complete trials with his native anatomy as capable. Then, following the experiment, we plotted the measured finger positions (which were provided as visual feedback during these 'catch' trials), the finger positions predicted by the RKF, and the targets to determine whether the monkey attempted to acquire the target despite temporary paralysis. If the nerve block remained effective, we would expect the RKF predictions to move in the general direction of the target with minimal movement of the measured finger positions. These plots for the final 'catch' trials during each of the four BCFES experiments are plotted in supplementary figure 7 and show that, despite temporary paralysis, the monkey continued to attempt to acquire the targets throughout the experiments.

Propofol anesthesia
To investigate the capabilities of FES alone without voluntary activation from the monkey, we lightly anesthetized Monkeys N and W with propofol.
After placing the monkey in the primate chair, comfortably restraining his head, and comfortably restraining his arms, we placed an IV catheter in the cephalic vein just distal to the monkey's right elbow, contralateral to the arm being used for FES experiments. We then flushed the catheter with sterile saline to ensure patency. We delivered a 2.5-3.0 mg kg −1 bolus with additive 0.2 mg kg −1 boluses of propofol until the primate was visibly unconscious to induce a light plane of anesthesia. Following induction, light anesthesia was maintained with a constant rate infusion of propofol at 7.5 mg kg h −1 and supplemental boluses of 0.2 mg kg −1 were given as needed to maintain the desired plane of anesthesia.
Oxygen supplementation was provided with a nasal cannula. Heart rate, respiratory rate, body temperature, and blood oxygen levels were monitored throughout the experiment with a pulse oximeter and pediatric ear thermometer. Blood oxygen levels never dropped below 90% and were always greater than or equal to 98% after providing supplemental oxygen.
Anesthetic depth was measured with jaw tension, respiratory rate, and heart rate. After a stable and light plane of anesthesia was reached, we proceeded with FES. Anesthetic events/procedures occurred no more than once per week and for no longer than 2 h with most lasting 1 h. There were two propofol anesthesia sessions for each animal.

FES
We delivered stimulation using the NNP Evaluation System [9]. Figure 1 illustrates how the system was connected to Monkey N. The evaluation system contains the same circuitry used as implants in people (NCT02329652) but in a form factor more conducive of experimentation and debugging. Our evaluation system consisted of one power module, including three 1000 mAh batteries to power the system and deliver stimulation (PRT-13813, Spark-Fun Electronics, Niwot, CO, USA), and three 4channel pulse generator modules. The system could deliver 12 total monopolar stimulation channels. All stimulation was current-controlled and delivered at 10 mA with a 32 ms inter-pulse interval. Pulses were charge-balanced biphasic with a square cathodic pulse delivered first and the subsequent anodic pulse exponentially balancing charge. Stimulation intensity was modulated by varying the width of the cathodic pulse (see supplementary figure 4). Pulse delivery was staggered within each pulse generator module to avoid too large of a current draw than could be provided by the power module. The pulse for channel 1 of a given pulse generator module was delivered 20 ms following the reception of the stimulation command (the time required to prepare the pulse based on the command), and each subsequent channel delivered its pulse 1 ms after the prior channel. All pulse generator modules received the stimulation command simultaneously.
To adapt our bipolar leads to function with monopolar stimulation, we only used one electrode of each pair to deliver current. With Monkey N's chronic electrodes, the electrode that was implanted further inside the muscle was used for monopolar stimulation. The second electrode of each bipolar pair that was further outside the muscle were tied with all other second electrodes connected to one pulse generator module. Each group of four electrodes that were further outside the muscle were connected to that pulse generator module's current return. With Monkey W's acute Generator Module #2 ch1 and ch2, respectively. Bolded muscle names represent muscles that were used in this study, though all muscles listed were connected to the system during testing. Each muscle has a red and black wire to represent the connections of each contact for each bipolar lead. Debug indicators were used to display system status during an experiment using light-emitting diodes and were not connected to the animal in any way. MICS refers to the medical implant communication service radio band that was used for wireless communication between the experimental system and the NNP, providing wireless isolation between the recording and stimulation systems (since the NNP was powered by batteries). electrodes, we connected the NNP to each electrode pair using hook grabbers. One electrode of each pair was connected to one monopolar pulse generator, and one of the two remaining disconnected electrodes was connected to the current return of the NNP. Due to the variability in placement of the acute electrodes with Monkey W, occasionally a muscle would contract due to the returning current though the muscle. If this occurred, we would instead tie the current return to a surface electrode (NC0748095, Biopac Systems, Inc., Goleta, CA, USA) with conductive gel (SignaGel, Parker Labs, Inc., Fairfield, NJ, USA) as needed, placed either on the chest or back of the neck.
Prior to usage of the BCFES or target-controlled FES systems, the flex sensors were recalibrated to the range of motion that could be achieved independently by FES. During some BCFES and targetcontrolled FES experiments, small wooden splints were taped across the monkey's proximal and distal interphalangeal joints following the nerve block or anesthesia. FES would occasionally flex the distal and proximal interphalangeal joints within the manipulandum in a way that could not be measured by the manipulandum and prevent extension of the manipulandum. Such an issue could be resolved by stimulating the dorsal interossei muscles, but we elected to not implant the hand's intrinsic muscles to avoid substantially increased risks of infection or surgical sites reopening post-operation. As such, FES could not independently re-extend the fingers, and the splints helped with keeping the proximal and distal interphalangeal joints straight so that the fingers flexed primarily at the metacarpophalangeal joint. Additionally, during BCFES experiments, we would occasionally stop sets of BCFES trials early in the event of an incompatible finger state (as previously described) or if the decoder consistently predicted a maximum flexion or extension state for at least one trial continuously. This was done in an attempt to avoid the muscles fatiguing too soon into the experiment. In such instances, the decoder was reinitialized and BCFES was re-enabled.

Patterns
To keep communication bandwidth low, stimulation was delivered by coordinated patterns [33,40]. In a patterned stimulation paradigm, electrodes belonging to a pattern have their pulse parameters governed by a 1D variable, given the range 0%-100% in this manuscript. In our case, 0% corresponded to maximum extension and 100% corresponded to maximum flexion that could be activated by FES. In this way, one command can govern stimulation parameters on any number of electrodes.
In our 1D implementation, we used one pattern to represent stimulation parameters for the electrodes in the FDPi, FDPmrs, and EDC muscles. As implemented on the NNP, patterned stimulation allows one to control pulse width and current amplitude at each command value. Command values were computed by the xPC Target computer from the finger positions predicted by the RKF or the target controller during BCFES and target-controlled FES experiments, respectively. The patterns we implemented did not modulate stimulation amplitude with command value (constant 10 mA) but did modulate pulse width between 0 and 255 µs. Our pattern delivered maximum stimulation to EDC at 0% command and maximum stimulation to FDPi and FDPmrs at 100% command. In the region around 50% command value, all of EDC, FDPi, and FDPmrs received stimulation at approximately 30% of the maximum pulse width according to the pattern, as has been done by others [4,33]. Generating co-contracting stimulation when not at the limits of range of motion can more consistently create the desired position at that particular command value and combat hysteresis. We have discussed previously how fatigue invoked by FES is generally only an issue in high force activities [41], so we expect our usage of co-contracting stimulation did not promote earlier muscle fatigue. Supplementary figure 4 illustrates an example pattern used during a BCFES experiment.

Closed-loop RKF training and usage
To investigate whether Monkey N could use a BMI to control the 1D BCFES system or individuate virtual finger movements during states of temporary paralysis, we implemented position/velocity RKFs in a similar fashion to what we have previously published [36]. We trained the standard Kalman filter (required to train an RKF) using at least 325 trials in manipulandum control mode with a 750 ms target hold time.
To train the 1D standard Kalman filter, we used the finger movements measured by the manipulandum when the manipulandum degrees of freedom were locked together such that all fingers moved together. To train the two-dimensional (2D) standard Kalman filter, we used the finger movements measured by the manipulandum when the manipulandum degrees of freedom were free to move independently (splitting movements of the index finger from movements of the MRS fingers).
The standard Kalman filter and RKF assumed a kinematic state containing one position and onevelocity for each degree of freedom. So for the 1D BCFES task, the standard Kalman filter and RKF predicted one position and one velocity to represent movements of all fingers together: where x t is the kinematic state at time t, P is position, and V is velocity. For the 2D virtual task, the standard Kalman filter and RKF predicted one position and one velocity for the index finger and one position and one velocity for the MRS fingers grouped together: where P I and V I are position and velocity of the index finger, respectively, and P MRS and V MRS are position and velocity of the MRS finger group, respectively. We configured the Kalman filters to predict the state at each 32 ms timestep based on the optimal combination of two other predictions: one from the previous timestep's state, and the other from comparing the measured neural activity and that predicted by the expected kinematics for the current timestep: The C matrix was computed by least squares regression to convert kinematics to neural features. The A matrix was also trained via least squares regression to convert the state at time t − 1 into the state at time t, but constrained to match the expected physical trajectories of the finger movements (i.e. position should be held constant until updated with a velocity). For the 1D BCFES task (all fingers moving together): And for the 2D virtual task: To train the RKF, the monkeys first used the standard Kalman filter to control the virtual hand directly in closed-loop for at least 250 trials (no FES). The RKF is trained on the velocities predicted by the standard Kalman filter during closed-loop control, but with the velocities modified afterwards offline based on the assumption that the user's intention was to optimally acquire the targets given the current state. For 1D RKF training (eventually used with FES), predicted velocities were negated in timesteps where movement was away from the target and the magnitude of the velocities were zeroed if inside the target. For 2D RKF training (for controlling the virtual hand directly, with no FES), the 2D velocity vectors predicted by the standard Kalman filter were rotated to point towards the 2D target vectors, keeping the same magnitude as was originally predicted by the standard Kalman filter. Timesteps in which a finger was in its 1D target had that finger's velocity set to zero magnitude. We used these modified velocities to recompute the RKF parameters, and RKF and BCFES trials throughout the results used these retrained parameters without any further modification to velocities. These procedures are similar to those originally introduced for the arm-reaching RKF, but here applied to 1D and 2D finger movements [42].
The RKF was always used for at least 100 trials when compared to other control methods (i.e. RKF vs. BCFES in the 1D task with FES, RKF vs. the twotimes recalibrated RKF (Re-RKF) in the 2D virtual task with no FES, etc). During all brain-control trials, the positions of the fingers as displayed on the screen were integrated from the RKF's predicted velocities. All of this was performed prior to the experiment's nerve block.

Determining stimulation command value 2.10.1. 1D BCFES
During BCFES trials, the predictions from the 1D RKF were used to control FES patterns (see section 2.8). To do so, we trained the 1D RKF to predict normalized finger positions in the range of 0% (full extension of all fingers) to 100% (full flexion of all fingers). These predictions were transmitted directly to the NNP as stimulus commands. Manual velocity biases were added as needed to the RKF's predicted velocities following the nerve block to assist with any changes in noise level resulting from disconnecting and reconnecting the recording hardware. Biases were only necessary during two of the four sessions with Monkey N and varied between −1 × 10 −2 and 2 × 10 −2 % per iteration (at 32 ms).
The two-finger RKF was never used to control stimulation, since our FES was limited to one degree of freedom control at the time of investigation. The two-finger RKF's predictions were only ever used to directly control movements of the virtual fingers.

Target-controlled FES
We used target information to control stimulation to better understand the capabilities and challenges when performing continuous FES. The state machine diagram in supplementary figure 1 illustrates this controller. At the beginning of each trial, the stimulation command value is set to the target's center position. For example, if the target was centered at 70%, then the stimulation command at the beginning of the trial was also set to 70%. Then, a stimulation update was delivered prior to every pulse (32 ms between updates) depending on the target's relative location to the current finger position. If the target required more finger flexion, stimulation command value was increased by approximately 1% within the pattern (which converts commands in the range of 0%-100% to stimulus pulse widths for the FES electrodes, see supplementary figure 4 and section 2.8).
If the target required more finger extension, stimulation command value was decreased by approximately 1% within the pattern. Updates were sent continuously every 32 ms in this fashion until the finger moved within 5.625% of the target's center (or the middle 75% of the target). Once in the target, stimulation command value was held constant until the finger inadvertently moved further than 5.625% from the target's center, the target was successfully acquired, or the trial timeout was reached.

Performance metrics 2.11.1. Closed-loop performance metrics
We estimated closed-loop performance with success rate and acquisition time, occasionally split into measures of time to target and orbiting time. Success rate was calculated as the total number of targets acquired successfully divided by the total number of targets presented. Acquisition time was computed as the total amount of time from the beginning of the trial to the time the target was successfully acquired and held, less the hold time. Failed trials were given acquisition times equal to the trial timeout less the hold time. Time to target was computed as the time from the beginning of the trial to the first instance the target was touched by the finger. Trials in which the finger never touched its target were given acquisition times and times to target equal to the trial timeout less the hold time. Orbiting time was computed as the time from first touching the target to the time the target was successfully acquired and held, less the hold time. The sum of a trial's time to target and orbiting time equals its acquisition time.
In all metrics, the following conditions were excluded from analysis to avoid biasing the results. Successful trials immediately following a failure were excluded to avoid situations in which the monkey would wait at the central position to acquire the central target once it returned following a failed outer target. Trials immediately following a change in control method were also excluded, as the state of the controller could not be determined prior to usage. For example, when switching from RKF control to manipulandum control, the displayed finger position could jump from outside of the target straight to the subsequent target, resulting in an unrealistically small acquisition time. In the Results text, we present two sets of statistics for acquisition times: one set representing the above exclusions, and one set representing the above exclusions and additionally excluding failed trials.
For the orbiting time metric, trials in which the finger never touched its target were excluded, as orbiting time could not be computed.

Open-loop performance metrics
We estimated open-loop prediction performance using the intra-class correlation (ICC) coefficient.
As opposed to Pearson's correlation, ICC is a better measure of agreement between two conditions. ICC is computed via the following equation (1): where σ 2 b is the variance between conditions and σ 2 w is the variance within conditions. Confidence intervals were computed via the following formulas and equation (2) [43]: where Z 1− α 2 is the z-score at the confidence level 1 − α 2 . Significant differences between ICCs were determined via the Fisher's transformation of θ for each ICC via equation (3): The p-value for z was calculated from a normal distribution.

Restoration of continuous hand function with BCFES in temporary paralysis
We first sought to estimate an NHP's capability of continuously controlling his own temporarily paralyzed hand using a BCFES neuroprosthesis. To do so, we trained an NHP to perform a finger task in which he controlled a virtual hand model using proportional movements of his physical fingers. A virtual target of a size equivalent to 15% or 16.5% of the range of motion (depending on the stage of BCFES training) was presented for movements of all four prehensile fingers together, and the monkey was first required to move his physical fingers such that the virtual fingers entered the target and held it continuously for 750 ms. Then, we trained and tested an RKF to predict the primate's intended finger movements from SBP recorded from Utah microelectrode arrays implanted in the motor-related hand area of precentral gyrus in real-time (see section 2) [37]. We used SBP, computed by taking the absolute value of the 300-1000 Hz band and averaging in 32 ms bins, as we have previously shown it can extract firing patterns at lower signal-to-noise ratios, is more specific to the spiking rates of single units, and achieves as good or better prediction performance than threshold crossing rates [35].
To demonstrate that we could control finger movements continuously with FES, we also implanted chronic bipolar stimulating electrodes into finger-related muscles of the forearm (see section 2 for a list of electrode locations and quantities). Then, we delivered a lidocaine with epinephrine solution to surround the median, radial, and ulnar nerves just proximal to the elbow to temporarily block voluntary muscle contractions of the hand. We delivered electrical stimulation using the NNP Evaluation System [9], which is a benchtop version of a human implantable device. We used the NNP to control the aperture of the hand's fingers with stimulation patterns, or a number between 0% and 100% controlling the stimulation parameters delivered to all electrodes [33,40]. Then, we tested the monkey's ability to control the virtual hand using his physical hand movements after the nerve block (purple route in figure 2(a)), using the RKF's predictions to control the virtual hand directly (blue route in figure 2(a)), and using the RKF to control the stimulation delivered by the NNP with the virtual hand controlled directly by his physical hand movements (red route in figure 2(a)).
Figure 2(c) shows example control traces and figure 2(d) shows single-trial statistics from all four experiment days under all control methods: manipulandum and RKF control prior to nerve block and manipulandum, RKF, and BCFES control following the nerve block. Using the BCFES system, Monkey N could acquire 83% of the presented targets (504 trials) in a median acquisition time of 1.5 s (1.3 s with 85 failed trials of 504 excluded). BCFES substantially improved control capabilities over Monkey N's native hand movements following the nerve block, which had an 8.8% success rate (91 trials) and a median acquisition time of 9.5 s (1.39 s with 83 failed trials of 91 excluded), which is the trial timeout. Supplementary Video 1 portrays this comparison between BCFES and able-bodied control during the nerve block. Following the nerve block, Monkey N's usage of the RKF to control the animated fingers did not change substantially, maintaining a 100% success rate with a median acquisition time of 0.52 s (869 trials). Figure 2(c) shows a representative BCFES control example with decoded commands and delivered stimulus pulse widths included. Most failures of the BCFES were towards further extension targets, as Monkey N's extensors fatigued quickly during BCFES usage. This is clearly showcased in figure 2(e), which displays individual successful trial traces to each of the closest and furthest outer targets with early, middle, and late trials averaged into three highlighted traces of median length. Late trials to extension targets generally had lower success rates than during early trials, with only 1 successful acquisition of 13 far extension targets during the late epochs compared to 10 of 16 successes to the same target during early epochs. Recruitment of Monkey N's EDC muscle was challenging even at maximum stimulation, which resulted in rapid fatiguing and his reliance on gradual recruitment to hit extension targets. Unfortunately, NHPs require higher FES frequencies [29][30][31] than those required for humans [4,6,33], and we have previously reported high stimulation frequency promotes fatiguing [41]. Surprisingly, success rates remained high for flexion targets through all epochs and median acquisition times generally decreased in later trials (significance between late and early epochs for the nearest flexion target, p < 0.01; all others not significant, p > 0.01, two-tailed twosample Wilcoxon rank sum test). This may be a result of Monkey N learning to better use the BCFES system or muscle fatigue improving the controllability of the stimulation.

High-speed time to target with target-controlled FES
We next sought to better understand how well our FES solution worked as a standalone finger prosthesis without BMI control. Such an analysis can be informative about what challenges a user must consciously balance when using a BCFES neuroprosthesis. To investigate this, we anesthetized Monkey N and Monkey W (who was implanted with acute FES electrodes, see section 2) with propofol and used target information to control the stimulation command on two experiment days each. Supplementary figure  1 illustrates the simple proportional controller, where at the beginning of a trial, the stimulation command is set to the target position (in the range of 0%-100%). Then, to provide any necessary minor corrections ideally without overshooting the target, the stimulation command was increased or decreased by approximately 1%, near the minimum control resolution, every iteration depending on the direction of the target from the current position of the finger. We purposefully selected this simplistic controller to inform us of the challenges an individual using FES without any controller assistance must face, despite its inefficiency relative to BCFES. Figures 3(a) and (b) illustrates acquisition time statistics for all trials with target-controlled stimulation for both monkeys, with colors indicating the direction of movement in the center-out task. Overall, times to target were quick for Monkey N (median 0.86 s) and slower but still rapid for Monkey W (median 1.7 s), comparable to typical times to target during BMI use. Times to target for flexions were generally faster with Monkey N (median 0.73 s) than extensions (median 1.4 s). This was likely due to Monkey N's flexor muscles showing better recruitment than the extensors which quickly exhibited muscle fatigue. This fatigue also resulted in substantially higher variability in times to target when reaching towards extension targets than flexion or central targets.
Interestingly, orbiting, or the action of dialing into a target after first hitting it, was a challenge in Monkey N but less-so in Monkey W (median 1.1 s for Monkey N, median 0.073 s for Monkey W). The individual trial traces in figure 4 and the top plots in figures 3(c) and (d) illustrate the orbiting, where the yellow regions show that orbiting occupies the majority of the time axis. Hysteresis in muscular activation from stimulation, particularly for the flexors, caused many of these orbiting periods, evidenced by the frequent passing of the target from both sides. In contrast, gradual recruitment of the extensors helped reduce orbiting (for extension vs. flexion and center targets: median 0.49 s vs. 1.24 s in Monkey N, p = 5.6 × 10 −3 ; 0.040 s vs. 0.080 s in Monkey W, p = 8.8 × 10 −3 , two-tailed two-sample Wilcoxon rank sum test). Notably, the acute electrodes implanted in Monkey W yielded a drastically lower median orbiting time, particularly during small bursts of consecutive trials without flexion targets as seen in figures 3(d) and 4. We hypothesize these acute electrodes enabled smoother recruitment of the muscles, potentially due to the lack of scar tissue surrounding the electrode sites or electrode placements closer to the motor nerves. Overall, orbiting poses a substantial challenge to usage of a continuous FES system with chronically implanted electrodes, causing 16 of 20 failures in Monkey N, and a moderate challenge with acutely implanted electrodes, causing 6 of 28 failures in Monkey W.

High performance two-finger brain-control in temporary paralysis
To close the gap towards a multi-finger BCFES neuroprosthesis, we next wanted to investigate how well our BMI systems could predict multi-finger movements following nerve block, i.e. controlling animated fingers when the hand has been temporarily paralyzed. Similar to our 1D finger task discussed thus far, we trained a 2D RKF to predict the simultaneous and independent movements of two finger groups (the index finger separate from the MRS fingers as a group) prior to blocking the nerves [36]. Then, we used the RKF to predict the 2D finger movements in real-time following the nerve block to directly control the virtual fingers (with no FES) on two separate days. Figure 5 illustrates the results of these experiments, where prior to the nerve block, the monkey's capabilities of completing the task with his physical hand and the RKF were high (100% success rate for both and 0.39 s and 0.75 s median acquisition times, respectively). Following the nerve block, the monkey's acquisition rate when using his physical hand dropped drastically (2.5% success rate due to chance acquisition, with 9.5 s median acquisition time, 0 s with 69 failed trials of 71 excluded), as expected due to the inability to physically move his fingers.
Then, we transitioned back to investigating usage of the BMI following the nerve block. When testing the RKF, we surprisingly found a substantial drop in performance (90% success rate with 1.4 s median acquisition time, 1.2 s with 74 failed trials of 707 excluded) despite no direct cortical interventions. In an attempt to restore performance, we performed the ReFIT training procedure an additional time using Figure 5. Recalibrated RKF restores high-performance control of virtual multi-finger movements following nerve block. Success rates and acquisition times for manipulandum, RKF, and recalibrated RKF (Re-RKF) two-finger control before and after nerve blocks are shown (with no FES). Numbers shown atop the acquisition times plot represent the median acquisition time. Gray regions represent times when nerves were blocked. Boxes within the plots show median in the center with the edges representing 25th to 75th percentiles. RKF before the nerve block and Re-RKF acquisition times were not statistically significantly different (p = 0.67, two-tailed Wilcoxon rank sum test), but all other acquisition time comparisons were statistically significant (p < 0.001, two-tailed Wilcoxon rank sum test).
the closed-loop RKF trials Monkey N performed following the nerve block. We termed this second-stage RKF the Re-RKF, indicated in green in figure 5. After recalibrating, closed-loop control with the Re-RKF returned to the level of performance achieved by the RKF prior to the nerve block (100% success rate for both and 0.75 s vs. 0.81 s median target acquisition time between RKF prior to the nerve block and Re-RKF, respectively; p = 0.67, two-tailed Wilcoxon rank sum test).
The abrupt drop in BMI performance following the nerve block motivated us to investigate what may have been its cause. When we blocked median, radial, and ulnar nerves, not only were the descending motor commands prevented from reaching their muscular targets, but we think proprioceptive and sensory feedback were also partially or completely blocked from transduction back to cortex. The RKF trained on able-bodied behavior might have learned to depend on some of this sensory information, which then becomes absent following a nerve block, possibly accounting for the performance drop. Fortunately, during these experiments, we simultaneously recorded sensory activity from a separate Utah array implanted in sensory-related postcentral gyrus alongside the motor-related precentral gyrus array used for closed-loop control. This gives us an avenue through which we can investigate how the nerve block impacts BMI control with the RKF.
We first asked whether we usually had sensory tuning to the task's kinematics represented on an array in sensory-related postcentral gyrus. To do so, we used historical data from Utah arrays implanted into the postcentral gyri of two animals (Monkey W and a previous implant in Monkey N). We trained an open-loop, offline Kalman filter to use SBP recorded from postcentral gyrus to predict the one degree of freedom finger movements measured by the manipulandum, doing so with ICCs of 0.81 and 0.58 for Monkeys N and W, respectively ( figure 6(a)). This suggests that the sensory data usually has good linear tuning to finger kinematics, which a regression would tend to find. We also saw the same result for the two finger task with Monkey N's active postcentral gyrus array, as shown by the purple bars in figure 6(d) (mean 0.65 ICC across both experiment days).
Next, we wondered if sensory tuning remained representative of the kinematics predicted by the BMI, both before and after the nerve block (illustrated by figure 6(b) with results summarized in figures 6(c) and (d)). Prior to the nerve block, an offline sensory Kalman filter accurately predicted the closedloop RKF predictions, but performance drastically dropped following the nerve block and during Re-RKF use (mean 0.64, 0.33, then 0.21 ICC, respectively, p < 0.001 between all comparisons, one-tailed two-sample z-test on the Fisher-transformed ICC). This suggests that there was a loss of information transfer from sensory-related to motor-related cortical areas during the finger task without sensory feedback. Therefore, for the Re-RKF to restore closedloop performance, performing the additional stage of ReFIT likely fit an observation model that was not dependent on sensory information.

Discussion
Modern FES has primarily focused on restoring discrete grasps and grips but the extent to which FES can continuously control end effectors is not well understood. In this work, we have demonstrated that an NHP can recover a substantial amount of continuous finger function following temporary paralysis of the hand by using a BCFES system. The BCFES system allowed the monkey to dramatically exceed the performance of his native hand in a 1D finger task following targeted regional blockade of the median, radial, and ulnar nerves, though it could not achieve the levels of performance of the able-bodied hand prior to temporary paralysis nor the same level of performance as when controlling an animation. By using information about the presented target to control stimulation without voluntary control from the monkey, we found that hysteresis, muscle recruitment failure, and controller latency may be the primary challenges in using a BCFES prosthesis. Fortunately, these challenges can be partially mitigated by the subject via brain-control. Finally, in the two-finger virtual task with no FES, we have also presented the first demonstration that the simultaneous and independent movements of multiple finger groups can be continuously predicted with high performance when the hand is temporarily paralyzed despite the absence of proprioceptive and sensory feedback. Recalibrating an RKF restored performance of the two-finger decoder to equivalent levels as the original RKF used prior to the regional block. This is a particularly surprising result, as it suggests that brain-controlled FES performance in a paretic scenario may not be limited by the BMI decoder but rather by current FES technologies.
Interestingly, closed-loop RKF control of 2D finger movements saw a drop in performance following nerve block while the same did not occur with the 1D RKF. Our analysis of sensory representation in motor cortex following the nerve block suggests that the loss of sensory feedback may have been the cause for the drop in performance. Perhaps the 1D finger task, which did not require the relatively complex action of individuating the fingers, was simple enough that sensory feedback was not critical to achieve maximal performance. When using sensory-related postcentral gyrus activity to predict the closed-loop 2D RKF and Re-RKF predictions following induction of nerve block, similar to analyses done by others [44], we noticed a substantial drop in ICC compared to before blockade. We hypothesize that some of the precentral gyrus SBP channels that the RKF found valuable contained some information transferred from postcentral gyrus, which is not particularly surprising given that sensory-related postcentral gyrus accurately represents hand joint angles [45] and projects to and receives from motor-related precentral gyrus [46][47][48]. Since precentral gyrus may have had substantially fewer sensory inputs during usage of the RKF following the nerve block, training the Re-RKF decoder may have used motor-related activity without much sensory modulation, which we think is what may have allowed it to function with high performance in the possible absence of sensory feedback. Although this hypothetical explanation is intriguing, further experimental validation will be necessary to support it, particularly without the capability of verifying the complete loss of sensation with an animal model.
An alternative explanation for how Monkey N could control two fingers of the virtual hand with the RKF after nerve block is the Re-RKF may have found visual information in motor cortex was helpful in the relative absence of sensory feedback. It has been previously reported that NHPs will not use their intact limbs when sensation is completely eliminated, but continue to use their limbs almost natively if partially deafferented [49,50]. Since our nerve blocking procedure was acute and likely not complete, we believe our scenario to be similar to the partially deafferented case reported by Twitchell [50], and the RKF and Re-RKF predictions following nerve block were primarily driven by Monkey N's motor intention. Perhaps if the nerve block had lasted substantially longer with Monkey N (i.e. days to weeks), he would have exhibited the same behavior of ignoring his intact paralyzed limb (as the completely deafferented monkeys had done) and shown no conscious effort at completing the task.
When investigating the capability of FES to continuously control finger movements without voluntary intervention from the monkey, we found substantial orbiting to reach flexion and extension targets that was likely an outcome of hysteresis, failure of muscle recruitment, and general fatiguing [51][52][53]. This contrasts with just BMI control, which typically sees reductions in orbiting time as the user becomes acclimated to the control system. Latency in between a command and the behavioral response resulting from stimulation could also contribute to oscillation about a target, where we found in supplementary figure 5 that the latency with BCFES was substantially higher than the latency with the RKF and manipulandum control systems. Therefore, this latency was also likely an obstacle for Monkey N during closed-loop BCFES, though learning to use the system as shown in figure 2 enabled him to reduce these oscillations. Additional feedback controllers, such as a well-tuned proportional-integral-derivative controller [54] or a cosine similarity controller [55], could reduce the cognitive load on the user when combating these systematic challenges of BCFES. The error between the decoder's predictions and an estimate of the present positions of the fingers, which could be captured by a glove worn by the user, could inform the error signal to the controller.
The BCFES results presented here expand upon the BCFES accomplishments of others [29-31, 33, 34] by beginning to investigate the realm of continuous control with FES, using similar technologies as what was previously used in clinical trials. Although our monkey model of temporary paralysis should not be considered equivalent to chronic tetraplegia, one could consider the model to be representative of an ideal case of tetraplegia without typical pathologies, such as spasticity or contractures. Even though the results presented here might represent best-case scenarios given the techniques and methods used, a user may be able to perform more natural hand movements and use the device for a wider variety of tasks just by adding continuous control to grasp and force. Furthermore, combining these techniques with sensory feedback via intracortical microstimulation [25,26] may enhance control capabilities. Although we have shown sensory feedback is not required for cortical control of dexterous finger movements, it may help with the accuracy of BCFES movements as it has been shown to improve motor control [25]. Including a hand-state measurement glove, as discussed in the previous paragraph, could acquire the necessary signals to deliver valuable proprioceptive and forcerelated feedback in addition to improving BCFES performance.
For the purposes of translating BCFES for functional restoration beyond laboratory use, the technologies presented in this manuscript all represent low-power solutions end-to-end. The stimulation system used in this work is an evaluation system for the NNP, a low-power, wireless, completely implantable stimulator presently undergoing a clinical trial (NCT02329652). The BMI used the 300-1000 Hz SBP, a low-power neural feature that is specific to single-unit activity despite a ten-fold reduction in processing requirements. We have already demonstrated SBP's efficacy on embedded and integrated systems [56][57][58][59][60], and others have used it in similar BMI and BCFES applications [33,61,62]. While much work remains in system integration and safety validation, BCFES has a strong potential for full-time use in people with paralysis.

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.