Active upper limb prostheses: a review on current state and upcoming breakthroughs

The term biosignal indicates every possible signal that can be detected and measured from biological beings, humans—in our case. Usually, the term is used for signals of electric nature (i.e. EMG), but actually every signal collected from the activity of different tissues or organs belonging to the human body, can be considered as a biosignal.

We here adopt this latter definition to group input sources that are described next. Given its large use both in research and commercial ULP devices, EMG deserves a dedicated subsection, while other biosignals are grouped together. We also dedicate a whole subsection to brain-derived signals, which are especially used in brain-machine and brain-computer interfaces (BMIs, BCIs), but that are also showing potential use for ULP applications. Table 1 summarizes biosignals for ULP control that will be described in the following subsections.

3.1.1.1. EMG

While cosmetics, electronic components and computational efforts have undergone a significant improvement, the control strategies currently used in prosthetic applications have not changed since their first appearance in the 1960s (Schmidl 1965). The EMG has been one of the major sources to control ULPs (Merletti and Farina 2016). These signals carry information about neuromuscular activity, and they are used to retrieve human intention. EMG is indeed a technique for studying the activation of the skeletal muscles through the recording of electrical potentials produced by muscle contraction (Hudgins et al 1993). The theory behind the sEMG electrodes is that they form a chemical equilibrium between the detecting surface of the electrode and the skin of the body through electrolytic conduction, so that the current can flow into the electrode.

Multiple methods have been used to obtain the intended gesture from the processed EMG signals, all of which exploit the fact that the amputees can still generate different and repeatable muscular patterns related to each forearm movement with residual muscles of the stump. Low-density EMG is commonly used in prosthetic application, both in research and commercial context. Noteworthy, EMG signals can also be collected with invasive methods. The sEMG can be thus classified according to the level of resolution and density of the sensors. In the following, we provide an overview of the different types of EMG-based biosignals.

3.1.1.1.1. sEMG

The sEMG can be classified according to the number of electrodes used (figure 4). Low-density EMG generally refers to the use of a small (<10) number of EMG bipolar sensors, that can be either wet, i.e. contain an electrolytic substance that serves as interface between skin and electrodes, or dry (Jamal 2012). Conversely, high-density EMG is typically composed by wet monopolar sensors spread on a planar patch, around 1 cm apart, and with the ground reference generally placed on the wrist or on the elbow (Drost et al 2006). Importantly, sEMG electrodes also differ in their electronic configuration, as they can be either preamplified or not (Zheng et al 2021). Merletti and Muceli (2019) provided a guide with the best practice to acquire and manipulate EMG data according with the different aims, from signal analysis to motion prediction.

Zoom In Zoom Out Reset image size

Prosthetic control with low-density EMG is generally obtained by using two bipolar electrodes placed on antagonist muscles. This configuration allows the control of the prosthetic system in a robust and simple way (Hudgins et al 1993). However, the detection of complex and simultaneous movements of the phantom limb can be improved by using an array of EMG electrodes placed on the superficial skin of the residual forearm (Coapt 2017, Dellacasa Bellingegni et al 2017, Ottobock 2019, Marinelli et al 2020), The use of sEMG in prosthetic applications has become the most widespread source of information about voluntary movement (Schmidl 1965) because of the direct correlation between EMG activity and subjects' intentions.

Differently from the low-density, the high-density sEMG (HD-sEMG) is based on a higher number of electrodes placed on a small portion of the body. Recently, a growing number of researchers has focused on the use of these electrodes aiming to increase the amount of collected data, although at the cost of a greater computational burden. HD-sEMG sensors have been used to discriminate muscular patterns related to different gestures. Their signals can be handled in various ways to retrieve unique and repeatable information, as described in section 4. These sensors have to be positioned according to the distribution of the underlining muscle fibers and this configuration provides a low resolution map of the synergistic activation of the muscles during movement production (Winters 1990, Sartori et al 2018). For example, from contraction of the muscles under the acquisition grids, it is possible to extract bi-dimensional images, in which the EMG amplitude is mapped to a color scale. These maps can be thus handled by complex algorithms, as the ones used for objects detection in robotic navigation (Chen et al 2020). The main limitation of the HD-sEMG, which currently bounds its application to a laboratory scenario, is the skin-electrode contact since it requires conductive gel to reduce the interface impedance. The wet area is mainly needed to reduce artifacts in the EMG signals since it is generally acquired in monopolar configuration. Another disadvantage of this technique consists in the fact that computation is time-consuming.

Overall, the main drawback of sEMG-based approaches is constituted by the influence that skin impedance, sweat, and electrode shift have on the stability of the input signals (De Luca 1997). Additionally, muscle crosstalk and the difficulty to reach deep muscles further limit the quality of the collected signal. In the context of ULP, the use of sEMG can be further complicated by the fact that the amputation strongly affects muscles strength and organization and therefore signal quality, as discussed in section 6.4.

3.1.1.1.2. Invasive EMG and surgical procedures

The invasive approach has been exploited to explore the activity related to the production of movement for many years (Adrian and Bronk 1929) and it is still investigated by many groups. However, the main drawback of this approach is constituted by the surgery and by the technological barriers still faced by the available equipment. On the other hand, invasive electromyography (iEMG) allows to measure single motor unit action potentials, enabling a higher selectivity and a better accuracy of the input signal, overcoming the limitations imposed by sEMG. There are several examples of iEMG, which vary in the type of electrodes and level of invasiveness, as detailed hereafter.

EMG can be invasively detected by inserting electrodes into the internal surface of muscles (Merletti and Farina 2009). This invasive technique exploits two different percutaneous electrodes: needles and fine wires (Jamal 2012, Rubin 2019). The most used are needle electrodes. These electrodes are concentric, and their bare hollow needles contain an insulated fine wire into their cannula, which is exposed on the beveled tip, which is the active recording site. Wire electrodes are typically made of non-oxidizing and stiff materials with insulation, they can be implanted more easily and are usually less painful than needle electrodes.

Since both these sensors are percutaneous, i.e. passing through unbroken skin and leaving an open passage between the internal structures of the body and the external world, the risk of infection is quite probable. For this reason, and because of their intrinsic discomfort due to the percutaneous wire that can easily break, their usage is limited to laboratory research (Hargrove et al 2007, Cloutier and Yang 2013a). A detailed description of invasive electrodes both to record biological signals and to deliver electrical stimulation can be found in Raspopovic et al (2021a).

In the last decades, growing attention has been paid to the development of intramuscular electrodes that could be implanted under the skin of the subject to achieve the advantages of invasive sensors and simultaneously avoid the risks and inconvenience of percutaneous instruments. For example, Weir et al (2008) developed an implantable myoelectric sensor (IMES), a system able to receive and process up to 32 implanted sensors with wireless telemetry. A transcutaneous magnetic link between the implanted electrodes and the external coil allows reverse telemetry, which transfer data from the sensors to the controller, commanding the control of the prosthesis, and forward telemetry to supply power and configuration settings to the electrodes. These sensors are designed for permanent long-term implantation without any kind of servicing requirement and have been tested on animals. Four months after the implantation of IMESs in the legs of three cats, the sensors were still functioning (Weir et al 2008). Intramuscular electrodes have been used in prosthetic application to decode 12 different hand gestures from 4 healthy subjects (Cipriani et al 2014). Moreover, it has been shown that the application of this invasive approach enhances the simultaneous control of multi-DoFs system (Smith et al 2014).

Recently, the group of Ortiz-Catalan showed an invasive procedure for ULP control. They positioned EMG electrodes under the skin of amputated subjects and sutured them directly on the external surface of the muscles (Ortiz-Catalan et al 2020). More precisely, sensors were sewn onto the epimysium of the two heads of the biceps' muscles and the long and lateral heads of the triceps muscles. These invasive electrodes were used in combination with an osseointegrated prosthesis, i.e. a system obtained following a very invasive surgical procedure, which allows to anchor the prosthesis to the remaining limb's bone (Ortiz-Catalan et al 2020). In the context of ULP, osseointegration is offered for trans-humeral amputees, and the prosthesis is anchored to the humerus with two mechanical elements: the fixture, a screw made of titanium placed inside a hole made in the bone that becomes osseointegrated, and the abutment, placed within the fixture and extending outside of the body in a percutaneous way, onto which the prosthesis is connected. This technique was tested on four osseointegrated patients.

This latter example indicates that also surgical approaches can be taken to improve the quality of the collected EMG. A promising surgical technique that is performed in case of high-level amputation is targeted muscle reinnervation (TMR). This method was developed by the group of Kuiken in the early 2000s and consists in transferring residual arm nerves to alternative muscle sites. Following reinnervation, these target muscles are able to produce EMG that can be collected and used to control prosthetic arms (Kuiken et al 2009). This strategy works at the condition that each reinnervated muscle produces an EMG signal in response to only one transferred nerve, with the consequence that native nerves innervating the target muscle has to be cut during the surgical procedure to avoid unwanted EMG signals (Kuiken et al 2017). In the last 15 years, TMR has allowed intuitive control of ULP to several subjects with high-level amputation for whom standard ULP devices allowed a poor restoration of motor functions (Kuiken et al 2017). Importantly, given that it is performed on complex amputations, this technique is strongly tailored to each patient's physical and clinical status (Cheesborough et al 2015, Mereu et al 2021).

Recently, a new surgical method for improving EMG-based control has emerged: the regenerative peripheral nerve interface (RNPI) (Vu et al 2020a). Just as TMR, its goal is to turn a muscle into a biological amplifier of the motor command, in order to improve the quality of the EMG signal recorded, processed and used to drive the prosthesis. To this end, RNPI exploits the regeneration capabilities of nerves and muscles, to implant a transected nerve into a free muscle graft. Following regeneration, revascularization and reinnervation by the transected nerve, the muscle graft effectively becomes a stable peripheral nerve bioamplifier, able to produce high-amplitude EMG signals (Urbanchek et al 2012). The potential of this novel interface has been tested by Vu et al (2020b): they used EMG signals collected by intramuscular bipolar electrodes implanted into RNPIs obtained in amputated individuals, who could successfully perform real-time control of an artificial hand. Surprisingly, subjects were able to control the device with a high level of accuracy even 300 d post-implantation, without recalibration of the control algorithm.

Another surgical technique, not directly related to EMG signals but worth mentioning, is cineplasty, an old method revived in the last years with a new and more modern approach. This method was introduced for the first time by Vanghetti in 1899 and then replicated by Sauerbruch ten years later (Tropea et al 2017). It consisted of the direct mechanical linking of residual muscles and/or residual tendons of the affected limb to the prosthesis through external cables (i.e. Bowden cables). In 2001, Heckathorne and Childress (2001) implemented an evolution of this surgical solution for the control of one DoF ULP by exploiting exteriorized tendons directly linked to a force sensor.

3.1.1.2. Other biosignals

3.1.1.3. Brain signals

Besides the detection of physiological changes in residual muscles of the stump or in the brain during movements, described above, there are many other input sources and techniques capable or with the potentiality to control an upper limb prosthesis. Some of these are mainly used in the research field and since they lack usability, they do not find a real application in everyday life of amputees, or they are conceived for patients without the possibility to exploit other more convenient and intuitive sources (i.e. tetraplegic people). Some of them, instead, have been still only proposed as proof-of-concept.

The most studied approach is based on the use of IMUs. IMU sensors are cheap, small and can therefore by easily embedded in the prosthesis. They can increase the amount of data useful to successfully discriminate between different gestures of ULP during distinct phases of the reaching movement. These devices exploit accelerometers, gyroscopes and magnetometers to understand which is the actual altitude, position and orientation of the prosthesis. These sensors deliver information through quaternions and they are often used together with EMG to improve the classifier robustness (Georgi et al 2015). Zhang et al (2011) depicted the possibility to manipulate objects and perform complex tasks using both IMU and EMG sensors. As a matter of fact, the accelerometers can capture information that sEMG sensors cannot easily detect, such as hand withdrawal or rotation (Chen et al 2007). It has been shown that the use of IMU sensor coupled to EMG is more advantageous than increasing the number of EMG sensors (Fougner et al 2011). Similar results have been achieved by Krasoulis et al (2017), who have combined EMG and IMU to feed pattern recognition systems (see section 4.3.1). They demonstrated that this combination could significantly improve the real-time completion rates compared to the traditional methods, exclusively based on sEMG signals. Moreover, the data coming from IMU can be used alone to control a single module, usually the wrist or the elbow (Merad et al 2018) or to realize other types of control for rehabilitation purposes, such as shadow control, in which the control policy consists in replicating the movement captured by the IMU sensors (Rapetti et al 2020). These devices were also placed on feet to directly control an ULP by implementing precise foot movements (Resnik et al 2014). The adoption of IMU sensors is specifically promising in sensor fusion approaches, as discussed in section 3.1.3. Besides EMG signals, IMU data have been also combined with NIRS (Zhao et al 2019) and MMG (Wilson and Vaidyanathan 2017, Woodward et al 2017).

Table 2 summarizes the use of IMU and other input sources investigated for the control of ULP, many of which are described in (Grushko et al 2020).

An integrative input source is not used as the main responsible for the actual command of the prosthesis, but it is used to help and to facilitate its control, which usually depends on EMG signals. The integrative input sources work in parallel and together with the main ones, integrating their information and implementing the so-called data-fusion or sensor-fusion methods, see also section 4.3.3. Table 3 summarizes integrative sources found in the literature that have been used for ULP control.

Non-invasive feedback restoration for upper limb amputees is a hot topic in the research community, and yet it has not achieved broad clinical application (Sensinger and Dosen 2020). Many solutions have been proposed, but the main problem lays in their poor robustness. (Ribeiro et al 2019) highlighted the most widespread types of non-invasive feedback, described in table 4.

The most investigated feedback relies on the sense of touch and therefore consists of cutaneous stimulation. This can be performed with different modalities namely, vibrational, mechanotactile or electrical stimulation.

The vibrational feedback is generally implemented with the addition of eccentric rotating motors placed in contact with the skin surface of the stump (Ribeiro et al 2019). This method is generally employed to augment the robustness of the control system by providing the user with additional information regarding the position of the prosthetic device but it lacks intuitiveness, as the association between perceived sensation and the corresponding information has to be learned by the user. For example, in Bark et al (2014), the motors were placed in four distinct areas of the stump to guide the user through the desired trajectory while grasping object and the results showed a significant decrease in the root mean square angle error of their limb during the learning process. More recently, Markovic et al (2019) proposed a joint-oriented feedback criterion consisting of three vibromotors placed on the arm to provide the information on which joint is currently activated by the user, thus restoring proprioceptive sensation. The experiment was performed by 12 able-body subjects and 2 amputees controlling 3 DoF prosthesis, and it was found that the myoelectric multi-amplitude control outperformed the pattern recognition method when the feedback was applied.

Differently from the vibrational, the mechanotactile feedback is based on the application of linear actuators on the skin and provides pressure sensation. Antfolk et al (2013a) exploited this technique and proposed a multisite mechanotactile system to investigate the localization and discrimination threshold of pressure stimuli on the residual limbs of trans-radial amputees. They demonstrated that subjects were able to discriminate between different location of sensation and to differentiate between three different levels of pressure. This study demonstrated that it is possible to transfer tactile input from an artificial hand to the forearm skin after a brief training period. Recently, Svensson et al (2017) used it to translate the interaction between a virtual reality (VR) environment and a virtual hand into user sensation. The authors showed that by placing the tactile actuators in correspondence with the areas of the skin involved in object manipulation, subjects were able to feel a real touch sensation that increased their sense of body ownership. For example, pressure applied to the prosthetic fingers was perceived as a tactile sensation on the skin (Svensson et al 2017).

The electrical feedback is based on transcutaneous stimulation. The elicited sensations range from perception of pressure (Jorgovanovic et al 2014) to slip sensations (Xu et al 2015), depending on the electrical parameters (i.e. current amplitude, pulse frequency, pulse width). One advantage of this approach with respect to the vibrotactile and mechanotactile ones is the lack of moving components avoiding problems of electrode displacement and, thus, improving the sensors-skin contact. Nevertheless, it is important to take into account that the noise introduced by the electric stimulation can corrupt the acquisition of muscular activity, causing errors if the ULP is myoelectrically controlled. Moreover, the perceptions are not strictly confined to the zone under the stimulating device but they can spread in a wider region if the area above a nerve is considered.

Another sensory modality exploited for feedback delivery is the acoustic one. Gigli et al (2020) recently tested a novel acquisition protocol with additional acoustic feedback in 18 able-body participants to improve myoelectric control. The protocol consisted in dynamically acquiring EMG data in multiple arm positions while returning an acoustic signal to urge the participants to hover with the arm in specific regions of their peri-personal space. The results showed that the interaction between user and prosthesis during the data acquisition step was able to significantly improve myoelectric control. Auditory feedback has also been employed to convey artificial proprioceptive and exteroceptive information. Lundborg et al (1999) and Gonzalez et al (2012) employed auditory feedback by encoding the movement of different fingers into different sounds. The method demonstrated that the inclusion of auditory feedback reduces the mental effort and increase the human-machine interaction; furthermore, better temporal performance and better grasping performance were obtained.

In the last years, there have been some examples exploiting vision to deliver sensory feedback. Indeed, visual stimulation can be provided as explicit feedback through screens during game-like exercises, helping the prosthetic user to learn how to control the device (e.g. adjusting trajectory or grasping force) (Markovic et al 2018). However, adding sensory information to the prosthetic user's perceptual experience in real contexts requires solutions like augmented reality (AR, occurring when computer-generated items overlay a real setting) or mixed reality (MR, a term that represented different combinations of real and virtual items) (Milgram and Kishino 1994, Speicher et al 2019). AR and MR environments, implemented through wearable solutions like head-mounted displays, can support the actual control of a prosthetic device through visual feedback that does not occlude the real context (Clemente et al 2016, Markovic et al 2017, Hazubski et al 2020). However, they can also be used for prosthetic use training (Anderson and Bischof 2014, Sharma et al 2018)—in such a case, VR (a fully computer-generated setting) can offer visual feedback too (Lamounier et al 2010, Sun et al 2021b), especially within game-based frameworks (Nissler et al 2019) for engaging the users and motivating their activity.

There are different technologies that can be employed to provide a sensation directly to the nerve (Cutrone and Micera 2019, Raspopovic et al 2021a). The most used employ intrafascicular electrodes, such as TIME and wire and thin-film LIFE, which can both record muscle activity (e.g. iEMG) and stimulate nerves. Other solutions are characterized by the fact that the electrodes are placed around the nerves, such as cuff electrodes and FINEs.

The first example of ULP with sensory stimulation dates back to 1979 and it was based on the remapping between pressure signals acquired by prosthesis sensors to an amplitude-frequency modulation. This consisted of a series of pulses delivered with a pulse rate proportional to the increment of the pinch force and provided through dry electrodes placed over the skin in correspondence of the median nerve, as described in Shannon (1979). Later, the group of Micera employed thin-film intrafascicular electrodes longitudinally implanted in peripheral nerves (tf-LIFE4) to deliver electrical stimulation. With this method, they were able to elicit sensation of missing hand in the fascicular projection territories of the corresponding nerves and to modulate the sensation by varying the pulse width and pulse frequency (Benvenuto et al 2010). Importantly, this method avoids muscle crosstalk, fundamental for guaranteeing myoelectric control. More recently, new bioinspired paradigms have been suggested to better induce natural sensations (Raspopovic et al 2021a). In particular, the study of Oddo et al (2016) showed that it is possible to restore textural features recorded by an artificial fingertip. This device embedded a neuromorphic real-time mechano-neuro-transductor, which emulated the firing dynamics of SA1 cutaneous afferents. The emulated firing rate was converted into temporal pattern of electrical spikes that were delivered to the human median nerve via percutaneous microstimulation in one trans-radial amputee.

Valle et al (2018) suggested a 'hybrid' encoding strategy based on simultaneous biomimetic frequency and amplitude modulation. This kind of stimulation was perceived more natural with respect to classical stimulation protocol, enabling better performance in tasks requiring fine identification of the applied force. This paradigm was tested and validated during a virtual egg test (Valle et al 2018), where the subject needed to modulate the force applied to move sensorized blocks. This encoding strategy not only improves gross manual dexterity in functional task but also improved the prosthesis embodiment, reducing abnormal phantom limb perceptions.

Similarly, Osborn et al (2018) implemented a neuromorphic feedback paradigm based on Izikevich neuron model to generate the current spike train to inject directly in the median and ulnar nerves, using beryllium copper (BeCu) probes. Their prosthesis proposes a neuromorphic multilayered artificial skin to perceive touch and pain. Their transcutaneous electrical nerve stimulation allows to elicit innocuous and noxious tactile perceptions in the phantom hand. The multilayered electronic dermis (e-dermis) produces receptor-like spiking neural activity that allows to discriminate object curvature, including sharpness in a more natural sensation spanning a range of tactile stimuli for prosthetic hands. The authors were able not only to restore finger touch discrimination and objects recognition, but also to provide a pain sensation when the prosthesis touched sharp objects. In particular, they found that pain sensation is generated by a stimulation of 15–20 Hz.

Tan et al (2014) suggested that simple electronic cuff placed around nerves in the upper arm can directly activate the neural pathways responsible for hand sensations. This neural interface enabled the restoration of different sensations at many locations on the neuroprosthetic hand. Different stimulation patterns could transform the typical 'tingling sensation' of electrical stimulation into multiple different natural sensations, enabling the amputees to perform fine motor tasks and improving the embodiment.

In George et al (2019) a biomimetic method was described to restore both force and haptic sensation. The sensory feedback was implemented to restore the force sensation and promote objects recognition: USEA electrodes were used to deliver stimulation proportional to the variation of contact force exchanged between the prosthesis and the object during manipulation. Instead, the haptic sensation was based on the distribution of stimulation delivered during contact with the object with a fixed frequency and amplitude. The characteristic of this encoding scheme is based on electrical biphasic, charge—balanced of 200 or 320 µs phase durations. The biomimetic model describes the instantaneous firing rate of the afferent population using the contact stimulus position, velocity, and acceleration simulating all tactile fibers to any spatiotemporal deformation of the skin and hand. This strategy allows the amputee to augment the active exploration experience and to discriminate object size and stiffness.

Liu et al (2021b) have shown that primary afferents encode different stimulus features in distinct yet overlapping ways: scanning speed and contact force are encoded primarily in firing rates, whereas texture is encoded in the spatial distribution of the activated fibers, and in precisely timed spiking sequences. When multiple aspects of tactile stimuli vary at the same time, these different neural codes allow for information to be multiplexed in the responses of single neuron and populations of neurons. Exploiting this sensory architecture with invasive methods may lead to the development of prosthetic devices able to truly evoke natural sensations.

Another promising approach is targeted sensory reinnervation (TSR), i.e. the sensory version of TMR, which consists in coupling a pressure sensor placed on the prosthetic device to surgically redirected cutaneous sensory nerves (Marasco et al 2011). This technique strongly helps discrimination of objects size and stiffness during active exploration, especially if the tactile feedback is biomimetic (George et al 2019). Recently, Marasco et al (2021) have developed a prosthetic system based on both targeted sensory and motor reinnervation. TSR was used to deliver both touch and kinesthetic feedback. The authors showed that the system was able to significantly improve device control and promote embodiment.

These results indicate that, in order to close the loop on user and provide useful sensation (regardless the specific feedback modality), an optimal feedback control policy is necessary (Sensinger and Dosen 2020), as discussed in section 4.4.

Figure 6 illustrates the main machine learning methods employed for ULP control. These methods generally solve a pattern recognition problem in which, given the input signal, an output movement have to be identified.

Zoom In Zoom Out Reset image size

The first Pattern Recognition (PR)-based control schemes arose around the second half of 1960s (Scheme and Englehart 2011). In this configuration, the acquired EMG signals are elaborated by the controller to determine the action to be performed by the prosthesis. The five pillars of this computation process are: pre-processing, data segmentation, FE, classification, and post-processing. Each step is briefly described in table 5.

EMG-based pattern recognition controllers are now investigated by many groups and are even available in commercial prostheses (Coapt 2017, Ottobock 2019, i-biomed 2021).

The PR-based controllers apply linear and non-linear methods to classify the EMG signal into a possible large number of movements. The two main families of classification methods used in this context are regression (Hahne et al 2014) and classification techniques (Hudgins et al 1993). While the former is usually simple to implement and train, the latter are generally more difficult to employ. The embedding of neural networks (NN) in an ULP strictly depends on the structure of the algorithm (number of layers and neurons), since complex architecture requires high computational effort (Hagan et al 1997).

Statistical regression models usually produce good results in terms of high accuracy percentages. However, the out-of-laboratory results are particularly poor, because these techniques are extremely sensitive to changes of the input signals (Parajuli et al 2019). Motivated by this issue, in the last decade, many groups focused on classification-based techniques to implement more reliable decoders. Importantly, training classifiers requires longer than training linear models, however, the formers can achieve better results during real-time execution. Different classifiers have been exploited in ULP control such as support vector machine, regularized least squares, whereas the gold-standard is the linear discriminant analysis (Scheme and Englehart 2011, Cloutier and Yang 2013b, Di Domenico et al 2021). Among NN, the most common architecture is the multi-layer perceptron (MLP) (Amrani et al 2017, Shahzaib and Shakil 2018). The MLP is a supervised machine learning (ML) technique, which exploits labeled data to train the algorithm. It is characterized by three types of layers: input, hidden and output layer. The first one contains the same number of neurons as the input signals (for example, features extracted from EMG signals), the second stage can have one or more layers where there are all the trainable neurons, while the last layer comprises all the output nodes representing the results (for example, classification likelihood of each class of movement). Neurons of a certain layer are fully connected to the neurons in the next layer via nonlinear activation functions. However, as for the regression algorithms, the performance results obtained in the lab are not easily replicated in the real-life scenario. Moreover, the complexity of the controlled prosthesis (e.g. the number of DoFs) corresponds to a higher number of neurons in the NN, with important consequence not only on the computational burden, but also on the memory consumption.

When considering an increase in the number of controllable DoFs, current pattern recognition approaches demonstrated poor performance (Piazza et al 2020). As a matter of fact, to enhance the classification rate (i.e. number of correctly recognized movement) a greater content of information should be handled. The higher the amount of input data, the more complex would the ML algorithm be.

Therefore, HD-sEMG can be exploited to increase the amount of muscular information but this comes at the cost of higher computational burden. It has been proven that the use of this type of data can be helpful in increasing the robustness against electrode shift (Pan et al 2015), allowing an improvement of the classification by exploiting spatial images of the muscular contractions (Geng et al 2016), and for retrieving measures of motor unit potentials, which can be difficult to assess without invasive techniques (Merletti et al 2008).

Different techniques can be exploited to extract motor units' activity from the HD-sEMGs. The main used decomposition algorithm is the blind source separation (with the Convolution Kernel Compensation described by Holobar and Zazula (2007)) which seems to be the most suitable since it does not make any prior assumptions on the action potential shapes. The main problems related to this technique is the lack of a useful output for the prosthesis control, since the decomposition provides an extraction of principal features of the EMG signals. On the one hand, the algorithm returns reliable information about neural activity, but, on the other hand, it increases the computational burden required to the system. Indeed, the Holobar algorithm has been used together with ML algorithms to control robotic arm in real-time (Barsakcioglu and farina 2018).

Another approach includes the exploitation of ML algorithms where the input EMG signals are considered as numeric values and the definition of the output is based on a Black Box technique. Therefore, the mathematical tools contained in the Black Box do not take into account the biomechanics of the amputated limb and they are not specific for prosthetic applications.

It is relevant to feed the ML algorithm via a set of EMG signals (muscular patterns) specific for different prosthesis movements in such a way that the classifier does not misclassify. However, it is not always feasible to acquire the same signals for each movement due to different sources of errors (i.e. muscle fatigue, sweating, electrode misalignment). Indeed, more complex classifiers belonging to the DL field are exploited to make the control more robust. A possible application can be the use of convolutional neural network (CNN), which exploits dimensionality reduction to extract complex features from the activation maps of the HD-sEMG without dramatically increasing the computation time (Olsson et al 2019). This type of algorithm is also ideal for increasing the number of DoFs (and therefore the number of classes to be recognized) while keeping a quite high accuracy rate (Hartwell et al 2018). Moreover, Zhai et al (2017) has proved that the exploitation of CNN can help in removing issues of daily life noise, updating its feature map to include this new information, avoiding the need of periodical readjustment.

Adaptive technique based on reinforcement learning (Vasan and Pilarski 2017, Wu et al 2022) has been recently investigated, with the aim of facilitating the learning process of prosthetic use. This approach is promising as it points towards the development of a 'human–prosthesis symbiosis in which human motor control and intelligent prosthesis control function as one system', as defined by the group of Huang et al (2021).

Other DL algorithms take into account time series with feedback loops with prior hidden layers (Sun et al 2021a). This architecture allows storing the history of the input signals by considering the information of previous time instants, also resulting in performance improvements with respect to simpler DL architectures (Amado Laezza 2018).

Recently, novel DL strategies have also been proposed for ULP: recurrent NNs process temporal or sequential information; temporal convolutional networks take advantage of a one-dimensional convolution layer running along the time dimension to learn the time dependence of a given input signal (Li et al 2021); transformers are attention-based architectures applied to HD-sEMG data (Burrello et al 2022, Montazerin et al 2022).

Overall, the main problem related to ML applied to the bionic field is the evident gap between the results observed in a closed safe environment, such as a laboratory, and in real daily life (Resnik 2011).

To overcome the limitations of ML algorithms for ULP control, some groups investigated the model-based approach, which consists of an accurate description of the muscles and bones involved in the movements starting from the Hill model of muscle fiber (Winters 1990). For example, the neuromusculoskeletal model extracts from the residual EMGs the activation dynamics of the limb (Pan et al 2018, Zhao et al 2022). The activation dynamics combined with the kinematics of the limb produces the contraction dynamics. This consists of the modification of fiber length involved in the motion along the specific DoFs. In particular, Sartori et al (2018) implemented a control strategy based on the physiology and kinematics of a real hand and tested it with an amputated subject performing some complex grasping tasks. This approach needs a calibration step to scale the model to the subject specific activation EMGs. Results showed great stability over the noise introduced by sensors or movements artifacts. Moreover, the amputee was able to reproduce simultaneous multi-DoF gestures. The limitation of this approach is its susceptibility to electrode shift and fatigue condition that affects the EMG acquisition. The real-life scenario is yet to be tested, but preliminary results appear very promising (Sartori et al 2018).

For ULP control using different input sources together with or without EMG signals, other methods can be adopted. In case of force myography, the same algorithms used for EMG input can be applied. For example, machine learning techniques can be used to analyze and synthetize output starting from FMG input (Cho et al 2016). The adoption of other input signals different from EMG clearly requires the implementation of ad-hoc methods for their processing. For example, voice control introduces audio analysis method to detect and translate command into prosthetic movements (Mainardi and Davalli 2007, Alkhafaf et al 2020). Further, tongue control allows the motion of the prosthesis using a wireless controller resembling a dental retainer and providing the functionality of a wireless joystick or keyboard (Johansen et al 2016).

The high complexity of ULP control has led to the development of sensor fusion approaches, in which input signals of different nature are simultaneously collected and then processed to estimate the intended movement more reliably and accurately.

On low-density sEMG, we can find robust and semi-autonomous control solutions based on custom multi-amplitude algorithms, as those implemented on the Michelangelo hand with closed-loop multi amplitude control (CMAC) (Markovic et al 2019, Mouchoux et al 2021). The adoption of IMU sensors may lead to further improvements, such as the automatic adaptation to unexpected external factors, including sweat, muscle fatigue, mental stress, electrode re-positioning and weather conditions. The state-of-the-art algorithms have to cope with these challenging issues. Therefore, the combination of EMG and IMU as input to a classifier could provide useful localization information of the hand position, which could delete possible false positives, actively improving the obtained accuracy (Krasoulis et al 2017, 2019b). Moreover, it has been observed that integrating EMG, IMU and artificial vision sensors could benefit both the classifier accuracy and the increment of available DoFs (Mouchoux et al 2021). Other promising research advancements demonstrated that mixing EMG with FMG could lead to an improved multi-DoFs control as proposed by (Nowak et al 2020). Similarly, Jiang et al (2020) proposed a sensor fusion approach among EMG and FMG. Moreover, by fusing FMG and IMU, other interesting results were presented by (Ferigo et al 2017). In addition, other research activities treated NIRS fused with EMG (Guo et al 2017b) and IMU respectively (Zhao et al 2019).

In conclusion, a data fusion aims at compensating some of the main limiting factors of single input approaches (such as EMG-based or others) as these latter suffer from artifacts, electrodes shift, etc.

Among the invasive approaches, surgical procedures aimed at augmenting the signal containing the motor commands have gained popularity in the last decades. Indeed, TMR is now routinely adopted in case of trans-humeral amputation, and it allows to increase the number of myoelectric input sites, which can be exploited for multi-DoFs control. Differently from invasive procedures for electrodes implantation, TMR is permanent, turning the reinnervated muscles into natural bioamplifiers of motor commands. It is also adopted for phantom limb pain reduction (Mereu et al 2021).

Similarly, the more recent RNPI represents another promising technique for bioamplification of motor signals and its successful demonstration in experimental trials encourages their potential adoption in clinical practice (Vu et al 2020b).

As for delivery of feedback information, invasive approaches represent the more intuitive and natural solution towards the retrieval of sensations, as demonstrated in (Osborn et al 2018, Nguyen et al 2021). However, the main limitation toward the diffusion of these techniques is represented by their invasiveness, i.e. poor compatibility of electrodes, risk of infection due to external cables, scar tissue formation on the nerve, etc.

Ideally, a prosthetic limb should be a perfect replication of the natural limb, both in terms of control and perception, such as Luke Skywalker's arm in the Star Wars saga. In this scenario, control signals should directly derive from the brain and communicate the intended movement to the robotic device, while sensory information should be encoded into stimulation patterns delivered to the brain. The field of neuroprosthetics, among other things, aims at addressing these fascinating goals and in the last 50 years several progresses have been made, indicating that these visionary scenarios might one day become true.

Fetz (1969) demonstrated that monkeys could voluntarily modulate the firing rates of neurons in the primary motor cortex, in the absence of movement. At the same time, Humphrey et al (1970) were able to predict arm displacement from the activity recorded from small populations of neurons in the motor cortex. These exciting and pioneering works thus proved the possibility of controlling artificial devices with the mind and eventually led to a rapid flourishing of investigations aimed at interfacing the brain with machines. These studies culminated with the first demonstration by the group of Nicolelis, of a robotic arm controlled with signals produced by an ensemble of neurons recorded from the motor cortex of a rat (Chapin et al 1999). At the beginning of this century, BMIs were thus born and scientists were therefore hoping that in few decades, fully functional bionic limbs would have been routinely adopted by amputees and paralyzed individuals (Nicolelis and Chapin 2002). Sadly, this is not at all how the story ended. Indeed, more than 20 years after the first demonstrations of brain-controlled devices, we still do not have the technology nor the computational capabilities to effectively control artificial devices with cortical brain signals.

However, in the last few years, some groups have presented promising examples of paralyzed individuals with neural implants in the motor and premotor cortices controlling artificial limbs for several months/year, while other groups worked on non-invasive applications on neurological populations, as detailed in section 3.1.1.3. Although the target population of these studies is mostly composed by stroke or paralyzed patients, exploitation of results for prosthetics applications clearly emerges, i.e. the possibility to perform device control by reliably and timely accessing to the subject's motor intentions. These examples indeed demonstrate that groundwork for brain control of motor prosthetics has been laid. However, it has been limited to the lab and mostly addressing paralyzed patients, for whom there are currently not viable solutions to enable dexterous device control as for amputees, whose residual motor functions can be successfully leveraged for prosthetic control signals.

In sum, brain control approaches are still far from clinical and personal applications, not only because of the poor controllability that they exert over the prosthetic device, but mainly because of the cumbersome apparatus they need for their collection and processing. However, the dream of brain-controlled devices has spread outside the academic labs and has contaminated also visionary entrepreneurs from venture capitals and tech giants, with the consequent birth of some important companies interested in brain-interfacing technology, such as Neuralink (Musk 2021), Facebook Reality Labs (Zuckerberg 2021), and Google DeepMind (Deepmind 2021). In conclusion, cutting edge research that we are currently witnessing both in academic and non-academic contexts may thus soon push the envelope of neuroprosthetics up to its diffusion in our everyday life, with important consequences also for amputees.

While great progress has been made in recognizing human motor intention and translating it into prosthesis joint movements, sensory feedback restoration is one of the many challenges that many research groups are still addressing (see section 3.2) (Farina et al 2021). Obtaining a reliable and efficient way to artificially convey sensory information to prosthetic users would allow to develop smart devices able to truly mimic the behavior of human limbs, establishing the premise for a true co-adaptation between user and machine.

However, such a sensorimotor prosthesis would still need to face technological barriers for its development and use in ADLs. For example, all the circuitry and components needed to operate the device (i.e. acquire signals for motor intention decoding and translate sensory information into stimulation patterns) should fit into the socket space. One possible solution to obtain an embedded device is to rely on a hybrid approach. For example, a non-invasive sensorimotor prosthesis could use EMG signals to extract motor intentions and vibrotactile feedback to deliver sensory information. This configuration requires the embedding of an analog to digital converter (ADC) amplifier to acquire EMG data, and motor drivers to control vibromotors. All these components could be placed on the same board, overcoming the problem of electrical coupling.

However, although this design is very articulated, we believe that this is not the real end point for a fully-integrated prosthetic system capable of maximizing patient acceptance. It simply turns out to be a developmental test bench for what will be the next generation of prostheses on a longer temporal horizon, namely neuroprostheses. By creating a modular architecture, in fact, it will be possible to maintain the physical prosthetic system and to progressively replace the non-invasive input and feedback sources with implanted ones, namely: (a) the standard sensors for surface EMG replaced by an intraneural implant; (b) the hardware for feedback delivery replaced by a chronic implant; and (c) the hardware on which run the control strategies replaced by a smaller and more performing dedicated hardware able to process neuromorphic algorithms (e.g. Application specific integrated circuit (ASIC)).

The realization of bidirectional ULP systems able to restore both sensory and motor functions will open new research scenario for embodiment process and neuroplastic phenomena. Indeed, about neuroplasticity in amputees using bionic hands as prostheses, Di Pino et al (2009) highlighted how the reorganization of the central nervous system after the usage of the device can be the source of indices of prosthetic effectiveness in functional recovery. Furthermore, the effects of the device on the central nervous system can make the prosthesis work as a neurorehabilitative solution mitigating aberrant plasticity phenomena and facilitating positive neural changes. Finally, novel human-machine interfaces should consider neuroplasticity principles for restoring the efferences and afferences of the central nervous system with the lost limb in order to exploit them for connecting a prosthesis.

The processes described above are an excellent expression of the technologies that can lead to a user-prosthesis co-adaptation. They care for providing the human with sensations matching the motor activity and the events occurring on or for the artificial limb, which needs to learn how to offer appropriate feedback to the user. Next paragraphs will describe how this can happen.

Establishing interactions between user and prosthesis requires a human-centered design of the technology 'behavior'. Since the term co-adaptation implies that two or more entities are adjusting to each other, possibly learning through the interaction itself for reaching a goal that can be the improvement of the human-machine performance. The attention to the interactive aspects of learning and training was recommended in Castellini et al (2016). In this context, intriguing opportunities come from theoretical frameworks in psychology like the constructivism to define and improve the paradigm of interactive ML in prosthetic training and control (Nowak et al 2018, Bettoni and Castellini 2021). Accordingly, a myocontrol system should learn and forget on demand, under request of each one of the components of the human-machine system. For instance, the users can label the violation of their expectations on the prosthesis interpretation of their commands, starting a novel data collection cycle. Additionally, the machine itself can highlight the need of collecting further data (especially from correct execution of a training exercise) through acoustic feedback to the user. Through this, the myocontrol models are updated.

Moreover, certain biosignal features could guide an automated labeling of the violation of the observer's expectations without any explicit command, as in neuroprosthetic interfaces using Error-related Potentials (Chavarriaga et al 2014). Such an advance (still explored in laboratories) could enable self-calibrating intention detection processes, leading to a true (and fruitful) human-machine symbiosis. Such symbiosis is based on the online processing of users' neurocognitive states. The machine interprets such states for implicitly adjusting (without direct and declarative commands of the user) its activity to the individual current capabilities and preferences. Exploiting this process as a further example of the closed-loop previously envisioned, the human-machine system will become a fully functional unit able to enact manual and bimanual biomimetic behaviors (Chavarriaga et al 2014).

However, when the practical applications of this kind of implicit learning will move outside the laboratories, its advantages should be evaluated in real-world cases of prosthetic learning. Currently, a human-machine explicit communication component in initial co-training sessions can highlight an active role of the users with positive impact on their self-efficacy and engagement. In this context, biofeedback strategies (self-regulation of perceptually represented physiological changes) in co-adaptive systems could be especially useful for easing the integration of both implicit and explicit aspects within the user-prosthesis co-training (even during daily re-calibrations) (Kalampratsidou and Torres 2017).