As prosthetic development moves towards the goal of natural control, harnessing amputees’ inherent ability to learn new motor skills may enable proficiency. This manuscript describes a structured rehabilitation protocol, which includes imitation, repetition, and reinforcement learning strategies, for improved multifunctional prosthetic control.
Advances in robotic systems have resulted in prostheses for the upper limb that can produce multifunctional movements. However, these sophisticated systems require upper limb amputees to learn complex control schemes. Humans have the ability to learn new movements through imitation and other learning strategies. This protocol describes a structured rehabilitation method, which includes imitation, repetition, and reinforcement learning, and aims to assess if this method can improve multifunctional prosthetic control. A left below elbow amputee, with 4 years of experience in prosthetic use, took part in this case study. The prosthesis used was a Michelangelo hand with wrist rotation, and the added features of wrist flexion and extension, which allowed more combinations of hand movements. The participant’s Southampton Hand Assessment Procedure score improved from 58 to 71 following structured training. This suggests that a structured training protocol of imitation, repetition and reinforcement may have a role in learning to control a new prosthetic hand. A larger clinical study is however required to support these findings.
Replacing hand function in amputees is a difficult endeavor. Coordinating highly skilled hand movements is not an innate ability, and takes humans years of learning to develop.1–5 After the traumatic loss of a hand, replicating this capability by prosthetic means is not a trivial task and may require a period of sustained learning.
Prosthetic design and interfacing methods for their control are subject to rapid technological innovations, with the goal of multifunctional control in a natural manner.6 The complexity of these control systems increases substantially to provide more functions for amputees. To ensure accurate control of these systems, and to reduce abandonment of new technologies, adequate training needs to be established. This is likely to be more successful if it is based on the amputees’ inherent learning strategies.
Vision can play an important role during learning of hand movements. Behavioral studies have shown that by observing the actions of others7 or using visual cues8, able-bodied individuals learn and coordinate new movements. Through a process of observation, understanding and execution of an observed action, individuals are able to imitate the actions of others. Specific cortical networks, which may include a mirror-neuron system (MNS), are believed to underlie this capability, and may have a role in controlling prosthetic limbs.9–11
The role of imitation might not just be limited to executing actions that have already been seen, but together with the MNS, allow the execution of movements that have not yet been observed but extrapolated from the observer’s motor repetoire.12 Indeed, imitation may not necessarily be an innate ability, but an accruement of motor skills over time that lead to experienced and sophisticated actions.13 The strength of observing actions, over just simply imagining them, has been shown to improve learning new tasks.14 Thus, imitation may be a pragmatic approach to training amputees, as evidence suggests it a goal directed process15, with the target in the rehabilitation setting of enabling useful prosthetic hand function.
Rehabilitation studies have separately shown that visual cues, such as virtual simulations of a prosthetic hand, encourage amputees during rehabilitation training.16 In addition, the use of repetition when conducted in a blocked paradigm has been shown to enable rapid learning of upper-limb prosthetic control.17 While virtual simulations have been proven to be equally effective as real control of prosthetic hands in enabling abled-body users to control myoelectric devices,18 their effect on amputees using standardized outcome measures is not clear. Finally, where protocols for upper limb amputation training exist, the role of imitation in learning of prosthetic control is not explicitly discussed.19,20
This study aims at understanding if the use of imitation, in combination with repetition and reinforcement, has a positive impact on the learning of multifunctional prosthetic control as part of a structured training program.
Presented herein is a case report of a transradial amputee who was trained to use a multifunctional prosthetic hand. The participant had previously become accustomed to operating traditional myoelectric prostheses. Using visual cues, both in the form of imitation of a healthy demonstrator and as simple computer visual feedback, the amputee quickly improved handling of his new device.
This study was carried out in accordance with the Declaration of Helsinki, as approved by the local research ethics committee. The study was explained in full detail to the participant prior to commencement, allowing the participant the time to weigh up the decision to voluntarily take part in the study and confirm his participation by informed, written consent.
Note: One man, aged 27 years, took part in the study. The participant had normal vision, was a left below-elbow amputee, and was an experienced user (4 years total prosthesis use). Prior to commencing this study the prosthesis he used on a daily basis was a 4 channel myoelectric prosthetic hand with wrist rotation for 12-15 hr per day for 15 months. The participant’s right hand had previously been surgically reconstructed, but had no other physical or neurological impairment.
1. Study Design
2. Materials
3. Control Algorithm
4. Software Framework
Note: The software framework used in this study allowed handling of the communication between the prosthetic hardware and the embedded control algorithm. It also offered visually supportive training tools needed for maximizing participant training.
5. Naïve Session
Note: During the naïve training session, the participant had no prior experience of the prosthetic control scheme used in this study.
6. Structured Training Session
7. Prosthetic Control
8. Outcome Measurement
The baseline SHAP performance of the participant with his daily prosthesis was 81 when measured by the clinical staff 8 months prior to testing. A SHAP score of 100 represents able-bodied hand function.24 The participant scored an overall SHAP score of 58 during the naïve session with the more advanced prosthesis control system. However, 3 months later and with no further interaction with the new system, aside from the structured training, the participant achieved a SHAP score of 71 with the same advanced system (Table 2).
When the overall SHAP score was broken down into functional profile assessment, it was observed that the participant had performed well in all functional groups (spherical, power, tip, lateral and extension grasp), except for tripod grasp. However, the largest observed improvement was during extension, a function that the new control scheme and prosthesis provided while his traditional prosthesis did not (Figure 5). This may have also contributed to the improvement in spherical grasp, which was better after the structured training session than the baseline or the naïve session. In addition complex ADL movements, which involved combined movements of the wrist and hand, such as jug and carton pouring were executed best after the structured training session using the advanced prosthetic system.
Figure 1. Example of visual cues used for participant reinforcement and system training. The blue target profile represents the desired level of EMG contraction produced during a certain movement. The red tracking line represents the participant’s efforts. Please click here to view a larger version of this figure.
Figure 2. The profiles for the active movements, referred to as polar-plots, of the individual movements produced by the participant during the imitation task. These were reinforced during system training and eventually used to control the prosthetic hand. Please note, that rest or no-movement is considered a unique action, and as such does not produce an overlay. Please click here to view a larger version of this figure.
Figure 3. This schematic represents the structured training session. The participant first observed and imitated the actions of demonstrator. Before viewing his performance as graphs on a computer screen, he repeated the learned movements with no visual feedback. The learned movements were reinforced by matching muscular contractions to recorded EMG patterns, and then used to train the system’s control algorithms, which enabled multifunctional prosthetic control.
Figure 4. Experimental setup during the structured training session. The participant had a full and unobstructed view of the demonstrator’s left hand during imitation. During the repetition phase, the demonstrator would give audible instruction to ensure the participant’s movements matched the contractions produced during the imitation phase. Finally, during system training, the movements were reinforced using visual cues that were displayed on the computer screen to both the participant and demonstrator.
Figure 5. The breakdown of overall SHAP scores between, the baseline (BL) the naïve session (NS) and the structured training session (STS). Please click here to view a larger version of this figure.
Prosthetic Function | Phantom Limb Motion |
Pronation | Wrist rotation inwards with fully relaxed fingers |
Supination | Wrist rotation outwards with fully relaxed fingers |
Flexion | Ulnar deviation |
Extension | Wrist extension |
Palmar grip | Thumb adduction slightly crossing posteriorly towards the back of the hand |
Fine pinch | Opposition of thumb to the first three fingers, slight extension of the little finger |
Hand open | Opening of the hand with focus on the extension of the middle three digits |
No movement | Full relaxation of the hand and wrist |
Table 1. Desired prosthetic functions mapped to the phantom limb motions, which the participant was capable of visualizing and executing with his remaining anatomy.
Abstract Objects | |||||||
BL | NS | STS | BL | NS | STS | ||
Light Sphere | 2.46 | 2.66 | 2.5 | Heavy Sphere | 3.25 | 4.78 | 2.1 |
Light Tripod | 2.35 | 3.56 | 2.78 | Heavy Tripod | 2.44 | 3.53 | 2.5 |
Light Power | 2.41 | 3.25 | 2.28 | Heavy Power | 2.41 | 3.22 | 2.72 |
Light Lateral* | 4.72 | 2.81 | 4.97 | Heavy Lateral | 5.1 | 5.31 | 5.22 |
Light Tip | 2.25 | 2.88 | 2.53 | Heavy Tip | 3.1 | 4.47 | 2.22 |
Light Extension | 1.96 | 3.88 | 2.37 | Heavy Extension | 2.9 | 4.88 | 2.59 |
Activities of Daily Living | |||||||
BL | NS | STS | BL | NS | STS | ||
Coins | 17.81 | 22.25 | 21.53 | Full Jar | 3.13 | 10.37 | 3.75 |
Button Board | 8.25 | 35.2 | 27.06 | Empty Tin | 2.53 | 4.15 | 2.82 |
Cutting | 18.15 | 27.47 | 25.59 | Tray Lift | 3.97 | 7.25 | 5.5 |
Page Turning | 8.18 | 11.97 | 5.19 | Key | 4.82 | 9.25 | 6.03 |
Jar Lid | 2.93 | 3.3 | 2.38 | Zip | 4.83 | 10.59 | 7.31 |
Jug Pouring | 10.16 | 12.37 | 8.93 | Screwdriver | 10.1 | 25.31 | 15.31 |
Carton Pouring | 11 | 11.35 | 9.72 | Door Handle | 2.24 | 3.53 | 2.75 |
SHAP score | 81 | 58 | 71 |
Table 2. SHAP results for the participant during the naïve session (NS), followed by the structured training session (STS) 3 months later, compared to his baseline (BL). *The participant only underperformed the light lateral task in the structured training session in comparison to the naïve session. The overall SHAP score is out of 100.
Our findings suggest for the participant in this study that structured training helped improve control of a multifunctional prosthetic hand during one single session. The structured program used here was a combination of imitation, repetition and reinforcement of hand movements that the participant was not able to complete with his traditional prosthetic hand.
Although the participant scored higher with his traditional prosthesis in the SHAP test, it is worth noting that he typically wore that device between 12-15 hr per day over a period of 15 months. As documented by the baseline SHAP score, it is clear that he had learned and become accustomed to his traditional prosthesis after a very long learning period. The difficulty in switching to the multifunctional hand after being so accustomed to his traditional prosthesis was emphasized by the sharp drop in performance observed in the naïve session. This was expected, as evidence suggests that as an individual learns new motor skills, they develop an internal model of the actions being performed.25 When there is some form of perturbation in this internal model, such as changing to a new prosthesis requiring new control inputs, the after effects of learning take some time to dissipate while a new internal model is created.26 Nevertheless, a single session of structured training allowed the participant to outperform his usual device in some of the tasks requested by the SHAP test, and to reach on overall score close to that obtained with the traditional device. The use of structured training as outlined in Section 6 of the protocol may be the critical step that might have enabled the participant to achieve proficient control.
Learning a new task for amputees is complicated by the absence of nerve receptors around the joints and in the muscles which are sensitive to positional and movement changes.27 These proprioceptors enable able-bodied humans to know where their hands are in relation to their body without the use of sight.28 When a limb is lost, these proprioceptors are lost, leading vision to play a stronger role in control than in normal conditions. Amputees must not only relearn how to control hand movements, but also have to do so using a device that provides no feedback other than that obtained visually. This makes the learning process more difficult.
As such, any training strategies that use prostheses that provide no tactile or proprioceptive feedback must place an emphasis on visual feedback. In our case, we attempted to do so using imitation of the desired movements. The complexity of imitation is exemplified by the distributed nature of the neural process.29,30 Separate regions in the frontal, temporal and parietal lobes are believed to be responsible for perceiving the motion of others31,32 and then integrating this information into an appropriate motor response.9,33,34 It is likely that during the participant’s development into adulthood, and prior to the amputation, the neural circuitry required to perform learnt hand movements had become clearly defined, so much so that natural hand movements were fast and instinctive. The distortion of anatomy following the amputation may have required new neural circuits to be formed to enable control of his traditional prosthetic limb. The improvement in SHAP score following the structured training session, suggests that these neural circuits were malleable enough to adapt to the new prosthetic control strategy, despite the lack of experience.
It is worth noting that the participant commented that the act of imitation allowed him to internally visualize hand movements and to generate the appropriate muscular contractions. He found this more intuitive than solely matching his contractions to visual representations on a computer screen. It is also known that amputees prefer to learn from other prosthetic users.11 The device and control algorithms used in this study were both novel. As such there were no previous experienced amputees who could act as demonstrators. Future improvements to this protocol would thus benefit from having an experienced amputee demonstrating the actions to be imitated.
While this study showed the benefit of structured training, the design was not sufficient to determine whether imitation, repetition, reinforcement or the combination of all three learning strategies contributed to the final outcome measure. Instead, this case study lays the foundation for further work to examine the neural circuitry involved in advanced prosthetic control.
The authors have nothing to disclose.
The authors wish to thank Mr Hans Oppel and his prosthetic technicians of Otto Bock Healthcare Products GmbH for manufacturing the socket used by the participant in this study. This study was financially supported by the European Research Council (ERC) via the ERC Advanced Grant DEMOVE (No. 267888), the Austrian Council for Research and Technology Development, and the Austrian Federal Ministry of Science, Research & Economy.
Michelangelo Hand | Otto Bock Healthcare Products GmbH, A | 8E500=L-M | |
AxonRotation | Otto Bock Healthcare Products GmbH, A | 9S503 | |
Wrist Flexor | Otto Bock Healthcare Products GmbH, A | – | prototype unit |
AxonMaster | Otto Bock Healthcare Products GmbH, A | 13E500 | |
Electrode | Otto Bock Healthcare Products GmbH, A | 13E200=50AC | |
ScissorFenceElectrodeCarrier | Otto Bock Healthcare Products GmbH, A | – | prototype unit |
Acquisition Software | Otto Bock Healthcare Products GmbH, A | – | prototype unit |
Carbon shaft | Otto Bock Healthcare Products GmbH, A | – | prototype unit |