We describe a novel virtual-reality based setup which exploits voluntary control of one hand to improve motor-skill performance in the other (non-trained) hand. This is achieved by providing real-time movement-based sensory feedback as if the non-trained hand is moving. This new approach may be used to enhance rehabilitation of patients with unilateral hemiparesis.
As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by observation or passive movement of trainee's hands by a robotic device). This obviously presents a major challenge in the rehabilitation of a paretic limb since voluntary control of physical movement is limited. Here, we describe a novel training scheme we have developed that has the potential to circumvent this major challenge. We exploited the voluntary control of one hand and provided real-time movement-based manipulated sensory feedback as if the other hand is moving. Visual manipulation through virtual reality (VR) was combined with a device that yokes left-hand fingers to passively follow right-hand voluntary finger movements. In healthy subjects, we demonstrate enhanced within-session performance gains of a limb in the absence of voluntary physical training. Results in healthy subjects suggest that training with the unique VR setup might also be beneficial for patients with upper limb hemiparesis by exploiting the voluntary control of their healthy hand to improve rehabilitation of their affected hand.
Physical practice is the most efficient form of training. Although this approach is well established1, it is very challenging in cases where the basic motor capability of the training hand is limited2. To bypass this problem, a large and growing body of literature examined various indirect approaches of motor training.
One such indirect training approach uses physical practice with one hand to introduce performance gains in the other (non-practiced) hand. This phenomenon, known as cross-education (CE) or intermanual transfer, has been studied extensively 3,4,5,6,7,8,9 and used to enhance performance in various motor tasks 10,11,12. For instance, in sport skill settings, studies have demonstrated that training basketball dribbling in one hand transfers to increased dribbling capabilities in the other, untrained hand 13,14,15.
In another indirect approach, motor learning is facilitated through the use of visual or sensory feedback. In learning by observation, it has been demonstrated that significant performance gains can be obtained simply by passively observing someone else perform the task16,17,18,19,20. Similarly, proprioceptive training, in which the limb is passively moved, was also shown to improve performance on motor tasks 12,21,22,23,24,25,26.
Together, these lines of research suggest that sensory input plays an important role in learning. Here, we demonstrate that manipulating online sensory feedback (visual and proprioceptive) during physical training of one limb results in augmented performance gain in the opposite limb. We describe a training regime that yields optimal performance outcome in a hand, in the absence of its voluntary physical training. The conceptual novelty of the proposed method resides in the fact that it combines the three different forms of learning – namely, learning by observation, CE, and passive movement. Here we examined whether the phenomenon of CE, together with mirrored visual feedback and passive movement, can be exploited to facilitate learning in healthy subjects in the absence of voluntary physical movement of the training limb.
The concept in this setup differs from direct attempts to physically train the hand. At the methodological level – we introduce a novel setup including advanced technologies such as 3D virtual reality, and custom built devices that allow manipulating visual and proprioceptive input in a natural environmental setting. Demonstrating improved outcome using the proposed training has key consequences for real-world learning. For example, children use sensory feedback in a manner that is different from that of adults27,28,29 and in order to optimize motor learning, children may require longer periods of practice. The use of CE together with manipulated sensory feedback might reduce training duration. Furthermore, acquisition of sport skills might be facilitated using this kind of sophisticated training. Finally, this can prove beneficial for the development of a new approach for rehabilitation of patients with unilateral motor deficits such as stroke.
The following protocol was conducted in accordance with guidelines approved by the Human Ethics Committee of Tel-Aviv University.The study includes 2 experiments – one using visual manipulation, and another combining visual with proprioceptive sensory manipulation. Subjects were healthy, right handed (according to the Edinburgh handedness questionnaire), with normal vision and no reported cognitive deficits or neurological problems. They were naïve to the purpose of the study and provided written informed consent to take part in the study.
1. Setting up the Virtual Reality environment
2. Conducting the experiment
NOTE: See Figure 1 for the experimental stages. Each subject underwent three instruction-evaluation-train-evaluation experimental sessions. The details of the instructions and evaluation stages are provided in the Representative results section.
3. Analyzing the behavioral data and calculating performance gains
36 subjects in two experiments trained to execute rapid sequences of right hand finger movements while sensory (visual/proprioceptive) feedback was manipulated. Fingers were numbered from index (1) to little finger (4) and each subject was asked to learn three different sequences in three consecutive experimental sessions such as: 4-1-3-2- 4, 4-2-3-1-4, and 3-1-4-2-3. Each sequence/session was associated with a specific training type and the association between sequence and training type was counterbalanced across subjects. At the beginning of each session, subjects were presented with an instruction slide that depicted two hand illustrations (right and left) with numbered fingers and a specific 5 number sequence underneath, representing the sequence of finger movements to be learned (see Figure 1). The instructions slide (12 s) was followed by the pre-training evaluation stage (30 s). At this stage, online visual feedback consisted of a display of two virtual hands whose finger movements were yoked in real-time to the subjects' actual finger movements (virtual hands were based on a model available in 5DT gloves toolbox). Thus, real left hand movement was accompanied by visual feedback of left (congruent) virtual hand movement. Subjects were instructed to repeatedly execute the sequence as fast and as accurately as possible with their left hand. In the following training stage, subjects trained on the sequence under a specific experimental condition in a self-paced manner. The training stage contained 20 blocks, each training block lasted 15 s followed by 9 s of yellow blank screen, which served as cue for resting period. We used 20 training blocks, which in our case were sufficient to obtain significant differences between conditions. Finally, a post-training evaluation stage identical to the pre-training evaluation was conducted. Each subject underwent three such instruction-evaluation-train-evaluation experimental sessions. Each experimental session was associated with a unique training condition and finger sequence. In experiment 1, we compared the G index values across the following training conditions: (1) training by observation – subjects passively observed the virtual left hand performing the sequence while both their real hands were immobile; (2) CE – subjects physically trained with their right hand while receiving congruent online visual feedback of right virtual hand movement; (3) CE + Visual manipulation (VM) – importantly, the VR setup allowed us to create a unique 3d experimental condition in which subjects physically trained with their right hand while receiving online visual feedback of left (incongruent) virtual hand movement (CE + VM condition). Left virtual hand finger movement was based on real right hand finger movement detected by the gloves (step 1.4). In all conditions – the palm of the subjects' hands were facing up. The pace of virtual hand finger movement in the training by observation condition (condition 1) was set based on the average pace of the subject during previous active right hand conditions (conditions 2 and 3). In cases where the order of training conditions due to counterbalancing was such that training-by-observation was first, the pace was set based on the average pace of the previous subject. All G index comparisons were performed in a within-subject paired-fashion across the different training conditions.
Left hand performance gains following training in condition 3 (CE + Visual manipulation) were significantly higher relative to the gains obtained following training by left hand observation (condition 1; p<0.01; two-tailed paired t-test) or following right hand training with congruent visual feedback – the traditional form of CE (condition 2; p<0.05; two tailed paired t-test; Figure 2 and Table 1). Interestingly, the training with incongruent visual feedback (CE + VM) yielded higher performance gain than the sum of gains obtained by two basic training types: physical training of the right hand, and training by observation of left-hand without physical movement. This super additive effect demonstrates that performance gains in the left hand are non-linearly enhanced when right hand training is supplemented with left hand visual feedback that is controlled by the subject. This implies that CE and learning by observation are interacting processes that can be combined to a novel learning scheme.
We also examined in another set of 18 healthy subjects whether the addition of passive left hand movement can further enhance left hand performance gains. To this end in study 2, subjects underwent a similar protocol with 3 training types while their hands were placed inside the aforementioned custom-built device (step 1.7) that controls left hand finger movement. In this experiment, subjects trained for 10 blocks. Each training block lasted 50 s followed by 10 s of a yellow blank screen which served as cue for resting period. The following three training types were used: (1) CE + VM – cross education accompanied by manipulated visual feedback (similar to condition 3 from study 1); (2) CE+PM – standard cross-education (i.e. right hand active movement + visual feedback of right virtual hand movement), together with yoked passive movement (PM) of the left hand; (3) CE+VM+ PM – subjects physically trained with their right hand while visual input was manipulated such that corresponding left virtual hand movement was displayed (similar to condition 3 used in the first study). However, in addition, right hand active finger movement resulted in yoked passive left hand finger movement through the device.
The addition of passive left-hand finger movement to the visual manipulation, yielded the highest left-hand performance gains (Figure 3 and Table 2), that were significantly higher than performance gains following the visual manipulation alone (condition 1; p<0.01; two-tailed paired t-test). It should be noted that although the CE+VM training condition was similar to that in study 1, absolute G values are only comparable across conditions within the same study. This is due to the fact that (1) training design was slightly different (in study 2 the palms faced down and not up due to the device, different duration/number of training blocks) and (2) each experiment was conducted on a different group of subjects. Importantly, within each study, each subject performed all three training types and G indices across conditions are compared in a paired fashion.
Figure 1. Experiment Design. Schematic illustration of a single experimental session in study 1. Each subject performed 3 such sessions. In each session, a unique sequence of five digits was presented together with a sketch of the mapped fingers. After instructions, subjects performed the sequence as fast and as accurate as possible using their left hand for initial evaluation of performance level. Next, subjects trained on the sequence by one of the training types (see representative results) in a self-paced manner. After training, subjects repeated the evaluation stage for re-assessment of performance level. In study 2, the overall design was similar, with different durations/amount of training blocks (detailed in the representative results). Hands in the illustration represent the active hand only (the visual feedback always contained two virtual hands). Please click here to view a larger version of this figure.
Figure 2. Study 1 – left hand performance gains. Physical training with the right hand while receiving online visual feedback as if the left hand is moving (CE + visual manipulation; VM; red) resulted in highest left hand performance gains relative to the other training conditions examined: left hand observation (yellow), and cross-education without visual manipulation (i.e. right hand training + congruent visual feedback of right virtual hand movement; green). Error bars denote SEM across 18 subjects. Please click here to view a larger version of this figure.
Figure 3. Study 2 – left hand performance gains. The highest left hand performance gain was obtained when cross education with visual manipulation was combined with passive left hand finger movement by the device (CE+VM+PM; light red). This improvement was significantly higher than that obtained following cross education with visual manipulation (CE+VM; red) and cross education with proprioceptive manipulation (CE+PM; green). Error bars denote SEM across 18 subjects. Please click here to view a larger version of this figure.
Table 1. Study 1 data. Individual subject's performance (P) during pre- and post-training evaluation stages in study 1. Each cell represents the number of correctly performed complete 5-digit sequences within 30 s. S – Subject number. Please click here to download this Table.
Table 2. Study 2 data. Same as Table 1 for study 2. Note that training duration and hand orientation in this experiment were different than experiment 1 (see text). Please click here to download this Table.
We describe a novel training setup and demonstrate how embedding virtual sensory feedback in a real-world environment optimizes motor learning in a hand that is not trained under voluntary control. We manipulated feedback in two modalities: visual and proprioceptive.
There are few critical steps in the presented protocol. First, the system consists of several separate components (gloves, VR headset, camera, and passive movement device) that should be carefully connected while setting up the VR environment. To that end, the experimenter should keep the exact order described in the protocol and verify subjects' convenience.
The combination of visual and proprioceptive manipulation during training introduced significantly higher performance gains in the non-trained hand relative to other existing training types such as learning by observation17, and CE3 with and without passive hand movements24,25,26.
It is an open question whether the enhanced performance gains in the current demonstration generalizes to other tasks, training durations, feedback modalities or hand identities (left active hand, or bi-manual movements). The current study was limited to right-handed subjects using a simple finger sequence task. Additionally, the proprioception manipulation in the current setup is based on a system that allows very limited movements (such as finger flexion/extension) for a relatively short-term training. Further work is required to establish the generalizability of the presented setup to other types of behaviors.
The current setup can be extended in several ways. First, new types of modalities can be added for example, binding different auditory sounds to different finger movements during the sequence task. This might result in a supra-additive effect which will further optimize learning in the untrained hand. Second, the current design of the system enables an easy swap between the voluntary moving hand (right hand in the current description) and the passively yoked hand (left hand). Future studies can capitalize on this flexibility to examine how directionality of transfer (between dominant and non-dominant hands3) can modify the level of performance gains when using the presented sensory manipulations. Finally, the unique VR setup we developed may be adapted to more complex tasks (as opposed to the simple finger sequence task). Virtual simulation of external objects such as balls, pins, and boards can be embedded into the real environment providing a rich and engaging training experience.
As for future applications, the effect described in this study can be readily used with clinical populations such as patients with upper-limb hemiparesis by introducing physical training with the healthy hand and providing visual feedback as if the affected hand is moving. Given that voluntary control of the affected limb is limited in such populations, this training scheme has the potential of circumventing the challenges of direct physical therapy of the affected hand and perhaps resulting in better recovery rates30,31. This approach, exploiting the phenomenon of cross-education and mirror-therapy, together with well-established rehabilitation tasks, has not been previously tested in clinical patients and has the potential for providing a more efficient rehabilitation regime. Finally, since this setup is partially MR compatible, it enables the use of whole-brain functional magnetic resonance imaging (fMRI) to probe the relevant neural circuits engaged during such training12.
The authors have nothing to disclose.
This study was supported by the I-CORE Program of the Planning and Budgeting Committee and the Israel Science Foundation (grant no. 51/11), and The Israel Science Foundation (grants no. 1771/13 and 2043/13) (R.M.); the Yosef Sagol Scholarship for Neuroscience Research, the Israeli Presidential Honorary Scholarship for Neuroscience Research, and the Sagol School of Neuroscience fellowship (O.O.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors thank E. Kagan and A. Hakim for help with data acquisition, Lihi Sadeh and Yuval Wilchfort with filming and setup, and O. Levy and Y. Siman-Tov from Rehabit-Tec System for providing access to the passive movement device.
Oculus Development Kit 1 | Oculus VR | The Oculus Rift DK1 is a virtual reality headset developed and manufactured by Oculus VR, and contains development kit. | |
5DT Data Glove 14 MRI Right-handed and left handed | Fifth dimension Technologies | 100-0009 and 100-0010 | The 5DT Data Glove Ultra is designed to satisfy the stringent requirements of modern Motion Capture and Animation Professionals. It offers comfort, ease of use, a small form factor and multiple application drivers. The high data quality, low cross-correlation and high data rate make it ideal for realistic realtime animation. |
PlayStation Eye Camera | Sony | The PlayStation Eye (trademarked PLAYSTATION Eye) is a digital camera device, similar to a webcam, for thePlayStation 3. The technology uses computer vision and gesture recognition to process images taken by the camera. | |
REHABILITATION SYSTEM REHABIT-TEC | Rehabit-Tec | www.rehabit-tec.com | The Rehabit-Tec Rehabilitation system is a rehabilitation system intended to allow a CVA injured individual advance self rehabilitation on the basis of mirror movements |