Transcranial magnetic stimulation, electromyography, and 3D motion capture are commonly used non-invasive techniques for investigating neuromuscular function in humans. In this paper, we describe a protocol that synchronously samples data generated by all three of these tools along with the unique addition of virtual reality stimulus presentation and feedback.
The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating “real-world” environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system’s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.
Virtual reality (VR) is rapidly becoming an accessible research tool for use in a number of fields, including the study of human motion. The study of upper limb movement is especially benefited by incorporating VR. Virtual reality permits the rapid customization of experimental parameters designed to investigate specific kinematic and dynamic features of arm movement control. These parameters can be individually adjusted for each subject. For example, the locations of virtual targets can be scaled to ensure identical initial arm posture across subjects. Virtual reality also allows the manipulation of visual feedback during experiments, which is an invaluable tool in visuomotor research1–5.
The use of realistic VR environments with other biomechanical tools will also permit naturalistic movement scenarios in which to test movement patterns. This arrangement is becoming increasingly valuable to the study and practice of rehabilitation after disease and injury6,7. Mimicking naturalistic movements and environments (e.g. performing movements in a virtual kitchen) in a clinical setting will enable rehabilitation specialists to more precisely describe an individual’s impairments in a real-world context. Highly individualized impairment descriptions will allow for more focused treatment strategies, potentially increasing the efficacy and reducing the duration of rehabilitation.
Combining VR with other tools such as transcranial magnetic stimulation (TMS), surface electromyography (EMG), and full body motion capture, creates an extremely powerful and flexible platform for studying the neuromuscular control of movement in humans. Transcranial magnetic stimulation is a powerful non-invasive method of measuring the excitability and functional integrity of descending motor pathways (e.g. corticospinal tract) through EMG responses such as motor evoked potentials (MEPs)8. Modern three-dimensional motion capture systems also enable researchers to study neuromuscular activity together with resulting movement kinematics and dynamics. This permits the creation of extremely detailed models of the musculoskeletal system as well as the testing of hypotheses regarding the structure and function of neural controllers. These studies will expand our scientific knowledge of the human sensorimotor system and lead to improvements in treatment of musculoskeletal and neurological disorders.
However, one major problem with multifunctional systems is the synchronization of separately recorded data streams (e.g. motion capture, EMG, etc.). The goal of this protocol is to describe a generalizable arrangement of common commercially available systems to simultaneously record biomechanical and physiological measurements during movement. Other investigators using equipment from different manufacturers may have to alter elements of this protocol to fit their specific needs. However, general principles from this protocol should still be applicable.
All participants involved in experimentation undergo informed consent procedures approved by the West Virginia University Institutional Review Board (IRB).
1. Overall System Characteristics, Design, and General Experimental Task
Note: The complete setup is comprised of the following major components: EMG equipment and associated digital acquisition (DAQ) equipment; a motion capture system (this protocol incorporates an active LED system); a TMS unit with a figure-of-eight coil and stereotaxic localization equipment; a VR headset and associated computer and software; and a custom synchronization circuit. Figure 1 schematically outlines the connectivity between the protocol components.
Figure 1: Connectivity of entire setup. This layout describes the general connectivity between the elements of our system. The synchronization circuit is described elsewhere in the text in more detail. The blue trace corresponds to the signal that starts both motion capture and EMG data streams. This event is the source of the temporal delay of up to 190 msec using the equipment described in this protocol. The red trace corresponds to the VR-initiated synchronization event that is concomitantly recorded by the motion capture and EMG systems and subsequently used for temporal alignment of the respective data streams. Please click here to view a larger version of this figure.
2. General Details of System Integration and Synchronization
Note: Synchronization of the separate data acquisition systems in this protocol (motion capture and EMG) is accomplished through the use of an event signal that is common to all recording streams. Using a common event, all of the signals can be temporally realigned after data collection to minimize real-time recording discrepancies (upwards of 190 msec using the equipment in this protocol). In this protocol, the common signal originates from the VR system as a parallel port signal. The common signal is routed to a circuit that permits synchronization of the separate data streams through direct recording with EMG signals and by simultaneously turning off a motion capture LED. The circuit is constructed using basic tools and techniques for building electronic components, similar to circuits described elsewhere9.
Figure 2: Trial flowchart. This flowchart outlines the stimulus and signal events that occur during a typical experimental trial that includes TMS stimulation. Parallel port codes that occur throughout a trial are shown in the DB25 schematic symbols (light blue).
Figure 3: Synchronization Circuit. This schematic displays the layout of our custom synchronization circuit. The default output of the NAND gate is a high voltage state; this voltage output is sent to the gate of a transistor through which the sync LED’s circuit is routed. This default state renders the circuit closed, which maintains the LED in a lighted state. Upon receiving a sync trigger parallel port signal (red trace in inset), an internal state of the 555 device is flipped rendering the output into a high state, shutting off the LED (blue trace). When this occurs, the voltage on C1 (green trace) builds up to a voltage that resets the internal state of the 555, reactivating the LED. The parallel port sync trigger signal is also directly routed to a BNC connector that is connected to the TMS input trigger port. Note: The direction of this trigger signal may have to be reversed (from positive- to negative-going or vice-versa) depending on an investigator’s specific equipment requirements. The addition of an “inverter” chip on this trigger output would easily accomplish this task. Please click here to view a larger version of this figure.
3. Experimental Procedures
Synchronization of the numerous data streams in this setup allows one to record the kinematics, continuous muscle activity (EMG), and instantaneous neuromuscular activity (MEPs) that occur during movements of the upper limb. Repeated trials of a given movement are required to reconstruct MEP response profiles over an entire movement. Figure 4 displays data collected from one subject. Figure 4A shows an example of these data streams during a single trial with the corresponding synchronization signals and events. Temporal alignment of the signals with respect to the synchronization event is a simple post-hoc procedure using signal analysis software (the signals are “shifted” in time using the synchronization event as a common temporal anchor). Signals can then be time-normalized by the duration of each movement trial. Without synchronization, the EMG and motion capture data streams can have a temporal discrepancy as great as 160-190 msec. However, by utilizing synchronization in addition to widely used TTL signaling, users should expect to minimize temporal errors between data streams to the limit of the sampling frequencies of their signals (approximately one msec in this example). Figure 4B shows average angular kinematics and dynamics across 24 trials for a single movement, the long head of the biceps EMG profile from trials without TMS during the same movements, and the corresponding reconstructed MEP profiles from trials with single-pulse TMS during movement to the same targets.
Figure 4: Alignment of EMG and Motion Capture. (A) Representative signals that are recorded during an experimental trial are displayed in the left column of charts. The blue and red circles correspond to the same VR-generated synchronization event recorded by two separate pieces of equipment (illustrated by dividing black line). These time points and respective data are later temporally aligned using custom software. The difference between these two time points can be upwards of 190 msec using when using the equipment described in this protocol; other investigators using different equipment may experience different delays. (B) After temporal alignment, averaged data can be created to describe the physiological, kinematic, and dynamic features of a movement. These data represent 24 trials of the same movement; the bars on the Bicep MEPs graph and the shaded areas on other graphs represent standard deviation. These data can subsequently be used to describe potential descending motor control signals with respect to muscle activity and movement kinematics and dynamics.
The objective of this article is to describe a method for incorporating VR into the study of human motion and a method for synchronizing various data streams. Virtual Reality will expand the capabilities of researchers that attempt to recreate real-world movement scenarios in a laboratory setting. Combining VR with other neuromuscular recording and stimulus methodologies forms a powerful suite of tools for comprehensively studying human motor control mechanisms. The resulting multidimensional datasets obtained during meticulously designed experiments can deepen our understanding of the neural control of movement.
One of the more important features of this system is the ability to synchronize electrophysiological and motion capture data streams with common VR-generated events. The custom circuit described in this protocol serves as a flexible, cost-effective foundation that can be altered to satisfy the unique requirements of other experimental paradigms and equipment, similar to solutions in other fields9. The common synchronization event is a parallel output command that originates from the computer that operates our VR software. The benefits of a standard parallel interface are its simplicity, speed, and flexibility. Within a parallel interface there are eight independent data lines, each representing a binary digit from 20 to 27; the sum of these digits can equal a range of numbers from 0 to 255. Each of the respective data lines can be utilized as separate and simultaneous trigger signals to interface with numerous systems. These trigger signals are usually simple square-wave voltage signals, commonly referred to as TTL signals or pulses.
During a movement trial, the common synchronization event is initiated based upon a participant’s location in a virtual environment tracked using an infrared LED-based motion capture system. The synchronization event signal (TTL) from our VR software is routed to the custom circuit which is designed to simultaneously transmit the VR synchronization event to our EMG data and motion capture streams (Figure 3). The EMG system records the TTL pulse with ongoing muscle activity. The VR signal is also routed through the active portion of the circuit, which controls the power supply to an LED from the motion capture system. Upon receiving the TTL pulse, the re-routed LED is turned off for a short period of time. This event is recorded by the motion capture system and is temporally synchronous with the TTL pulse recorded by the EMG system. This event can subsequently be used to align the signals for analyses.
The active portion of the circuit (schematic shown in Figure 3) is primarily based upon a specific integrated circuit (IC) or “chip”, commonly known as a “555 timer circuit”16. The output of the 555 timing circuit (normally a low voltage) enters into a NAND (Negated AND) gate along with a constant voltage provided by the USB power. A NAND gate is an electrical logic component that outputs a low value (i.e. 0V) when the two inputs are high (e.g. rail voltage). The inset in Figure 3 details the operation of our circuit upon receiving a synchronization event signal. The duration that the circuit turns off the LED depends on the values used for R1 and C1, and is found by the equation: t = 1.1*R1*C1. The currently described experiment required resistance and capacitance values of one megaohm and one microfarad, respectively, to produce synchronization light quiescence shorter than the duration of a typical movement (approximately one second for this design).
The current protocol’s method for synchronization has numerous benefits over commercially available options. The circuit components and necessary tools for its assembly are readily available at electrical component suppliers for minimal cost9. Additionally, a simple hardware-based solution for synchronization allows experimenters to more easily debug problems that may arise during experimental sessions. Finally, by utilizing fairly ubiquitous TTL signaling, one can easily adapt to new experimental designs that utilize different methodologies and equipment (e.g. EEG). A potential disadvantage of the multifunctional system described in this protocol is the complexity of experimental setups with numerous data collection systems. This can result in long experimental sessions, participant fatigue, and multiple opportunities for system failures. Experimenters can minimize problems through designing succinct experimental paradigms that aim to investigate very specific neuromuscular phenomena.
The circuit and overall synchronization procedure implemented in this protocol aimed to provide generalizable guidelines for performing biomechanical experiments with multiple, simultaneously recorded data streams. The protocol describes procedures to synchronize data streams from any equipment with analog inputs or triggers or LED signals. However, investigators using passive motion tracking systems without LEDs, will likely have to alter the currently described solution. Systems with passive motion capture and other recording and stimulating equipment that is digitally triggered will not need to rely on the synchronizing circuit. Instead, such systems would rely upon custom software-based solutions, the design of which can be inferred from the example of the current system. Thus, the protocol provides generalizable principles to assist designing solutions for other unique scenarios.
The authors have nothing to disclose.
This work was supported by NIH grant P20 GM109098, NSF and WVU ADVANCE Sponsorship Program (VG), and WVU departmental start-up funds.
Transcranial magnetic stimulator | Magstim | N/A | TMS stimulator and coils |
Impulse X2 | PhaseSpace | N/A | Motion capture system |
MA300 Advanced Multi-Channel EMG System | Motion Lab Systems | MA300-28 | EMG pre-amplifier and amplifier |
Norotrode EMG electrodes | Myotronics | N/A | EMG electrodes |
BNC-2111 Single-Ended, Shielded BNC Connector Block | National Instruments | 779347-01 | BNC Connector Block |
NI PXI-1033 5-Slot PXI Chassis with Integrated MXI-Express Controller |
National Instruments | 779757-01 | DAQ chassis |
NI PXI-6254 16-Bit, 1 MS/s (Multichannel), 1.25 MS/s (1-Channel), 32 Analog Inputs |
National Instruments | 779118-01 | DAQ card |
SHC68-68-EPM Cable (2m) | National Instruments | 192061-02 | Shielded cable |
DK1 or DK2 | Oculus VR | N/A | Ocuclus Rift headset |
Vizard 5 Lite | WorldViz | N/A | Virtual reality software |
C1 and C2 capacitors | varied | N/A | Adjust values to suit |
R1 and R2 resistors | varied | N/A | Adjust values to suit |
CD4011 NAND gate | varied | N/A | NAND gate |
2N2222 transistor | varied | N/A | Transistor |
NE555 timer circuit | varied | N/A | Timer circuit |
DB25 and USB connectors | varied | N/A | parallel and USB connectors |