Summary

WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control

Published: August 15, 2020
doi:

Summary

WheelCon is a novel, free and open-source platform to design video games that noninvasively simulates mountain biking down a steep, twisting, bumpy trail. It contains components presenting in human sensorimotor control (delay, quantization, noise, disturbance, and multiple feedback loops) and allows researchers to study the layered architecture in sensorimotor control.

Abstract

Feedback control theory has been extensively implemented to theoretically model human sensorimotor control. However, experimental platforms capable of manipulating important components of multiple feedback loops lack development. This paper describes WheelCon, an open-source platform aimed at resolving such insufficiencies. Using only a computer, a standard display, and inexpensive gaming steering wheel equipped with a force feedback motor, WheelCon safely simulates the canonical sensorimotor task of riding a mountain bike down a steep, twisting, bumpy trail. The platform provides flexibility, as will be demonstrated in the demos provided, so that researchers may manipulate the disturbances, delay, and quantization (data rate) in the layered feedback loops, including a high-level advanced plan layer and a low-level delayed reflex layer. In this paper, we illustrate WheelCon's graphical user interface (GUI), the input and output of existing demos, and how to design new games. In addition, we present the basic feedback model and the experimental results from the demo games, which align well with the model's prediction. The WheelCon platform can be downloaded at https://github.com/Doyle-Lab/WheelCon. In short, the platform is featured to be cheap, simple to use, and flexible to program for effective sensorimotor neuroscience research and control engineering education.

Introduction

The human sensorimotor control system is extremely robust1, although the sensing is distributed, variable, sparse, quantized, noisy and delayed2,3,4; the computing in the central nervous system is slow5,6,7; and the muscle actuation fatigues and saturates8. Many computational theoretical models have been proposed to explain the complicated human sensorimotor control process4,9,10,11,12,13,14, which is a tradeoff process in human reach and response15,16. For example, feedback control theory predicts the optimal control policy12, Bayesian theory models sensorimotor learning17,18,19 and information theory sensorimotor foundation20,21. In contrast to the abundance of theoretical models, experimental platforms capable of manipulating important components of multiple feedback loops lack development. This is in part due to the fact that designing a platform to bridge and test these aspects of sensorimotor control requires a diverse range of expertise, extending from motor control theory, signal processing, and interaction, all the way to computer graphics and programming. Researchers often develop their own custom hardware/software systems to characterize human sensorimotor control performance, which can limit the ability to compare/contrast and integrate datasets across research groups. The development of an easy-to-use and validated system could broaden the quantitative characterization of sensorimotor control.

In this paper, we present the WheelCon platform, a novel, free and open-source platform to design video games for a virtual environment that noninvasively simulates a Fitts’ Law reaching game and a mountain bike task with downing a steep, twisting and bumpy trail. The Fitts’ law for reaching task quantifies the tradeoff between speed and accuracy in which the time required for reaching a target of width at distance scales as22,23. The 'mountain-bike task' is a combination of a pursuit and compensatory tracking task, which are two classic components of research on human sensorimotor performance, especially in terms of studying feedback loops.

WheelCon contains the highly demanded basic components presented in each theory: delay, quantization, noise, disturbance, and multiple feedback loops. It is a potential tool for studying the following diverse questions in human sensorimotor control:

• How the human sensorimotor system deals with the delay and quantization in neural signaling, which is fundamentally constrained by the limited resources (such as the space and metabolic costs) in the brain24,25;
• How neural correlation in the human cortex with sensorimotor control26;
• How humans handle the unpredictable, external disturbances in sensorimotor control27;
• How the hierarchical control loops layered and integrated within human sensorimotor system16,28,29;
• The consequence of the delay and quantization in human visual feedback30 and reflex feedback31 in sensorimotor control;
• The optimal policy and strategy for sensorimotor learning under delay and quantization16,17,24,29.

WheelCon integrates with a steering wheel and can simulate game conditions that manipulate the variables in these questions, such as signaling delay, quantization, noise, and disturbance, while recording the dynamic control policy and system errors. It also allows researchers to study the layered architecture in sensorimotor control. In the example of riding a mountain bike, two control layers are involved in this task: the high-layer plan and the low-layer reflex. For visible disturbances (i.e., the trail), we plan before the disturbance arrives. For disturbances unknown in advance (i.e., small bumps), the control relies on delayed reflexes. Feedback control theory proposes that effective layered architectures can integrate the higher layers' goals, plans, decisions with the lower layers' sensing, reflex, and action24. WheelCon provides experimental tools to induce distinctive disturbances in the plan and reflex layers separately for testing such a layered architecture (Figure 1).

We provide a cheap, easy to use and flexible to program platform, WheelCon that bridges the gap between theoretical and experimental studies on neuroscience. To be specific, it can be used for examining the effects of delay, quantization, disturbance, potentially speed-accuracy tradeoffs. The variables that can be manipulated in control loops are shown in Table 1. It can also be applied for studying decision making and multiplexing ability across different control layers in human sensorimotor control. Moreover, WheelCon is compatible with noninvasive neural recordings, such as electroencephalography (EEG), to measure the neural response during sensorimotor control32,33,34,35, and the non-invasive brain stimulation techniques, such as Transcranial Electrical Stimulation (tES) and Transcranial Magnetic Stimulation (TMS), to manipulate the neural activity36,37.

Protocol

The development and application of the protocol were approved by the California Institute of Technology Institutional Review Board (IRB) and the Southern University of Science and Technology IRB. The subject provided informed consent prior to any procedures being performed.

1. System preparation and setup

  1. The recommended basic hardware is a 2 GHz dual-core processor and 4 GB of system memory.
  2. Build the gaming platform under the Unity platform, while using C# programing language. The Logitech gaming wheel driver and Logitech Steering Wheel SDK are needed for gaming platform development.
  3. The gaming platform executable files only support Windows 10 Operating System (OS). Therefore, on a PC running Windows 10, download and install the corresponding racing wheel driver. Then download the compressed WheelCon software (https://github.com/Doyle-Lab/WheelCon/archive/master.zip) and extract the files to the local hard drive.
  4. Mount the racing wheel securely at the sitting level in front of a monitor, and then connect the wheel's USB cable to the PC and the power adapter to an outlet.
  5. Start the driver GUI to test for correct input readout and force feedback. Importantly, keep the driver GUI running in the background during the test.
  6. To start the program, double-click on WheelCon.exe in the 'WheelCon-masterExecutable & Output Files' directory.
  7. On the configuration screen, choose settings for monitor and click Play! (Figure 2a). The main menu will appear. Make sure the display size and location are as specified.
    NOTE: The 'Wheel Sensitivity' value, defining cursor speed, ranges from 0 to 1, and defaults to 0.5. In case the range of motion afforded by the racing wheel does not suit specific task parameters, adjust this value. For example, decrease the sensitivity for the aging population. However, for comparing between tasks, it is necessary to keep this value constant for the battery and across groups.

2. Task implementation

  1. Fitts' law reaching game
    NOTE: The Fitts’ law reaching game simulates the reaching process. The subject requires to turn the wheel to place the vertical line into the desired region (Figure 2d).
    1. Seat the subject comfortably behind the wheels. Adjust the wheel height if necessary.
    2. On the main menu, click Fitts' Law Task (Figure 2b) and type in a name for the output file indicating subject identification and task information on the textbox.
    3. Click on Select File, choose Fitt's Law.txt in the 'WheelCon-master Demo Input Files' directory, and then click Begin Game.
    4. Instruct the subject to move the green vertical line with the wheel to place it within the gray zone. This task serves to familiarize the subject with maneuvering the wheel, as well as with the color convention used throughout different tasks.
  2. Mountain bike tasks
    NOTE: The mountain bike task is a combination of pursuit and compensatory tracking task. It simulates riding a mountain bike down a steep, twisted and bumpy trail. The subject can see the trail and turn the wheel to track it, while a motor can torque the wheel to mimic invisible bumps in the trial (Figure 2e).
    1. Game 1: Testing the effect of the visual delay
      NOTE: In this game, the length of the look-ahead window (advanced warning vs. delay) is manipulated.
      1. On the main menu, click Mountain Bike Task (Figure 2c) and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Vision_Delay.txt in the 'WheelCon-master Demo Input Files' directory, and then click on Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
    2. Game 2: Testing the effect of action delay
      NOTE: In this game, a delay of various lengths is added between wheel movement and action output.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Action_Delay.txt in the 'WheelCon-master Demo Input Files' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
    3. Game 3: Testing the effect of visual quantization
      NOTE: In this game, visual input is quantized to limit the data rate.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Vision Quantization.txt in the 'WheelCon-master Demo Input Files' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
    4. Game 4: Testing the effect of action quantization
      NOTE: In this game, action output is quantized to limit the data rate.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Action Quantization.txt in the 'WheelCon-master Demo Input Files' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
    5. Game 5: Testing the effect of bump and trail disturbance
      NOTE: This task consists of three scenarios:
      a) "Bumps", tracking a constant trail subject despite torque disturbances on the wheel that mimic hitting bumps when riding a mountain bike;
      b) "Trail", tracking a moving trail with random turns but without bumps;
      c) "Trail with Bumps", tracking a moving trail with random turns and bumps.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Bump & Trail.txt in the 'WheelCon-master Demo Input Files' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.

3. Data output

  1. Locate the TXT output file in the 'WheelCon-masterExecutable & Output FilesMountainBikeData' directory, and then open with Matlab' WheelCon Data Analysis Code.m' in the 'WheelCon-masterSource Code' directory.
  2. Specify in the MATLAB script the folder and file_names variables according to the output file directory, and then run the script (Ctrl + Enter), and the output variables will be saved as column vectors to the Workspace. The error and control policy will be exported for each sampling time. See Table 2 for the detailed description.

4. Input file development

  1. Open 'WheelCon Mntn Bike Trail Design Code.m' in the 'WheelCon-masterSource Code' directory.
  2. Uncomment (Ctrl + T) the section for the desired game parameters and run the script (Ctrl + Enter). The input file will be saved in the 'WheelCon-masterSource Code' directory' in .txt format. Each column in the input files is one control variable. Refer to Table 1 for the list of control variables.

Representative Results

Modelling Feedback Control

We show a simplified feedback control model shown in Figure 1. The system dynamics is given by:

Equation 1

where x(t) is the error at time t, r(t) is the trail disturbance w(t), is the bump disturbance, and u(t) is the control action.

Modeling Action Delay in Trail Disturbance

When there is a delay T in action, and a trail disturbance r(t), we model the control action by

Equation 2

The game starts with zero initial condition: x(0) = 0. The controller κ generates the control command u(t) using the full information on the histories of state, disturbance, and control input. Here, the net delay T is composed of the internal delays in the human sensorimotor feedback and the delays externally added. The control command is executed with delay T ≥ 0. Sensorimotor control in the risk-aware setting motivates the use of L1 optimal control, and as such, the goal is to verify the following robust control problem

Equation 3

This problem admits a simple and intuitive solution. The optimal cost is given by

Equation 4

This optimal cost is achieved by the worst-case control policy u(t + T) = −r(t), which yields

Equation 5

Modeling Action Quantization in Trail Disturbance

When the data rate, R, in the control loop is limited, the control action is generated by the following feedback loop with communication constraints,

Equation 6

where Equation 12 is a controller, and Equation 13 is a quantizer with data rate R ≥ 1, i.e. S is a finite set of cardinality 2R. The disturbance r(t) is infinity-norm bound and without loss of generality, Equation 14. The worst-case state deviation is lower-bounded by

Equation 7

and the minimum control effort is given by

Equation 8

Measures of Error

To quantify the performance, we measured the infinity norm error Equation 15, mean absolute error (MAE) and root mean square error (RMSE). The infinity norm is defined as the maximum of the absolute errors, where

Equation 9

Mean absolute error is calculated as follows

Equation 10

Root mean squared error is calculated as follows

Equation 11

Game 1: Visual Advanced Warning or Delay

Game 1 evaluates how the length of the look-ahead window (advanced warning/delay) affects sensorimotor control performance without being exposed to additional disturbances.

Game 1 lasts for 360 seconds and consists of one continuous "Trail", which reduces the amount of look-ahead every 30 seconds. The game begins with 1 s of advanced warning, and then decreases to 0.75 s, and then to 0.5 s. From there, the game decreases the look ahead by 0.1 s until a minimum of -0.4 s is reached. Positive delay, or negative advanced warning, means only the trail behind the player is visible.

An evolution of error dynamics of the player as the game progresses with 1 s advanced warning and 0.4 s delay were depicted in Figure 3a-3b separately. Both the plot display only the middle 20 seconds of each of the 30-second intervals to neglect the effects of the player adjusting to the new look-ahead window. The progression of the error in the blocks looks stable in the 1 s advanced warning setting while in the 0.4s delay setting, the error flips upside down during the progress. To quantify that effect in more detail, we evaluate L1-/L2-/L– norm for the error dynamics for every 20 s group corresponding to a delay level. Summarizing these calculations in a plot gives Figure 3c, which demonstrates how the players' error-norm does not change until the advanced warning reaches 0.5 s and then increases in an approximately linear fashion.

Game 2: Delay in Action Output

Unlike Game 1's external visual delay, Game 2 adds specific internal delay to the action output; in other words, the current control policy u(t) works at u(t + Tact) where Tact is the external delay in action. Game 2 lasts for 180 s. Adjusting Tact every 30 s, Tact starts at 0 s, and increments by 0.1 s until it reaches 0.4 s.

The effects of delay in action are shown in Figure 4. Similar to the vision delay, the error increases linearly with the delay, which is well in line with the prediction from theory in Eq(3).

Game 3 and Game 4: Quantization in Vision input and Action Output

Game 3 and Game 4 study the effects of quantization in vision input and action output, respectively. Each game is 210 s long, and the quantization changes every 30 s, with the data rate increasing from 1 to 7 bits. For example, when the Rvis is 1 in Game 3, the desired position (gray line in the gaming GUI) is presented either in the center-left or the center-right of the screen. When Rvis = n, the desired position can be presented in 2n possible locations on the screen. For Game 4, when Ract = 1, the player is either going left or right with one speed. When Ract is n, the player can steer the wheel to go left or right with 2n-1 speeds.

The effects of quantization (limited data rate) in the vision and action are shown in Figure 5. Consistent with the theory's prediction in Eq(6), sensorimotor control performance improves with higher data rates and reaches the optimal control performance when R is around 5.

Game 5: Bump and trail disturbance

Game 5 is designed to test the effects of bump and trail disturbances on human sensorimotor control. Game 5 consists of three scenarios:

a) "Bumps", tracking a constant trail subject despite torque disturbances on the wheel that mimic hitting bumps when riding a mountain bike;
b) "Trail", tracking a moving trail with random turns but without bumps;
c) "Trail with Bumps", tracking a moving trail with random turns and bumps.

Each scenario lasts for 60 s in the order (Bumps, Trail, Trail with Bumps) with a 5 s rest preceding each scenario. Furthermore, the disturbances and the trail during the isolated phases are duplicated in the combined "Trails with Bumps" phase, so that a proper performance comparison can be drawn between the separate tasks and the one where the player must multiplex. During the entire game, there is 1 s of advanced warning in vision input, no delay in action output, and a 10-bit data rate for both vision and action.

As the disturbance, we use a random, binary signal, whose amplitude is the maximum possible torque the motor of the steering wheel can exert. In every 100 ms, the torque switches between max positive and negative (100 or -100 for the wheel). A similar random binary switching controls the trail derivative. More specifically, the trail travels at a constant speed but randomly switches its direction such that it always stays in the screen range comfortably visible to the player. We adjusted the velocity of the trail on the screen such that the required steering wheel turning rate is approximately 75°/s. Figure 6 illustrates the 5 s snapshots of the error dynamics for each scenario during the game.

NOTE: Since this is used to study the sensorimotor control performance with limits of delay and data rate, we only analyzed the data after the subjects were trained for the task, and their performance became stable. The learning effects have been excluded from the data. Moreover, the feedback control model has not considered learning.

Figure 1
Figure 1: Basic block diagram for an experimental platform with subject and gaming wheel with a motor.
Each box is a component that communicates or computes and has potentially both delay and quantization, including within the game in G. The advance warning T is also implemented on a computer screen with vision. Please click here to view a larger version of this figure.

Figure 2
Figure 2: The user-graphic interface for WheelCon.
(a) the main menu; (b) the Fitt's Law Task menu; (c) the Mountain Bike Task menu; (d) the video game interface for Fitt's Law Task; (e) the video game interface for Mountain Bike Task. Please click here to view a larger version of this figure.

Figure 3
Figure 3: Adding delay in vision input during the mountain-bike task.
(a-b) The system dynamics with time for the session with 1 s advanced warning (a) and with 0.4 s delay (b). The black line and blue line are the trail position and the player position, respectively. The red line is the error dynamics. (c) Error increases with the increasing delay. The negative delay means advanced warning. Please click here to view a larger version of this figure.

Figure 4
Figure 4: Effects of external delay in action on performance.
The L norm of error, MAE and RMSE increases with the increasing delay. Please click here to view a larger version of this figure.

Figure 5
Figure 5: Quantization in vision input (a) and action output (b).
The L error, MAE and RMSE are shown in the blue, black and red line, respectively. Please click here to view a larger version of this figure.

Figure 6
Figure 6: Effects of bump and trail disturbance on human sensorimotor control.
(a) the error dynamics induced by bump disturbance; (b) the error dynamics induced by trail disturbance; (c) the error dynamics induced by bump and trail disturbance; (d) the overlayed error in bumps (blue), trails (red), Trails with bumps (green). The purple-empty and orange-filled stem plots indicate the timing and direction of the bump disturbances and trail disturbance, respectively. Note that both the wheel forces and the trail rates are square waves, and the stems indicate where these square waves switch (i.e., derivatives of the forces and rates). Please click here to view a larger version of this figure.

Notation Variable Unit Constrains
w(t) Bump disturbance n Newton 0 ≤ w(t) ≤ 100
r(t) Trail disturbance 100 pixels 0 ≤ r(t) ≤ 100;
r(t)  w(t)
Tvis Vision advance warning/delay second -1 ≤ Tvis < 1
Tact Action delay second 0 ≤ Tvis < ∞
Rvis External data rate in vision input bit 1 ≤ Rvis ≤ 10
Ract External data rate in the wheel output bit 1 ≤ Ract ≤ 10
Qvis External quantizer on the vision input 1 Qvis = 2Rvis
Qact External quantizer on the wheel output 1 Qact = 2Rvis

Table 1: The variables in control loops which can be manipulated.

Notation Variable Unit
t Time second
x(t) Error dynamics, measured by the distance between the player’s position and the desired position 100 pixels
u(t) Control policy, measured by the deviated angle of the steering wheel degree

Table 2: The state variable and input signal in the dynamic system.

Platform Open Source ? Download Link Function
2D Virtual Environment (2DVE) No 1.Reach-to-grasp movements for arm rehabilitation after stroke;
ETH MIKE No 1.For the assessment of proprioceptive, motor and sensorimotor hand impairments
OpenSim Yes https://opensim.stanford.edu/ 1. To study neuromuscular coordination;
2. Let users develop models of musculoskeletal structures and create dynamic simulations of movements.
PneuGlove No 1.To train and evaluate finger individuation in children with hemiplegic cerebral palsy
VirtualEnaction Yes http://virtualenaction.gforge.inria.fr/ 1. For systemic neuroscience simulation;
2. To verify the functional models of the brain;
3.To experiment complex survival behaviors
VR-SPIRIT No 1.To improve predictive abilities in social scenarios.
2. Rehabilitation intensive training for paediatric patients
WheelCon Yes https://github.com/Doyle-Lab/WheelCon 1. To verify the Fitts' Law in human sensorimotor control;
2.To study the layered architecture in Sensorimotor control.

Table 3: The list of some existing sensorimotor platforms.

Discussion

In this paper, we have presented a free, open-source gaming platform, WheelCon, for studying the effects of delay, quantization, disturbance, and layered feedback loops in human sensorimotor control. We have shown the hardware, the software, and the GUI. The settings of a single sensorimotor control loop with delay and quantization have been implemented, which allows us to measure the effects of delay, quantization, and disturbance in sensorimotor control. The experimental results are well in line with the prediction from the feedback control theory.

The protocols provide a way to noninvasively manipulate external delays and limit the data rate in both vision inputs and action output, and to analyze the sensorimotor control performance. In the protocol, we ask the participants to play the game under several task scenarios that have been pre-defined in the platform. With these tasks, we verified the linear effect of delay (Figure 3 and Figure 4) and the nonlinear effect of quantization (Figure 5). These effects imply the optimization given the speed-accuracy tradeoff in human sensorimotor control. The protocol also allows us to study the layered feedback loops in the human sensorimotor system with a high-level advanced plan layer and a low-level delayed reflex layer (Figure 6).

In this protocol, it is crucial to train, but not over train, the participant in advance; otherwise, the learning effects or the tiredness of participants will impact the predictions of the model. The variability of the sensorimotor ability across participants is inevitable, and therefore some parameters in the protocol (i.e., the wheel sensitivity and the torque of bump) need to be tuned based on participants' age, strength, and motor skills. Matching these parameters across groups is necessary. Here, we suggest that the user chooses an appropriate sensitivity for both the old and young group in order to make a comparison.

One limitation of the method is that the model presented here did not consider the learning process. It is important to note that we only analyzed the data after the subjects were well trained, and their performance became stable to avoid the learning effects.

Nicolas Denoyelle et al. developed a platform (VirtualEnaction) for systemic neuroscience simulation38. VirtualEnaction can be used to validate functional models of the brain. Besides, some rehabilitation platform has been developed for performing sensorimotor tasks39,40,41,42,43. Scott L. Delp et al. developed an open-source software (OpenSim) to create and analyze dynamic simulations of Movement in rehabilitation science43. Marika Demers et al. proposed a 2D virtual environment for arm rehabilitation after stroke41. Zbytniewska Monika et al. design a robotic device for the assessment of hand sensorimotor impairments42. James V. McCall et al. proposed a platform for finger rehabilitation in children with hemiplegic cerebral palsy40. Niccolò Butti et al. develop a VR-based platform about social prediction improvement and rehabilitation intensive training for pediatric patients39. Table 3 listed the comparisons between these platforms with the platform presented here.

For the future research directions, the platform is compatible with noninvasive neural recordings technique (EEG) to measure the neural response during sensorimotor control. Investigating the mapping relationship between sensorimotor control and the EEG spatial-frequency signal, we might reveal the brain mechanism of sensorimotor control. It will be an important research question for understanding the human sensorimotor system. Moreover, most of the theory in this study is based on optimal control after learning. The stopping time for training is quite arbitrarily and empirically chosen in the study. It is therefore an important issue to evaluate whether participants hit a hard asymptote or a plateau44. Future studies could investigate the sensorimotor learning study using WheelCon to further test the asymptote/plateau and their possible explanations with sensorimotor learning theory.

Disclosures

The authors have nothing to disclose.

Acknowledgements

We thank Mr. Zhengyang Wang for reshaping the scripts, shooting and editing the video, and Mr. Ziyuan Ye for editing the video. This study got support from CIT Endowment & National Science Foundation (to JCD), Boswell fellowship (to QL) and High-level University Fund (No. G02386301, G02386401), Guangdong Natural Science Foundation Joint Fund (No. 2019A1515111038). 

Materials

Gaming Wheel Logitech
Windows 10 OS Microsoft

References

  1. Franklin, D. W., Wolpert, D. M. Computational Mechanisms of Sensorimotor Control. Neuron. 72 (3), 425-442 (2011).
  2. Bays, P. M., Wolpert, D. M. Computational principles of sensorimotor control that minimize uncertainty and variability. The Journal of Physiology. 578 (2), 387-396 (2007).
  3. Desmurget, M., Grafton, S. Forward modeling allows feedback control for fast reaching movements. Trends in Cognitive Sciences. 4 (11), 423-431 (2000).
  4. Sanger, T. D., Merzenich, M. M. Computational Model of the Role of Sensory Disorganization in Focal Task-Specific Dystonia. Journal of Neurophysiology. 84 (5), 2458-2464 (2000).
  5. Mohler, H., Okada, T. Benzodiazepine receptor: demonstration in the central nervous system. Science. 198 (4319), 849-851 (1977).
  6. Muller, L., Chavane, F., Reynolds, J., Sejnowski, T. J. Cortical travelling waves: mechanisms and computational principles. Nature Reviews Neuroscience. 19 (5), 255-268 (2018).
  7. Zhang, H., Watrous, A. J., Patel, A., Jacobs, J. Theta and Alpha Oscillations Are Traveling Waves in the Human Neocortex. Neuron. 98 (6), 1269-1281 (2018).
  8. Blinks, J. R., Rüdel, R., Taylor, S. R. Calcium transients in isolated amphibian skeletal muscle fibres: detection with aequorin. The Journal of Physiology. 277 (1), 291-323 (1978).
  9. Gallivan, J. P., Chapman, C. S., Wolpert, D. M., Flanagan, J. R. Decision-making in sensorimotor control. Nature Reviews Neuroscience. 19 (9), 519-534 (2018).
  10. Sanger, T. D. Basic and Translational Neuroscience of Childhood-Onset Dystonia: A Control-Theory Perspective. Annual Review of Neuroscience. 41 (1), 41-59 (2018).
  11. Todorov, E. Optimality principles in sensorimotor control. Nature Neuroscience. 7 (9), 907-915 (2004).
  12. Todorov, E., Jordan, M. I. Optimal feedback control as a theory of motor coordination. Nature Neuroscience. 5 (11), 1226-1235 (2002).
  13. Wolpert, D. M., Flanagan, J. R. Computations underlying sensorimotor learning. Current Opinion in Neurobiology. 37, 7-11 (2016).
  14. Kiper, P., et al. Computational models and motor learning paradigms: Could they provide insights for neuroplasticity after stroke? An overview. Journal of the Neurological Sciences. 369, 141-148 (2016).
  15. Cluff, T., Crevecoeur, F., Scott, S. H. Tradeoffs in optimal control capture patterns of human sensorimotor control and adaptation. bioRxiv. , 730713 (2019).
  16. Nakahira, Y., Liu, Q., Bernat, N., Sejnowski, T., Doyle, J. Theoretical foundations for layered architectures and speed-accuracy tradeoffs in sensorimotor control. 2019 American Control Conference (ACC). , 809-814 (2019).
  17. Körding, K. P., Wolpert, D. M. Bayesian integration in sensorimotor learning. Nature. 427 (6971), 244-247 (2004).
  18. Chambers, C., Sokhey, T., Gaebler-Spira, D., Kording, K. P. The development of Bayesian integration in sensorimotor estimation. Journal of Vision. 18 (12), 8 (2018).
  19. Karmali, F., Whitman, G. T., Lewis, R. F. Bayesian optimal adaptation explains age-related human sensorimotor changes. Journal of Neurophysiology. 119 (2), 509-520 (2017).
  20. Gori, J., Rioul, O. Information-Theoretic Analysis of the Speed-Accuracy Tradeoff with Feedback. 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). , 3452-3457 (2018).
  21. Trendafilov, D., Polani, D. Information-theoretic Sensorimotor Foundations of Fitts’ Law. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. , 1-6 (2019).
  22. Fitts, P. M., Peterson, J. R. Information capacity of discrete motor responses. Journal of Experimental Psychology. 67 (2), 103-112 (1964).
  23. Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology. 47 (6), 381-391 (1954).
  24. Nakahira, Y., Matni, N., Doyle, J. C. Hard limits on robust control over delayed and quantized communication channels with applications to sensorimotor control. 2015 54th IEEE Conference on Decision and Control (CDC). , 7522-7529 (2015).
  25. . Neural Correlates of Sensorimotor Control in Human Cortex: State Estimates and Reference Frames Available from: https://resolver.caltech.edu/CaltechTHESIS:05302019-163325527 (2019)
  26. Miall, R. C., Wolpert, D. M. Forward Models for Physiological Motor Control. Neural Networks. 9 (8), 1265-1279 (1996).
  27. Battaglia-Mayer, A., Caminiti, R., Lacquaniti, F., Zago, M. Multiple Levels of Representation of Reaching in the Parieto-frontal Network. Cerebral Cortex. 13 (10), 1009-1022 (2003).
  28. Scott, S. H. Optimal feedback control and the neural basis of volitional motor control. Nature Reviews Neuroscience. 5 (7), 532-545 (2004).
  29. Saunders, J. A., Knill, D. C. Humans use continuous visual feedback from the hand to control fast reaching movements. Experimental Brain Research. 152 (3), 341-352 (2003).
  30. Insperger, T., Milton, J., Stépán, G. Acceleration feedback improves balancing against reflex delay. Journal of The Royal Society Interface. 10 (79), 20120763 (2013).
  31. Birbaumer, N. Breaking the silence: Brain-computer interfaces (BCI) for communication and motor control. Psychophysiology. 43 (6), 517-532 (2006).
  32. Liu, Q., Farahibozorg, S., Porcaro, C., Wenderoth, N., Mantini, D. Detecting large-scale networks in the human brain using high-density electroencephalography. Human Brain Mapping. 38 (9), 4631-4643 (2017).
  33. Nicolelis, M. A. L. Brain-machine interfaces to restore motor function and probe neural circuits. Nature Reviews Neuroscience. 4 (5), 417-422 (2003).
  34. Nordin, A. D., Hairston, W. D., Ferris, D. P. Faster Gait Speeds Reduce Alpha and Beta EEG Spectral Power From Human Sensorimotor Cortex. IEEE Transactions on Biomedical Engineering. 67 (3), 842-853 (2020).
  35. Hallett, M. Transcranial magnetic stimulation and the human brain. Nature. 406 (6792), 147-150 (2000).
  36. Madhavan, S., Weber, K. A., Stinear, J. W. Non-invasive brain stimulation enhances fine motor control of the hemiparetic ankle: implications for rehabilitation. Experimental Brain Research. 209 (1), 9-17 (2011).
  37. Denoyelle, N., Pouget, F., Viéville, T., Alexandre, F. . VirtualEnaction: A Platform for Systemic Neuroscience Simulation. , (2014).
  38. Butti, N., et al. Virtual Reality Social Prediction Improvement and Rehabilitation Intensive Training (VR-SPIRIT) for paediatric patients with congenital cerebellar diseases: study protocol of a randomised controlled trial. Trials. 21 (1), 82 (2020).
  39. McCall, J. V., Ludovice, M. C., Blaylock, J. A., Kamper, D. G. A Platform for Rehabilitation of Finger Individuation in Children with Hemiplegic Cerebral Palsy. 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). , 343-348 (2019).
  40. Demers, M., Levin, M. F. Kinematic Validity of Reaching in a 2D Virtual Environment for Arm Rehabilitation After Stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 28 (3), 679-686 (2020).
  41. Zbytniewska, M., et al. Design and Characterization of a Robotic Device for the Assessment of Hand Proprioceptive, Motor, and Sensorimotor Impairments. 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). , 441-446 (2019).
  42. Delp, S. L., et al. OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement. IEEE Transactions on Biomedical Engineering. 54 (11), 1940-1950 (2007).
  43. Gray, W. D. Plateaus and Asymptotes: Spurious and Real Limits in Human Performance. Current Directions in Psychological Science. 26 (1), 59-67 (2017).

Play Video

Cite This Article
Liu, Q., Nakahira, Y., Liang, Z., Mohideen, A., Dai, A., Choi, S. H., Pan, A., Ho, D. M., Doyle, J. C. WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control. J. Vis. Exp. (162), e61092, doi:10.3791/61092 (2020).

View Video