WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control

Modelling Feedback Control

We show a simplified feedback control model shown in Figure 1. The system dynamics is given by:

where x(t) is the error at time t, r(t) is the trail disturbance w(t), is the bump disturbance, and u(t) is the control action.

Modeling Action Delay in Trail Disturbance

When there is a delay T in action, and a trail disturbance r(t), we model the control action by

The game starts with zero initial condition: x(0) = 0. The controller κ generates the control command u(t) using the full information on the histories of state, disturbance, and control input. Here, the net delay T is composed of the internal delays in the human sensorimotor feedback and the delays externally added. The control command is executed with delay T ≥ 0. Sensorimotor control in the risk-aware setting motivates the use of L1 optimal control, and as such, the goal is to verify the following robust control problem

This problem admits a simple and intuitive solution. The optimal cost is given by

This optimal cost is achieved by the worst-case control policy u(t + T) = −r(t), which yields

Modeling Action Quantization in Trail Disturbance

When the data rate, R, in the control loop is limited, the control action is generated by the following feedback loop with communication constraints,

where is a controller, and is a quantizer with data rate R ≥ 1, i.e. S is a finite set of cardinality 2^R. The disturbance r(t) is infinity-norm bound and without loss of generality, . The worst-case state deviation is lower-bounded by

and the minimum control effort is given by

Measures of Error

To quantify the performance, we measured the infinity norm error , mean absolute error (MAE) and root mean square error (RMSE). The infinity norm is defined as the maximum of the absolute errors, where

Mean absolute error is calculated as follows

Root mean squared error is calculated as follows

Game 1: Visual Advanced Warning or Delay

Game 1 evaluates how the length of the look-ahead window (advanced warning/delay) affects sensorimotor control performance without being exposed to additional disturbances.

Game 1 lasts for 360 seconds and consists of one continuous "Trail", which reduces the amount of look-ahead every 30 seconds. The game begins with 1 s of advanced warning, and then decreases to 0.75 s, and then to 0.5 s. From there, the game decreases the look ahead by 0.1 s until a minimum of -0.4 s is reached. Positive delay, or negative advanced warning, means only the trail behind the player is visible.

An evolution of error dynamics of the player as the game progresses with 1 s advanced warning and 0.4 s delay were depicted in Figure 3a-3b separately. Both the plot display only the middle 20 seconds of each of the 30-second intervals to neglect the effects of the player adjusting to the new look-ahead window. The progression of the error in the blocks looks stable in the 1 s advanced warning setting while in the 0.4s delay setting, the error flips upside down during the progress. To quantify that effect in more detail, we evaluate L1-/L2-/L_∞– norm for the error dynamics for every 20 s group corresponding to a delay level. Summarizing these calculations in a plot gives Figure 3c, which demonstrates how the players' error-norm does not change until the advanced warning reaches 0.5 s and then increases in an approximately linear fashion.

Game 2: Delay in Action Output

Unlike Game 1's external visual delay, Game 2 adds specific internal delay to the action output; in other words, the current control policy u(t) works at u(t + T_act) where T_act is the external delay in action. Game 2 lasts for 180 s. Adjusting T_act every 30 s, T_act starts at 0 s, and increments by 0.1 s until it reaches 0.4 s.

The effects of delay in action are shown in Figure 4. Similar to the vision delay, the error increases linearly with the delay, which is well in line with the prediction from theory in Eq(3).

Game 3 and Game 4: Quantization in Vision input and Action Output

Game 3 and Game 4 study the effects of quantization in vision input and action output, respectively. Each game is 210 s long, and the quantization changes every 30 s, with the data rate increasing from 1 to 7 bits. For example, when the R_vis is 1 in Game 3, the desired position (gray line in the gaming GUI) is presented either in the center-left or the center-right of the screen. When R_vis = n, the desired position can be presented in 2ⁿ possible locations on the screen. For Game 4, when R_act = 1, the player is either going left or right with one speed. When R_act is n, the player can steer the wheel to go left or right with 2^n-1 speeds.

The effects of quantization (limited data rate) in the vision and action are shown in Figure 5. Consistent with the theory's prediction in Eq(6), sensorimotor control performance improves with higher data rates and reaches the optimal control performance when R is around 5.

Game 5: Bump and trail disturbance

Game 5 is designed to test the effects of bump and trail disturbances on human sensorimotor control. Game 5 consists of three scenarios:

a) "Bumps", tracking a constant trail subject despite torque disturbances on the wheel that mimic hitting bumps when riding a mountain bike;
b) "Trail", tracking a moving trail with random turns but without bumps;
c) "Trail with Bumps", tracking a moving trail with random turns and bumps.

Each scenario lasts for 60 s in the order (Bumps, Trail, Trail with Bumps) with a 5 s rest preceding each scenario. Furthermore, the disturbances and the trail during the isolated phases are duplicated in the combined "Trails with Bumps" phase, so that a proper performance comparison can be drawn between the separate tasks and the one where the player must multiplex. During the entire game, there is 1 s of advanced warning in vision input, no delay in action output, and a 10-bit data rate for both vision and action.

As the disturbance, we use a random, binary signal, whose amplitude is the maximum possible torque the motor of the steering wheel can exert. In every 100 ms, the torque switches between max positive and negative (100 or -100 for the wheel). A similar random binary switching controls the trail derivative. More specifically, the trail travels at a constant speed but randomly switches its direction such that it always stays in the screen range comfortably visible to the player. We adjusted the velocity of the trail on the screen such that the required steering wheel turning rate is approximately 75°/s. Figure 6 illustrates the 5 s snapshots of the error dynamics for each scenario during the game.

NOTE: Since this is used to study the sensorimotor control performance with limits of delay and data rate, we only analyzed the data after the subjects were trained for the task, and their performance became stable. The learning effects have been excluded from the data. Moreover, the feedback control model has not considered learning.

Figure 1: Basic block diagram for an experimental platform with subject and gaming wheel with a motor.
Each box is a component that communicates or computes and has potentially both delay and quantization, including within the game in G. The advance warning T is also implemented on a computer screen with vision. Please click here to view a larger version of this figure.

Figure 2: The user-graphic interface for WheelCon.
(a) the main menu; (b) the Fitt's Law Task menu; (c) the Mountain Bike Task menu; (d) the video game interface for Fitt's Law Task; (e) the video game interface for Mountain Bike Task. Please click here to view a larger version of this figure.

Figure 3: Adding delay in vision input during the mountain-bike task.
(a-b) The system dynamics with time for the session with 1 s advanced warning (a) and with 0.4 s delay (b). The black line and blue line are the trail position and the player position, respectively. The red line is the error dynamics. (c) Error increases with the increasing delay. The negative delay means advanced warning. Please click here to view a larger version of this figure.

Figure 4: Effects of external delay in action on performance.
The L_∞ norm of error, MAE and RMSE increases with the increasing delay. Please click here to view a larger version of this figure.

Figure 5: Quantization in vision input (a) and action output (b).
The L_∞ error, MAE and RMSE are shown in the blue, black and red line, respectively. Please click here to view a larger version of this figure.

Figure 6: Effects of bump and trail disturbance on human sensorimotor control.
(a) the error dynamics induced by bump disturbance; (b) the error dynamics induced by trail disturbance; (c) the error dynamics induced by bump and trail disturbance; (d) the overlayed error in bumps (blue), trails (red), Trails with bumps (green). The purple-empty and orange-filled stem plots indicate the timing and direction of the bump disturbances and trail disturbance, respectively. Note that both the wheel forces and the trail rates are square waves, and the stems indicate where these square waves switch (i.e., derivatives of the forces and rates). Please click here to view a larger version of this figure.

Table 1: The variables in control loops which can be manipulated.

Table 2: The state variable and input signal in the dynamic system.

Table 3: The list of some existing sensorimotor platforms.