It is unclear how top-down signals from the ventral visual stream affect movement. We developed a paradigm to test motor behavior towards a target on a 3D depth inversion illusion. Significant differences are reported in both deliberate, goal-directed movements and automatic actions under illusory and veridical viewing conditions.
Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are deliberately planned, while others spontaneously occur below conscious awareness. The deliberate component of this dichotomy has been studied extensively in several contexts, while the spontaneous component remains largely under-explored. Moreover, how perceptual processes modulate these movement classes is still unclear. In particular, a currently debated issue is whether the visuomotor system is governed by the spatial percept produced by a visual illusion or whether it is not affected by the illusion and is governed instead by the veridical percept. Bistable percepts such as 3D depth inversion illusions (DIIs) provide an excellent context to study such interactions and balance, particularly when used in combination with reach-to-grasp movements. In this study, a methodology is developed that uses a DII to clarify the role of top-down processes on motor action, particularly exploring how reaches toward a target on a DII are affected in both deliberate and spontaneous movement domains.
Vision-for-Perception vs. Vision-for-Action
In order to successfully navigate the environment, information from the visual system is utilized to help coordinate human movement. How visual information is selected and prioritized to influence motor actions remains unclear. Two major anatomical projections arise from the primary visual cortex to form the ventral (“what”, or “vision for perception”) pathway, extending to the temporal area, and the dorsal (“where”, or “vision for action”) pathway, to the parietal lobe1-2. The ventral stream is implicated in utilizing visual information for perceptual processes such as object recognition and identification, whereas the dorsal stream is thought to exclusively process signals for action guidance and spatial awareness. The question asked is whether or not top-down processes from the ventral stream shape the way in which movements are executed.
The famous case study of Patient DF, evaluated by Goodale and Milner in 1992, provided strong evidence and support for the visual two-streams hypothesis, which claims that ventral and dorsal stream processes are separable for perception and action3. In theory, bottom-up signals of motion parallax and binocular disparity can override top-down perceptual information such as prior knowledge and familiarity in order to accurately guide our actions, suggesting that motor planning is impervious to ventral stream control. DF, who suffered from visual form agnosia caused by bilateral ventral occipital lesions, retained accurate grasping ability towards objects that she had difficulty recognizing, supporting the premise of the visual two-streams hypothesis3-4. Because of case studies like DF, it was assumed that the functional ventral-dorsal stream dichotomy also existed in healthy, nonpathological individuals. However, whether or not these findings provide evidence for an absolute division of labor for perception and action in neurotypical populations has been hotly debated over the past twenty years5-10.
The Use of Illusions to Segregate Perception and Action
To test the visual two-streams hypothesis in neurotypical subjects, researchers employ visual illusions to investigate how skewed perceptual judgments of the environment affect our motor actions. The Ebbinghaus/Titchener Illusion, for example, uses a disk target surrounded by smaller disks that appears to be larger than another disk of the same size surrounded by larger circles; this is due to a size-contrast effect11. When participants reach to grasp the disk target, if the two-streams hypothesis holds true, then the grip aperture of the hand grabbing at the disk target would be unaffected by the illusion, causing the participant to act on the true geometry of the disk target rather than rely on incorrect perceptual size estimates. Aglioti et al. in fact report this behavior, reasoning that separate visual processes govern skilled actions and conscious perception11. Conversely, other groups have contested these results, finding no dissociation between perception and action processes when carefully controlling the matching of perceptual and grasping tasks, proposing an integration of visual stream information rather than a separation12. Despite several follow-up studies conducted to validate or refute the visual two-streams hypothesis using the Ebbinghaus Illusion, there are competing pieces of evidence to support both sides of the argument13.
To further explore the influence of visual perception on action processes, 3D depth inversion illusions (DII) have also been utilized. DIIs produce illusory motion and perceived depth reversal of scenes in which physically concave angles are perceived as convex and vice versa14. The Hollow Face Illusion is an example of a DII that generates the perception of a normal, convex face although the stimulus is physically concave, implicating the role of top-down influences such as prior knowledge and convexity bias to elicit the illusory percept15-16. Despite efforts to characterize motor behavior in reaching towards targets on the Hollow Face Illusion, evidence remains equivocal: one study reports an effect on motor output17 while another does not18. These studies rely on comparing perceptual depth estimates to endpoint distance calculations of the hand relative to targets located on the Hollow Face Illusion. Conflicting results on actions performed on this type of stimuli may be a result of the variations in methods used by researchers. Because the way in which ventral and dorsal stream information is utilized is still up to debate, this controversy sparks the need for a more robust stimulus with additional advanced measures of motor behavior.
This is precisely why a technique was developed using reverse-perspective stimuli, commonly referred to as “reverspectives”, which form another class of DIIs14. Linear perspective cues that are painted on piecewise 3D planar surfaces produce competition between the physical geometry of the stimulus and the actual painted scene. Data-driven sensory signals, such as binocular disparity and motion parallax favor the veridical percept of the physical geometry, whereas experience-based familiarity with perspective favors the depth-inversion percept (Figure 1). The advantage of the reverspective is that it allows for the placement of a target on a stimulus surface whose perceived spatial orientation under the illusion differs by nearly 90 degrees from its physical orientation (Figures 1e and 1f). This huge difference greatly facilitates testing whether reach-for-grasping movements are or are not influenced by the illusion. This notion is key to exploring whether or not motor actions performed on the reverspective are affected by top-down influences from the ventral stream.
Movement Classes in Perception-Action Models
If different motor strategies are employed under illusory and veridical percepts when grabbing towards a target on a reverspective stimulus, then it can be easily tracked by studying the curvature of the hand's approach. Moreover, an analysis of the entire unfolding movement from initiation of the goal-directed movement to the spontaneous, automatic retraction of the hand back to its resting state may in fact bypass any shortcomings found in past methods of testing for perceptual influence on motor output. Recent studies highlight the significance of studying the balance between these two movement classes as well as the use of the spontaneous segments by the nervous systems for predictive and anticipatory control19-21,23-24. The newly statistically defined class of spontaneous-automatic movements provides new metrics and features that turn out to be as crucial as the goal-directed ones have been thus far to track sensory-motor changes and to quantify subtle aspects of natural behaviors.
To our knowledge, existing research on the visual two-streams hypothesis only focuses on goal-directed acts, thereby ignoring any effects on automatic transitional movements that are significant components to completing the visuomotor action loop. Emphasis therefore must be placed on the importance of automatic motions in order to fully capture both modes of motor behavior in the present paradigm to clarify issues concerning visual perception-action models. Here methods are developed to investigate the role of top-down signaling in the visual ventral stream on modulating motor behavior in the deliberate, goal-directed action domain in conjunction with spontaneous, transitional movements using a robust DII reverse-perspective stimulus.
Rationale
It is hypothesized that, if top-down visual processes influence the sensory-motor system, full movement trajectories toward the embedded target in the 3D reverse-perspective scene under the illusory percept will differ from the target approach elicited by the veridical percept (Figures 1e and 1f). Moreover, since the illusory percept of the reverspective stimulus is very similar to that obtained by a proper (“forced”) perspective stimulus, reaches performed toward an embedded target on a reverspective should therefore be similar in characteristics to reaches conducted under the influence of the illusion on the reverspective stimulus (Figures 1c and 1f).
If top-down visual influences do not impact the movement trajectory, then it is hypothesized that reaches made under the illusory percept would exhibit the same characteristics as reaches made under the veridical percept on the reverspective stimulus (Figure 1e). In other words, both illusory and veridical percept reaches would be similar in nature, such that both forward trajectory paths would act on the true geometry of the stimulus. How effects observed in the forward reach translate in the automatic retraction of the hand is unknown. By employing a full motor analysis, we aim to advance our understanding of action and perception loops to clarify the existing issues at hand.
1. Building the Stimulus Apparatus
2. Participants
3. Experimental Procedure
4. Data Analysis
1. Hand Path Trajectories
Results are shown for Representative Subject VT. The Wilk's Lambda Test Statistic allows for the reduction of our three-dimensional space data into a scalar value by the use of determinants. The Wilk's lambda statistic uses the likelihood ratio test , in which the 'within' sum of squares and products form matrix E, and the 'total' sum of squares and products form matrix (E+H). The rule states that, when , the null hypothesis is rejected. In , is the level of confidence, is the number of variables or dimensions, and and are the degrees of freedom for the hypothesis and error, respectively, in which is the number of conditions and is the number of trials. In our case, and . Therefore, we obtain from the look-up table found in Rencher's Methods for Multivariate Analysis22.
Hand path trajectory analysis using the Wilk's Lambda Test reveal a statistically significant difference between REV-ILLU and REV-VER conditions in the forward, goal-directed movement (Figure 3a), as throughout the entire path's progression (Figure 3d). This behavior is also preserved in the noninstructed retraction as seen in the graph (Figures 4a and 4d). As expected, the comparison between the REV-VER and PRO conditions differs significantly in both the forward and retractory movements (Figures 3b, 3e, 4b, and 4e). Since the unfolding of movement is critical in determining differences in approach, Wilk's lambda values are plotted based on the percentage of hand path trajectory complete (Figures 3d-3f and 4d-4f). Averaging these values over the entire path would not fully recapitulate the kinematics of the entire action loop. Wilk's lambda values for forward hand path trajectories in the REV-VER vs. REV-ILLU comparison (Figure 3d) are similar to those found in the REV-VER vs. PRO comparison (Figure 3e). The same holds true for the retraction of the hand (Figures 4d and 4e). The REV-ILLU and PRO conditions do not differ significantly in either movement class, as for all lambda values based on the percentage of path complete in both the forward and retraction cases (Figures 3c, 3f, 4c, and 4f).
2. Hand Orientation
When examining the orientation of the hand as it approaches the target in each condition, hand-approach vectors in REV-VER cases differ from those in the REV-ILLU and PRO cases (Figure 5c). REV-ILLU and PRO conditions produce similar hand postures when orienting towards the perceived target for REV-ILLU and the physical target for PROPER conditions. The angle formed between the mean unit approach vector for REV-ILLU trials and the unit vector normal to the target surface produces a 97.5197° ± 3.2228 difference (Figure 5d). Recall that the reverse-perspective stimulus generates nearly 90-degree maximal differences between illusory and veridical states. This therefore suggests that Representative Subject VT oriented her hand towards the perceived target and not the physical location of the target under the illusory percept.
Figure 1. The Proper- and Reverse- Perspective Stimuli. (a-c) The proper- or "forced" perspective. (a) Front view of the painted stimulus. (b) Orthographic views. (c) Top view: the veridical percept of a concave scene with an arrow illustrating a typical reach trajectory for the target. (a, d-f) The reverse-perspective gives rise to two percepts, shown in parts (e) and (f). (a) Front view of the painted stimulus. (d) Orthographic views. (e) Top view: the veridical percept of a convex scene with an arrow illustrating a typical reach trajectory. (f) Top view: the illusory percept of a concave scene – shown by dotted lines – with an arrow illustrating a typical reach trajectory. The dotted-line figure shows the perceived illusory 3D shape only. The position of the object is not accurate; in fact, the illusory object was deliberately offset toward the observer in order to clarify the reach trajectory. The curvature of all trajectories is exaggerated to illustrate the differences that might result, depending on the percept. The percept of (f) provides an excellent test for examining whether the reaching trajectory is governed by the illusion (trajectory of (f)) or by the physical surface (trajectory of (e)). Note that the proper- and reverse-perspectives share the same front view (a). Please click here to view a larger version of this figure.
Figure 2. Training Stimuli. (a-b) The rectangular panel has the same spatial orientation as the right wall of the middle building in the proper-perspective 3D stimulus. (c-d) The rectangular panel has the same orientation as the right wall of the middle building in the reverse-perspective 3D stimulus. (a, c) Schematic drawings of top views to illustrate the placement of the panels, with arrows indicating typical reach trajectories. The curvature of the trajectories is exaggerated to illustrate the difference. (b, d) Photographs of the stimuli as they appeared to participants. Please click here to view a larger version of this figure.
Figure 3. Forward Hand Path Trajectory Analysis. (a-c) Mean trajectories plotted in white with confidence intervals (colored tubes) for each point in the trajectory for reverse-perspective veridical (REV-VER in green), reverse-perspective illusory (REV-ILLU in blue), and proper-perspective (PROPER in red) conditions for the goal-directed, intended forward movement. (d-f) Lambda values for pairwise comparisons of conditions based on percentage of path complete. Using the Wilk's Lambda Test, when , the null hypothesis is rejected. is given by the dotted line. In (d) REV-VER vs. REV-ILLU and (e) REV-VER vs. PROPER comparisons, , indicating a significant difference between hand path trajectories. For the (f) REV-ILLU vs. PROPER .5comparison, , therefore hand path trajectories between conditions do not differ significantly. Please click here to view a larger version of this figure.
Figure 4. Retraction Hand Path Trajectory Analysis. (a-c) Mean trajectories plotted in white with confidence intervals (colored tubes) for each point in the trajectory for reverse-perspective veridical (REV-VER in green), reverse-perspective illusory (REV-ILLU in blue), and proper-perspective (PROPER in red) conditions for the spontaneous, automatic retraction of the hand. (d-f) Lambda values for pairwise comparisons of conditions based on percentage of path complete. Using the Wilk's Lambda Test, when , the null hypothesis is rejected. is given by the dotted line. In (d) REV-VER vs. REV-ILLU and (e) REV-VER vs. PROPER comparisons, , indicating a significant difference between hand path trajectories. For the (f) REV-ILLU vs. PROPER comparison, , therefore hand path trajectories between conditions do not differ significantly. Please click here to view a larger version of this figure.
Figure 5. Hand Orientation. (a) The unit approach vector of the hand is defined by sensors located on the thumb, index, and wrist positions. (b) Unit (dashed) vectors normal to the target surface for the proper-perspective (top) and reverse-perspective (middle and bottom) stimuli. Under the illusory percept (bottom panel, dotted lines) this vector is perceived as nearly perpendicular to the physical unit vector (middle). (c) Hand approach vectors plotted for REV-VER (green), REV-ILLU (blue), and PROPER (red) trials, (d) Mean angle values formed between the unit approach vector and the physical unit vector normal to the target surface corresponding to the stimulus used. Please click here to view a larger version of this figure.
Our methods provide a platform to test the validity of perception-action models by analyzing the entire unfolding of movement in relation to the experimental task. The paradigm can be modified to test other types of visual stimuli to broaden this area of research. For example, other 3D DIIs can be tested on the apparatus to see how interactions between top-down and bottom-up processes translate to various stimuli. The methods can also be tailored to test clinical populations that may have perturbations in perception and action processes. Moreover, the motion capture system utilized in our study can be replaced with other types of recording equipment to best suit the experimental task. The possible generalization of these methods for other applications therefore holds significant value in the advancement of human behavioral research.
However, as with any technique, the current paradigm has its limitations. Because of the removal of haptic feedback and online visual control by turning off the lights and retracting the stimulus, the present study does not allow for the simultaneous recording of eye movements in conjunction with the execution of movement. Eye movements can help identify whether or not participants use an allocentric or egocentric frame of reference to employ a top-down or bottom-up strategy25. Because the current design does not have the capability of implementing this additional measure, it is constrained to only capturing the body's kinematic function. Alternative strategies to remove haptic feedback and online visual control may be sought out to capture eye movement measures.
Besides this setback, the experimental design has several advantages over existing methods. Since past studies focused on the deliberate, goal-direction actions and end-point data, researchers overlooked any effects in the non-instructed, automatic retraction, and in the actual unfolding of the movement from initiation to rest. The protocol presented here takes into account both deliberate and automatic forms of movement to help build a better understanding of sensory-motor behavior under different perceptual states. Unlike other strategies, this paradigm focuses on both spatial and temporal effects to gain a full understanding of the visuomotor loop. Moreover, the strength of the reverspective stimulus used in this experiment trumps other DIIs used in the past (e.g. hollow-face illusion) as its configuration generates nearly 90° differences in perceived surface orientation under veridical and illusory states while remaining close enough to the participant for he/she to interact with it. This maximal difference aids in the disambiguation of the role of top-down processes on sensory-motor behavior.
Since the study of top-down influences on sensory-motor processes is important not just in the normative system, but also in clinical populations, this paradigm may prove to be a useful tool to study them. Future applications of this protocol may include tailoring the study for pathologies such as Schizophrenia (SZ). It is known that a certain subpopulation of patients with SZ exhibit a decrease in top-down functioning and have known issues in perceptual organization26-28. Thus, understanding how this translates to the motor domain can advance our knowledge to develop better diagnostic tools and therapies for SZ.
This protocol was carefully designed to investigate the role of top-down processes on sensory-motor behavior, specifically when a participant is asked to reach at a target on a stimulus that produces multiple percepts. The critical steps within this protocol are in the selection of the stimuli and in the high resolution of motion capture from the initiation of movement back to resting state. Also, the powerful statistical analyses help elucidate whether or not the illusory percept influences motor strategies. Because this experimental design allows for the high resolution recording of natural intended and spontaneous motor behaviors, the analytical platform developed may help elucidate the existing issues in perception-action models that have long been debated. The preliminary results for Representative Subject VT illustrate this potential.
The authors have nothing to disclose.
The authors would like to acknowledge the members of the Laboratory of Vision Research and the Sensory-Motor Integration Laboratory for helping run participants in this study, Polina Yanovich, Joshua Dobias, and Robert W. Isenhower for help in the initial design phase, and Tom Grace for his help in building the stimulus. This work was supported by the following sources: the NSF Graduate Research Fellowship Program: Award #DGE-0937373, the NSF CyberEnabled Discovery and Innovation Type I (Idea): Grant #094158, and the Rutgers-UMDNJ NIH Biotechnology Training Program: Grant # 5T32GM008339-22.
Laboratory bench | |||
Slidable Track with Retractable Spring | built in-house | ||
Retractable Spring | |||
Adjustable Lamps | |||
Switch Box | |||
Circuit Board | |||
Arduino | Smart Projects, Italy | ||
MATLAB | The MathWorks Inc., Natick, MA, USA | ||
Randot-dot Stereo Test | |||
Reverse-Perspective Stimulus | built in-house | ||
Proper-Perspective Stimulus | built in-house | ||
Training Stimuli | built in-house | ||
Polhemus Motion Capture System | Liberty, Colchester, VT, USA | ||
The Motion Monitor Motion-Tracking Software | Innovative Sports Training, Inc., Chicago, IL | ||
Sport Sweatbands | |||
De-Focusing Lens |