This experimental protocol combined eye tracking and the assessment of presaccadic visual sensitivity in a dual task paradigm, consisting of a free choice saccade task and a visual discrimination task, to investigate the deployment of visual spatial attention before both, accurate and averaging saccades.
This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.
Accumulated evidence argues for strong coupling between oculomotor and attentional control. The influential premotor theory of attention1,2 provides a particularly strict account regarding this coupling, suggesting that covert shifts of visual attention correspond to saccade programming without subsequent execution. Indeed, shared neuronal correlates of attentional and oculomotor control have been identified via functional magnetic resonance imaging (fMRI)3 and sub-threshold micro-stimulation of the Frontal Eye Fields (FEF) and the Superior Colliculi (SC) produces attentional benefits measured both behaviorally and electrophysiologically at the stimulated movement field position, even if no eye movements are induced4,5,6,7. Psychophysical experiments furthermore revealed that visual attention is consistently shifted towards the target of a saccade during oculomotor preparation8,9. However, dissociations at the neuronal level10,11,12 and observations that saccade preparation does not necessarily entail a corresponding shift of attention13,14,15,16 cast some doubt on an obligatory coupling between saccade programming and visual spatial attention.
Here, we revisited the nature of the coupling between attention and oculomotor programming at the behavioral level using a dual task which entailed a free choice saccade task and a visual discrimination task. Crucially, two saccade targets were presented at an angular distance of 30° on half of the trials, producing a distinct global effect17,18,19,20 associated with a substantial number of saccades landing in between the two targets (averaging saccades). Since we randomly presented a discrimination target shortly before saccade onset at one out of 24 equidistant locations (including the two saccade target locations, the location in between them and 21 control locations), we were able to assess and compare the presaccadic deployment of visual attention when saccade goal and saccade endpoint were either spatially associated (accurate saccades) or dissociated (averaging saccades).
To test whether visual attention is an obligatory consequence of oculomotor programming, and, therefore, always shifted towards the saccade endpoint, we analyzed visual sensitivity for all 24 locations as a function of the saccade landing direction. While accurate saccades were associated with a consistent presaccadic enhancement of visual sensitivity at their endpoint, we found no such enhancement at the endpoint of averaging saccades. This dissociation rules out an obligatorily coupling of visual attention to the executed oculomotor program at the behavioral level and suggests that attentional and oculomotor control are dissociable at some cortical or subcortical processing stage. Importantly, we observed an equal enhancement of visual sensitivity at the two saccade targets before averaging saccades, suggesting that the global effect arises from unresolved saccade target selection before saccade onset.
This protocol was designed according to the ethical requirement specified by the Ludwig-Maximilians-Universität München and with the approval of the ethics board of the department.
1. Participants
2. Experimental Setup
3. Eye Tracking
4. Instructions
5. Experimental Design, Stimuli, and Trial Chronology
6. Data Preprocessing and Analysis
Here, we only present some central, representative results. The entirety of results can be found in our recent publication23. Note that data were mainly analyzed irrespective of the duration of the saccade targets (i.e. transient and continuous cueing conditions were combined for the final analyses). For statistical comparisons, we drew 10,000 bootstrap samples (with replacement) from the distribution of single subject means and derived two-tailed p values from the distribution of differences between the bootstrapped samples.
The detection of saccade onsets and offsets was based on the velocity distribution of the gaze24. We used a moving average over 20 subsequent eye-position samples to determine saccade onsets/offsets whenever the velocity of the eye exceeded/fell below the median of the moving average by 3 SDs for at least 20 ms. Corrective saccades were defined as eye movements executed after the offline selected main saccade sequence and were only included in the respective corrective saccade analysis if they landed between 7° and 13° from the fixation target and were initiated within the first 500 ms following the main saccade sequence as well as before the participant's manual response.
Before proceeding to final data analysis, data were rotated (see 6.2). Consequently, after data rotation, the most counterclockwise saccade target ST1 was always represented at +45°/+15° (in the 90° and 30° conditions, respectively), the location BTW in between the saccade targets at 0° (in both the 90° and 30° conditions), and the most clockwise saccade target ST2 at -45°/-15° (in the 90° and 30° conditions, respectively) relative to the angle 0. Locations other than ST1, ST2, and BTW were considered as control locations (CTRL) in both, the 90° and 30° conditions.
Our protocol allowed us to evaluate saccades in response to oculomotor competition among two saccade targets presented at different angular distances based on the recorded eye data. As expected, the saccade endpoint distributions associated with the 90° (Figure 3A and 3C) and 30° (Figure 3B and 3D) conditions differed substantially. We observed mostly accurate saccades towards one of the saccade targets in the 90 ° condition, where 41.0% ± 1.0% of saccades ended within the sector including the most counterclockwise saccade target ST1 and 41.8% ± 1.9% within the sector including the most clockwise saccade target ST2 (Figure 3C). In the 30° condition, in contrast, participants executed a substantial number of averaging saccades. Here, 33.6% ± 2.4% of the saccades ended within the sector in between the 2 saccade targets BTW, 29.95% ± 1.6% ended within the sector including ST1, and 32.0% ± 1.8% within the sector including ST2 (Figure 3D).
Furthermore, the assessment of visual sensitivity at all 24 locations distributed across the visual field allowed us to analyze the spatial deployment of attention during oculomotor programming in detail. Overall, taking into account saccades of all directions, we observed a selective facilitation of visual sensitivity at the two saccade targets relative to the control locations CTRL (corresponding to the average across all positions except for ST1, ST2, and BTW) in both the 90° (ST1: d' = 2.2 ± 0.3 versus CTRL: d' = 0.3 ± 0.1, p < 0.0001; ST2: d' = 2.2 ± 0.4 versus CTRL, p < 0.0001; ST1 versus ST2, p = 0.8964; Figure 4A) and 30 ° (ST1: d' = 2.2 ± 0.3 versus CTRL: d' = 0.3 ± 0.1, p < 0.0001; ST2: d' = 2.1 ± 0.3 versus CTRL, p < 0.0001; ST1 versus ST2, p = 0.6026; Figure 4B) conditions. While visual sensitivity at the intermediate location was significantly lower than at the saccade target locations (BTW: d' = 0.6 ± 0.2 versus ST1, p < 0.0001; BTW versus ST2, p < 0.0001; Figure 4B), it was, however, slightly increased relative to the control locations in the 30° condition (BTW versus CTRL, p = 0.0010).
In order to disentangle whether visual attention is obligatorily deployed at the endpoint of saccades, we analyzed visual sensitivity at all locations as a function of the saccade landing direction (see step 6.3 in the protocol). Crucially, the specific saccade landing distribution observed in the 30° condition of this protocol made it possible to analyze the deployment of visual attention before saccades associated with spatially distinct endpoints in response to identical visual input. More specifically, by analyzing visual sensitivity before averaging saccades, we could determine whether or not attention shifts towards the endpoint of saccades even when it does not spatially coincide with a saccade goal. We observed that visual sensitivity was significantly enhanced at the endpoint of accurate saccades in both the 90° (ST1+2 saccaded: d' = 3.0 ± 0.4 versus ST1+2 non-saccaded: d' = 1.7 ± 0.4, p < 0.0001; Figure 4E) and the 30° (ST1+2 saccaded: d' = 2.7 ± 0.4 versus ST1+2 non-saccaded: d' = 2.0 ± 0.3, p = 0.0080; Figure 4F) condition. In contrast, before averaging saccades, visual sensitivity was not enhanced at the saccade endpoint but slightly reduced (BTW saccaded: d' = 0.4 ± 0.2 versus BTW non-saccaded: d' = 0.7 ± 0.2, p < 0.0001; Figure 4F). Thus, visual attention was not obligatorily shifted towards the endpoint of the upcoming saccade. Interestingly, averaging saccades were associated with an equal enhancement of visual sensitivity at the two surrounding saccade targets (ST1: d' = 2.2 ± 0.4 versus ST2: d' = 2.2 ± 0.4, p = 0.8402; Figure 4D), suggesting that attentional selection among the saccade targets was not readily resolved before the onset of averaging saccades.
To further evaluate a potential correlate of attentional selection before averaging saccades, data were analyzed as a function of the landing direction of corrective saccades, which can be frequently observed upon the execution of averaging saccades. We did not observe a significant benefit at the endpoint of corrective saccades following an averaging saccade (corrective saccade directed towards ST1+2: d' = 2.8 ± 0.5 versus corrective saccade not directed towards ST1+2: d' = 2.5 ± 0.8, p = 0.68300; Figure 5C), which supports the interpretation that attentional selection was not resolved before averaging saccades.
Figure 1: Instructions as presented to the participants. Visualization of the experimental instructions as presented to the participants at the beginning of each block. Please click here to view a larger version of this figure.
Figure 2: Experimental procedure and normalized saccade landing frequency maps. (A) Stimulus timing and display. Participants prepared a saccade from the fixation target (FT) to one of the two potential saccade targets (ST1 and ST2), presented simultaneously at two randomly chosen stimulus streams with an inter-target angular distance of either 90° (top panels) or 30° (bottom panels). The saccade targets were either shown continuously (cST1+2) or transiently (tST1+2). Stimulus streams could either be distractor streams (DS), composed of alternating vertical Gabors and masks (40 Hz) or discrimination target streams (DTS) which included the presentation of a brief discrimination target (DT, 25 ms), a clockwise or counterclockwise tilted Gabor, shown between 75 and 175 ms after the saccade targets onset. Participants saccaded towards one of the saccade targets and had to report the orientation of the discrimination target, appearing randomly at one of the 24 stimulus stream locations. Note that stimuli are sketched in order to increase their visibility. Actual stimuli match those shown in the stimulus streams depiction. (B) Normalized saccade landing frequency maps averaged across participants (n = 10) for the 90° (top) and 30° (bottom) conditions (collapsed across the transient and continuous ST presentation). This figure has been reprinted from Wollenberg et al. (2018)23. Please click here to view a larger version of this figure.
Figure 3: Saccade metrics. (A-B) Circular plots show the averaged frequency distribution of the saccade landing direction binned in evenly distributed angular sectors of 5°, in the 90° (A) and 30° conditions (B). Stimulus configuration is rotated as to align the two saccade targets symmetrically around the geometrical angle zero (see central insets). (C-D) Bar graphs illustrate averaged frequency of trials as a function of the saccade landing direction binned in 24 evenly distributed angular sectors of 15°. Data are shown for the three positions of interest (ST1, BTW and ST2) in the 90° (C) and 30° conditions (D). (E-H) Averaged saccade latency (E, F) and amplitude (G, H) observed for the same three positions of interest in the 90° (E, G) and 30° conditions (F, H). All data are shown irrespective of the duration (continuously or transiently) of the saccade targets. Light gray areas and error bars represent SEM. Polar plot black lines and corresponding light gray areas show linear interpolation between data points. This figure has been reprinted from Wollenberg et al. (2018)23. Please click here to view a larger version of this figure.
Figure 4: Visual sensitivity. (A-B). Circular plots show averaged visual sensitivity (d') as a function of the DT position in the 90° (A) and 30° conditions (B), irrespective of the duration of the saccade targets and across all saccade directions observed. Bar graphs illustrate visual sensitivity for four positions of interest (ST1, BTW, ST2, CTRL). (C-D) Visual sensitivity as a function of the DT position relative to the saccade landing direction in the 90° (C) and 30° conditions (D), irrespective of the duration of the saccade targets (blue: saccade to ST1; green: saccade to BTW; red: saccade to ST2). For each saccade direction, we took the average sensitivity for each discrimination target location. For example, the blue line plots visual sensitivity when saccades were made towards ST1 and the discrimination target was either at ST1 (+15° on the polar plot), BTW (15° counterclockwise to ST1; 0° on the polar plot) or ST2 (30° counterclockwise to ST1; + 345° on the polar plot), and so on. (E-F) Bar graphs illustrate sensitivity observed for DT shown at the saccaded (purple: e.g., DT at ST1 and saccade to ST1) and the non-saccaded positions (light-purple: e.g. DT at ST1 and saccade to ST2 or BTW) in the 90° (E) and the 30° (F) conditions. Conventions are as in Figure 3. This figure has been reprinted from Wollenberg et al. (2018)23. Please click here to view a larger version of this figure.
Figure 5: Corrective saccades. (A) Circular plot shows averaged frequency distribution of the corrective saccade landing direction following an averaging saccade. (B) The bar graph illustrates averaged frequency of trials as a function of the corrective saccade landing direction following an averaging saccade for three positions of interest (ST1, BTW and ST2). (C) The bar graph illustrates visual sensitivity as a function of the direction of the first corrective saccade for all trials in which an averaging saccade was executed. Purple bars show visual sensitivity for trials in which the corrective saccade was directed towards the location at which the DT appeared (e.g. DT at ST1 and corrective saccade towards ST1). Light purple bars show visual sensitivity for trials in which the corrective saccade was directed towards a different location than the location at which the DT appeared (e.g. DT at ST1 and corrective saccade towards ST2 or BTW). Conventions are as in Figure 3-4. This figure has been reprinted from Wollenberg et al. (2018)23. Please click here to view a larger version of this figure.
This experimental protocol employed the concurrent assessment of gaze behavior and presaccadic visual sensitivity in a free choice saccade task. It allowed us to analyze whether visual attention is indeed obligatorily coupled to oculomotor programming at the behavioral level and therefore systematically deployed at the endpoint of saccades. Two saccade targets were presented in close proximity (30°) on half of the trials in which we observed a distinct global effect, reflected in a saccade landing distribution consisting of both, accurate and averaging saccades. To conclude on the spatial coupling between visual attention and the saccadic endpoint, visual sensitivity was analyzed and compared across different locations as a function of the saccade landing direction. While we observed a consistent and selective enhancement of visual sensitivity at the endpoint of accurate saccades, visual sensitivity was not enhanced at the endpoint of averaging saccades. Rather, visual sensitivity was equally facilitated at the two saccade targets before the execution of averaging saccades, suggesting that saccade averaging arises from unresolved attentional selection among the saccade targets. Our results, therefore, demonstrate that visual attention is not obligatorily deployed at the endpoint of the executed oculomotor program. A corrective saccade analysis, which demonstrated that the even deployment of visual sensitivity across the two saccade targets before averaging saccades did not systematically vary as a function the corrective saccade direction, further confirmed this interpretation. In this regard, our data are consistent with a recent model by Zirnsak et al.25 which assumes a target selection process that gradually builds up over time.
Importantly, our protocol differs from other behavioral studies that reported evidence for a dissociation between attention and saccades in several aspects. While some studies based their conclusion on saccadic reaction times14,15,16, we used a direct measure of visual spatial attention, namely visual sensitivity during saccade preparation. The specific parameters used throughout this protocol were effective in replicating the classical presaccadic shift of attention8,9 as evident in the consistent enhancement of visual sensitivity at the endpoint of accurate saccades. Thus, the protocol allowed for reliable detection of systematic modulations of visual attention specific to saccade preparation. This is an important prerequisite to validly interpret presaccadic attentional effects, and in particular, the observed absence of attentional enhancement at the endpoint of averaging saccades.
A central and distinct aspect of this protocol was the random presentation of the discrimination target among various locations. Thereby, we could sample visual sensitivity across the entire visual field, which consequently allowed us to not only determine whether attention is facilitated at the endpoint of saccades but also to investigate the spread of attention around this area including adjacent locations. The discrete allocation of attention at the two saccade targets (limited to less than ~2.6°, the distance between two of our adjacent stimuli) we observed before averaging saccades contradicts an early account which suggested that averaging saccades may reflect a coarse processing of the visual scene17 and argues for an enhancement of local rather than global visual information processing. Further, the random presentation of the discrimination target rendered its location completely unpredictable to participants. Thus, our protocol generally facilitated an even deployment of visual attention across the visual field with respect to the discrimination task. We consider this fact important with regard to any potential attentional effects and conclusions related to saccade preparation.
However, since eye movements are not typically performed while trying to discriminate stimuli across the entire visual field in natural vision, the results obtained in this protocol can not unrestrictedly account for oculomotor behavior in every day life. Furthermore, the oculomotor task did inevitably bias the deployment of attention via the saccade target cueing. The saccade targets did not only introduce relevant oculomotor goals but also salient exogenous cues that likely attracted attention. Thus, it is possible that visual discrimination performance at the intermediate location was generally deteriorated due to some masking elicited by the two surrounding saccade targets. In order to decrease the impact of the saccade targets with respect to discrimination performance at the intermediate location and to evaluate potential masking effects, we decided to present the saccade targets only transiently (for 50 ms) instead of continuously (until the trial end) on half of the trials. Consequently, even though both cueing conditions introduced visual onsets at the saccade target locations, the saccade targets had always disappeared before the onset of the discrimination target in the transient cueing condition. While the majority of results were very consistent across both cueing conditions and were therefore combined in the final analysis, we indeed observed an indication of a masking effect in the continuous cueing condition relative to the transient cueing condition. Overall, irrespective of the saccade direction, visual sensitivity at the intermediate location was slightly decreased in the continuous compared to the transient cueing condition. Given the notion that transient saccade target cueing apparently minimizes masking of the intermediate location while still being effective in eliciting averaging saccades, future studies using a protocol similar to ours should consider employing transient cueing of targets. However, despite our efforts to minimize masking effects via the transient cueing of targets, we can not rule out that the transient cueing still introduced a forward masking effect, which might potentially account for the poor discrimination performance at the intermediate location to some extent.
Taken together, our protocol allowed to directly address the coupling between visual attention and oculomotor programming, and to reveal a distinct spatial dissociation between attention and the endpoint of averaging saccades at the behavioral level. Our results argue against a mandatory coupling between visual attention and oculomotor programming suggested in the premotor theory of attention. Future studies should employ paradigms including concurrent neurophysiological recordings within areas such as the FEF and SC to further resolve the coupling between visual attention and oculomotor programming.
The authors have nothing to disclose.
This research was supported by grants of the Deutsche Forschungsgemeinschaft to H.D. (DE336/5-1 and RTG 2175 "Perception in context and its neural basis") and to M.S. (SZ343/1) and a Marie Sklodowska-Curie Action Individual Fellowship to M.S. (704537).
Computer | Apple iMac (Cupertino, CA) | ||
CRT Screen | Sony GDM F900 (Tokyo, Japan) | 24 inch screen with a spatial resolution of 1024 x 640 pixels and a vertical refresh rate of 120 Hz | |
Eye Tracker | EyeLink 1000 Desktop Mount (SR Research, Osgoode, Ontario, Canada) | operating at a sampling rate of 1 kHz | |
Software | Matlab (The MathWorks, Natick, MA) / toolboxes: Psychophics, EyeLink |