We present a protocol that employs eye tracking to monitor eye movements during an interval comparison (duration perception) task based on visual events. The aim is to provide a preliminary guide to separate oculomotor responses to duration perception tasks (comparison or discrimination of time intervals) from responses to the stimulus itself.
Eye-tracking methods may allow the online monitoring of cognitive processing during visual duration perception tasks, where participants are asked to estimate, discriminate, or compare time intervals defined by visual events like flashing circles. However, and to our knowledge, attempts to validate this possibility have remained inconclusive so far, and results remain focused on offline behavioral decisions made after stimulus appearance. This paper presents an eye-tracking protocol for exploring the cognitive processes preceding behavioral responses in an interval comparison task, where participants viewed two consecutive intervals and had to decide whether it speeded up (first interval longer than second) or slowed down (second interval longer).
Our main concern was disentangling oculomotor responses to the visual stimulus itself from correlates of duration related to judgments. To achieve this, we defined three consecutive time windows based on critical events: baseline onset, the onset of the first interval, the onset of the second interval, and the end of the stimulus. We then extracted traditional oculomotor measures for each (number of fixations, pupil size) and focused on time-window-related changes to separate the responses to the visual stimulus from those related to interval comparison per se. As we show in the illustrative results, eye-tracking data showed significant differences that were consistent with behavioral results, raising hypotheses on the mechanisms engaged. This protocol is embryonic and will require many improvements, but it represents an important step forward in the current state of art.
Time perception abilities have attracted increasing research attention over the last years, in part due to accumulating evidence that these may be linked to reading skills or pathological conditions1,2,3,4,5. Visual duration perception-the ability to estimate, discriminate, or compare time intervals defined by visual events-is one subfield of interest6,7in which eye-tracking methods could make a contribution. However, results remain focused on poststimulus behavioral decisions like pressing a button to indicate how much time has passed (estimate), whether time intervals are the same or different (discrimination), or which of a series of time intervals is the longest or shortest. A few studies have attempted to correlate behavioral results with eye-tracking data8,9 but they failed to find correlations between the two, suggesting that a direct relation is absent.
In the current paper, we present a protocol for registering and analyzing oculomotor responses during stimulus presentation in a visual duration perception task. Specifically, the description refers to an interval comparison task where participants saw sequences of three events that defined two time intervals and were asked to judge whether they speeded up (first interval longer than second) or slowed down (first shorter than second). The time intervals used in the study spanned from 133 to 733 ms, adhering to the principles of the Temporal Sampling Framework (TSF)10. TSF suggests that the brain's oscillatory activity, particularly in frequency bands such as delta oscillations (1-4 Hz), synchronizes with incoming speech units such as sequences of stress accents. This synchronization enhances the encoding of speech, improves attention to speech units, and helps extract sequential regularities that may be relevant in understanding conditions like dyslexia, which exhibit atypical low-frequency oscillations. The goal of the study in which we developed the method presented here was to determine whether dyslexics' difficulties in visual duration perception (group effects on the interval comparison task) reflect problems in processing the visual object itself, namely movement and luminance contrasts11. If this was the case, we expected that dyslexics' disadvantage towards controls would be larger for stimuli with movement and low luminance contrasts (interaction between group and stimulus type).
The main result of the original study was driven by poststimulus behavioral judgments. Eye-tracking data – pupil size and number of fixations – recorded during stimulus presentation were used to explore processes preceding the behavioral decisions. We believe, however, that the current protocol may be used independently from behavioral data collection, provided that the goals are set accordingly. It may also be possible to adjust it for interval discrimination tasks. Using it in time estimation tasks is not so immediate, but we would not rule out that possibility. We used pupil size because it reflects cognitive load12,13,14, among other states, and may thus provide information on participants' skills (higher load meaning fewer skills). Regarding the number of fixations, more fixations may reflect the participants' stronger engagement with the task15,16. The original study used five stimulus types. For simplification, we only used two in the current protocol (Ball vs. Flash, representing a movement-related contrast).
The main challenge we tried to address was disentangling responses to the visual stimulus itself from those related to interval comparison since it is known that oculomotor responses change according to characteristics such as movement or luminance contrasts17. Based on the premise that the visual stimulus is processed as soon as it appears on screen (first interval), and interval comparison is only made possible once the second time interval begins, we defined three time windows: prestimulus window, first interval, second interval (behavioral response not included). By analyzing changes from the prestimulus window over the first interval, we would get indices of participants' responses to the stimulus itself. Comparing the first to the second interval would tap into possible oculomotor signatures of interval comparison-the task participants were asked to perform.
Fifty-two participants (25 diagnosed with dyslexia or signaled as potential cases and 27 controls) were recruited from the community (through social media and convenience email contacts) and a university course. Following a confirmatory neuropsychological assessment and subsequent data analysis (for more details, see Goswami10), seven participants were excluded from the study. This exclusion comprised four individuals with dyslexia who did not meet the criteria, two dyslexic participants with outlier values in the primary experimental task, and one control participant whose eye-tracking data was affected by noise. The final sample was composed of 45 participants, 19 dyslexic adults (one male), and 26 controls (five male). All participants were native Portuguese speakers, had normal or corrected-to-normal vision, and none had diagnosed hearing, neurological, or speech problems. The protocol described here was approved by the local ethics committee of the Faculty of Psychology and Educational Sciences at the University of Porto (ref. number 2021/06-07b), and all participants signed informed consent according to the Declaration of Helsinki.
1. Stimulus creation
2. Experiment preparation
3. Apparatus setup
Figure 1: The eye-tracking setup. The spatial arrangement of the recording system, is composed of the stimulation computer, the recording computer, the eye-tracker, the response device (button box), and the keyboard. Participants sat 55 cm away from the stimulation screen. Please click here to view a larger version of this figure.
4. Preparation of data collection
5. Running the experiment
6. Creating time windows for analysis
Figure 2: Stimulus type. Sequences of bouncing balls (left) and flashes (right) that were used in the experiment. The dashed lines indicate the time windows used for analysis: TW0 is the prestimulus period; TW1 is the stimulus's first appearance on the screen and marks the first interval-when the participant has information about stimuli characteristics and the length of the first interval, and TW2 marks the second Interval-when the participant can compare the first to the second interval to elaborate a decision (slowed-down or speeded-up). Please click here to view a larger version of this figure.
7. Extracting measures
8. Remove trials with artifacts
9. Statistical analysis
To better understand TW-related changes, our analysis focused on the interaction of time windows (TW0 vs. TW1, TW1 vs.TW2) with stimulus type and group. As depicted in Figure 3, both TW-related comparisons (TW01 and TW12) showed different levels of change according to Stimulus (TW x Stimulus interaction), with Balls eliciting more TW-related changes in oculomotor responses than flashes in both groups (no TW x stimulus x group interaction). This occurred for both the pupil size and the number of fixations. Regarding group influences, we found a TW x group interaction on the change in the number of fixations from TW0 to TW1 (response to stimulus onset): dyslexics showed decreased change, mainly due to lower prestimulus values. Interactions between TW, stimulus, and group were absent. This shows that group influences were similar for both balls and flashes.
Figure 3: Results. Time-window-related changes in pupil size and number of fixations as a function of group (control vs. dyslexic, TW x Group) and stimulus type (Balls, B, vs. Flashes, F, TW x Stimulus). TW 0-1 addresses the contrast between no stimulus and stimulus visibility; TW 1-2 compares the first and second intervals to address interval comparison. The 95% confidence intervals are represented by vertical bars. Balls elicited more changes than flashes from TW0 over TW1 (more decrease) and from TW1 over TW2 (more increase) in both eye-tracking measures and both groups (TW x stimulus, no TW x stimulus x group). Changes in the number of fixations across TW 0-1 were smaller in dyslexics than controls regardless of stimulus type (TW x group, no TW x stimulus x group). Please click here to view a larger version of this figure.
Figure 4: Behavioral results. (A) Discrimination between speed-up and slow-down sequences (d-prime) per group and stimulus type. (B) Significant correlations between behavioral performance (d-prime) and time-window-related changes in eye movements, both stimulus-averaged. Please click here to view a larger version of this figure.
Critically, these values paralleled behavioral findings (Figure 4A), in line with the main study: behavioral findings pointed to stimulus effects (less accuracy for Balls than for Flashes) and group effects (worse performance in dyslexics), without group x stimulus interactions. Moreover, in the original study with five different stimuli, we correlated behavioral with eye-tracking data (number of fixations) averaged for all stimulus types and found a correlation in the dyslexic group: smaller changes from TW0 over TW1 coexisted with improved performance. Altogether, the results seemed consistent with the hypothesis that these (adult) dyslexics may be resorting to compensatory strategies for deliberate control of attention to the stimulus itself in the prestimulus period (fewer fixations on the empty screen would favor focusing on the stimulus by the time it appeared). We found no such correlation in controls, suggesting that they might not need to resort to strategies to keep focus. The restricted dataset used here for illustration (two stimuli only, Balls and Flashes) showed the same pattern (Figure 4B): dyslexics, but not controls, showed significant correlations between d-prime (behavioral discrimination index) and TW01-related changes.
In sum, the eye-tracking results addressing participants' responses to both stimulus onset (TW 0-1) and interval comparison (TW 1-2) replicated the behavioral evidence that balls versus flashes elicit different responses in individuals with and without dyslexia (TW x stimulus on eye-tracking measures, stimulus effects on d-prime). One part of the eye-tracking results also paralleled the group effects on d-prime, in that changes in the number of fixations at stimulus onset (TW 0-1) were smaller in dyslexics. Moreover, interactions between stimulus and group (different levels of deviance in dyslexics for balls vs. flashes) were null for behavioral and eye-tracking data. Finally, the correlation between behavioral performance and the oculomotor response was significant in the dyslexic group.
Sequence | Tip | Interval 1 | Interval 2 | Difference |
1 | Speed up | 433 | 300 | 133 |
2 | Speed up | 300 | 167 | 133 |
3 | Speed up | 467 | 433 | 34 |
4 | Speed up | 733 | 167 | 566 |
5 | Speed up | 467 | 300 | 167 |
6 | Speed up | 433 | 134 | 299 |
7 | Speed up | 534 | 233 | 301 |
8 | Speed up | 500 | 433 | 67 |
9 | Slow down | 300 | 433 | -133 |
10 | Slow down | 167 | 300 | -133 |
11 | Slow down | 433 | 467 | -34 |
12 | Slow down | 167 | 733 | -566 |
13 | Slow down | 300 | 467 | -167 |
14 | Slow down | 133 | 434 | -301 |
15 | Slow down | 233 | 534 | -301 |
16 | Slow down | 433 | 500 | -67 |
Average interval | 377.1 | |||
Average difference | 212.6 | |||
Average difference/interval | 294.8 |
Table 1: Interval duration. Stimulus sequences for speed-up and slow-down sequences in milliseconds.
The current protocol contains a novel component that might be critical to tackling current obstacles to incorporating eye-tracking in visual duration perception tasks. The critical step here is the definition of time windows based on cognitive processes that putatively take place in each of these time windows. In the system we used, time windows can only be defined as Areas of Interest (a space-related concept that is coupled with time in these systems), but in other systems, it is possible to do this by exporting different segments of the trial. In addition to this temporal segmentation of the trial, it is important to focus on analyzing changes across time windows rather than the parameters per time window.
Concerning the modifications to the protocol that had to be made, they were mostly related to the dimensions of the area of interest. We made a first attempt using dynamic AOIs – defining a spatial selection around the stimulus that followed it, rather than the whole screen. However, we soon realized that we could be missing relevant events outside that area. Given that our measures were unrelated to focus on the stimulus (pupil size was expected to change according to cognitive load and not according to attention to the flash or ball; the number of fixations was expected to reflect spatial search), we chose to use the full screen as the region of interest.
The current protocol is an embryonic proposal that is still subject to many refinements. We will only highlight two of these, even though there is much more room for improvement. The first concerns the differences in the length of the three time windows, which preclude us from interpreting time window effects on the number of fixations (e.g., a longer time window entails more fixations, hence the decrease from TW0 to TW1, see Figure 3). One way of addressing this problem would be to consider the number of fixations per time unit.
The second relates to the correspondence between time windows and putative ongoing processes, which includes various issues. One is that TW1 does not represent just stimulus appearance but probably also an explicit form of interval estimation (first interval) subsidiary to interval comparison and likely absent in TW0. In a similar fashion, changes across time windows may also reflect changes in general processes such as sustained attention and working memory18, even though some of these changes could be expected in an interval comparison task (working memory load is expected to increase from TW1 over TW2). One way to attenuate these potential confounds would be to introduce control tasks related to pure duration estimation, sustained attention and working memory, and then base the eye-tracking data analysis on the comparison between experimental (interval comparison) and control tasks. Another issue is that the duration of TW0 was irrelevant to the task, and it is known that task-irrelevant durations may be deleterious to performance19. Future work could focus on improving this, namely by creating a difference of 300 ms between TW0 (irrelevant interval) and TW1 to better delimit visual processing responses, since a short event can be biased to be perceived earlier or later than its presentation by simply adding another event in near temporal proximity20,21.
Finally, spontaneous eye blinks can affect time perception by distorting it (dilating time if an eye blink precedes the interval, contracting if it occurs simultaneously), potentially introducing variability in intra-individual timing performance22. One way of minimizing this problem would be to apply an eye-blink-based correction factor in participants' behavioral judgments (e.g., assign a reliability rate to each judgment depending on the presence of blinks before or during the stimuli. Additionally, incorporating the statistical approach of treating trials as random variables may also aid in addressing this problem.
Regarding future research, an important topic to address would be the association between spontaneous eye blink rate (EBR) and time perception. EBR has been known to be a non-invasive indirect marker of central dopamine function (DA)23, and, more recently, high ERB was associated with poorer temporal perception. The study suggests an implication of dopamine in interval timing and points to the use of ERB as a proxy of dopamine measure24. Another important topic is the functional meaning of the (change-related) measures we analyzed, which is yet to be determined in the context of our paradigm. In the original study, as well as in the current simplified dataset, increases in pupil size from TW0 to TW1 were consistent with the idea of increased cognitive load, but the absence of group effects on this measure precludes further considerations. One pattern that seems to present is that smaller changes across time windows correlated with better behavioral performance (Flashes better than Balls, and d-prime in dyslexics related to smaller changes), but further research is needed.
Despite its limitations, the current protocol is, to our knowledge, the first to show parallel results in eye-tracking and behavioral data (same profile of effects), as well as some evidence of the correlation between the two.
The authors have nothing to disclose.
This work was supported by the Portuguese Foundation for Science and Technology under grants UIDB/00050/2020; and PTDC/PSI-GER/5845/2020. The APC was entirely funded by the Portuguese Foundation for Science and Technology under grant PTDC/PSI-GER/5845/2020 (http://doi.org/10.54499/PTDC/PSI-GER/5845/2020).
Adobe Animate | Adobe | It is a tool for designing flash animation films, GIFs, and cartoons. | |
EyeLink Data Viewer | It is robust software that provides a comprehensive solution for visualizing and analyzing gaze data captured by EyeLink eye trackers. It is accessible on Windows, macOS, and Linux platforms. Equipped with advanced capabilities, Data Viewer enables effortless visualization, grouping, processing, and reporting of EyeLink gaze data. | ||
Eye-tracking system | SR Research | EyeLink 1000 Portable Duo | It has a portable duo camera, a Laptop PC Host, and a response device. The EyeLink integrates with SR Research Experiment Builder, Data Viewer, and WebLink as well as many third-party stimulus presentation software and tools. |
Monitor | Samsung Syncmaster | 957DF | It is a 19" flat monitor |
SR Research Experiment Builder | SR Research | It is an advanced and user-friendly drag-and-drop graphical programming platform designed for developing computer-based experiments in psychology and neuroscience. Utilizing Python as its foundation, this platform is compatible with both Windows and macOS, facilitating the creation of experiments that involve both EyeLink eye-tracking and non-eye-tracking functionalities. |