The study of cognitive planning combining EEG and eye-tracking systems provides a multimodal approach to investigate the neural mechanisms that mediate cognitive control and goal-directed behavior in humans. Here, we describe a protocol for investigating the role of brain oscillations and eye movements in planning performance.
The planning process, characterized by the capability to formulate an organized plan to reach a goal, is essential for human goal-directed behavior. Since planning is compromised in several neuropsychiatric disorders, the implementation of proper clinical and experimental tests to examine planning is critical. Due to the nature of the deployment of planning, in which several cognitive domains participate, the assessment of planning and the design of behavioral paradigms coupled with neuroimaging methods are current challenges in cognitive neuroscience. A planning task was evaluated in combination with an electroencephalogram (EEG) system and eye movement recordings in 27 healthy adult participants. Planning can be separated into two stages: a mental planning stage in which a sequence of steps is internally represented and an execution stage in which motor action is used to achieve a previously planned goal. Our protocol included a planning task and a control task. The planning task involved solving 36 maze trials, each representing a zoo map. The task had four periods: i) planning, where the subjects were instructed to plan a path to visit the locations of four animals according to a set of rules; ii) maintenance, where the subjects had to retain the planned path in their working memory; iii) execution, where the subjects used eye movements to trace the previously planned path as indicated by the eye-tracker system; and iv) response, where the subjects reported the order of the visited animals. The control task had a similar structure, but the cognitive planning component was removed by modifying the task goal. The spatial and temporal patterns of the EEG revealed that planning induces a gradual and lasting rise in frontal-midline theta activity (FMθ) over time. The source of this activity was identified within the prefrontal cortex by source analyses. Our results suggested that the experimental paradigm combining EEG and eye-tracker systems was optimal for evaluating cognitive planning.
During the past 10 years, extensive research has been conducted to examine the role of oscillatory neural dynamics on both cognition and behavior. These studies have established that frequency-specific interactions between specialized and widespread cortical regions play a crucial role in cognition and cognitive control1,2,3. This approach highlights the rhythmic nature of brain activity, which helps coordinate large-scale cortical dynamics and underpins cognitive processing and goal-directed behavior4,5. There is substantial evidence showing that rhythmic oscillations in the brain are involved in various cognitive processes, including perception6, attentio7,8,9, decision-making10, memory reactivation11, working memory12, and cognitive control13. Different oscillatory mechanisms have been proposed to guide goal-directed behavior, with transient large-scale frequency-specific networks providing a framework for cognitive processing1,14,15. For example, recent findings suggest that specific frequency bands in the brain may reflect a feedback mechanism that regulates spiking activity, providing a temporal reference frame to coordinate cortical excitability and spike timing for producing behavior16,17,18. A review is provided by Helfrich and Knight19.
This body of evidence raises questions about how the prefrontal cortex (PFC) encodes planning task contexts and related behaviorally relevant rules. The PFC has long been thought to support cognitive control and goal-directed behavior through the oscillatory patterns of neural activity it generates, selectively biasing the neural activity in distant brain regions and controlling the flow of information in large-scale neural networks20. Additionally, it has been proposed that regions exhibiting local synchrony are more likely to participate in inter-regional activity21,22,23. In particular, cortical theta-band (4-8 Hz) oscillations, as measured by scalp electroencephalogram (EEG), have been proposed as a potential mechanism for transmitting top-down control across broad networks13. Specifically, theta-band activity in humans reflects high-level cognitive processes, such as memory encoding and retrieval, working memory retention, novelty detection, decision-making, and top-down control12,24,25,26.
Related to this, Cavanagh and Frank13 proposed two sequential mechanisms of control processes: the recognition of the need for control and the instantiation of control. The recognition of the need for control may be indicated by frontal midline theta (FMθ) activity originating from the medial prefrontal cortex (mPFC), which has been described in terms of event-related potential (ERP) components that reflect mPFC-related control processes in response to various situations, such as novel information27,28,29, conflicting stimulus-response requirements30, error feedback31, and error detection32. These ERP components, which reflect the need for increased cognitive control in the presence of novelty, conflict, punishment, or error, exhibit a common spectral signature in the theta band recorded at frontal midline electrodes26,27,33,34,35,36,37,38,39,40,41,42,43,44.
The EEG responses of FMθ activity display a pattern of phase reset and power enhancement in the theta frequency band26. Despite the limitations of the EEG method in terms of its spatial resolution, various sources of evidence have been collected to demonstrate that FMθ activity is generated by the mid-cingulate cortex (MCC)13. These theta dynamics are believed to serve as temporal frameworks that regulate the neuronal processes of the mPFC, which are subsequently augmented in response to events requiring heightened control26. This has been established through source analysis31,33,45,46,47, concurrent EEG and functional magnetic resonance imaging (fMRI) recordings48,49, and invasive EEG recordings in humans50 and monkeys51,52,53.
Based on these observations, the frontal midline theta is considered to serve as a universal mechanism, a common language, for executing adaptive control in different situations where there is a lack of certainty regarding the actions and outcomes, such as during planning. The behavioral paradigm that we propose in this protocol has been used to study cognitive planning and its temporal and neural characteristics. Although various mechanisms for cognitive control have been reported in other scenarios, the current protocol has allowed for the recent description of planning and its associated neural and temporal properties54. The cognitive process of planning comprises two distinct phases: the mental planning phase, during which an internal representation of a sequence of plans is developed55, and the planning execution phase, in which a set of motor actions are executed to achieve the previously planned goal56. Planning is known to require the integration of various components of executive functions, including working memory, attentional control, and response inhibition, making the experimental manipulation and isolated measurement of these processes challenging57,58.
Neuroimaging studies on cognitive planning have commonly used behavioral paradigms such as the Tower of London59,60,61; however, in order to control the confounding factors, the tasks used for studying cognitive planning can become limited and artificial, leading to less predictive and ecological validity62,63,64,65. To overcome this problem in the neuropsychology field, real-world planning situations have been proposed as ecological tasks62,63. The Zoo Map Task subtest in the Behavioral Assessment of the Dysexecutive Syndrome battery measures planning and organizational skills in a more natural and relevant manner64,66. This test is a pencil-and-paper test that involves planning a route to visit 6 out of 12 locations on a zoo map. The locations are common places that can be found in a regular zoo, such as an elephant house, lion cage, rest area, coffee shop, etc. There are two conditions that evaluate different levels of planning: i) the formulation condition, where the subjects are instructed to plan a route to visit six places in the order of their choice but according to a set of rules; and ii) the execution condition, where the subjects are instructed to visit six places in a specific order and following a set of rules as well. These two conditions provide information about planning skills in ill-structured (formulation) and well-structured (execution) problems67. The first is presented as a more demanding cognitive task in an open situation because it requires subjects to develop a logical strategy to achieve the goal. Before tracing a path, a sequence of operators must be devised; otherwise, errors are likely to occur. On the other hand, the execution condition requires a lower cognitive demand because solving a task involving following a specific imposed strategy only requires the subject to monitor the implementation of the formulated plan to achieve the goal66. On the other hand, the Porteus Maze is a well-known task in the field of psychology, particularly in the areas of cognitive psychology and neuropsychology, and it has been widely used as a tool to assess various aspects of cognition, such as problem-solving and planning68,69. The Porteus Maze task is a pencil-and-paper task that starts with a simple visual stimuli analysis and becomes increasingly difficult. The subject must find and trace the correct path from a starting point to an exit (among several options) while following rules, such as avoiding intersecting paths and dead ends, and acting as quickly as possible68. Each time a fork appears while drawing the path, the subjects make decisions to reach the goal and avoid breaking the given rules69.
Considering the limitations and strengths of the commonly used and ecological tasks, we designed our behavioral paradigm mainly based on the Zoo Map Task66 and the Porteus Maze Task68. The behavioral paradigm consists of four distinct stages that encompass the cognitive process of planning in a daily life scenario. These stages are as follows: Stage 1, planning, where the participants are tasked with creating a route to visit various locations on a map, ensuring adherence to the established rules; Stage 2, maintenance, where the participants are required to keep the planned route in their working memory; Stage 3, execution, where the participants execute their previously planned route by drawing and closely monitoring its accuracy; and Stage 4, response, where the participants report the sequence of animals visited according to their planned route54. Our paradigm enables the measurement of different parameters of planning ability using different stages, which reflect the various components of planning (such as working memory, executive attention, and visuospatial skills) in a more realistic manner since mapping out routes is a common occurrence in daily life. Additionally, to control for confounding factors, the paradigm includes a control task with a planning task structure and equivalent stimuli, which engages the executive cognitive components also involved in planning but excludes the planning process component. This allows for the separation of the planning process component for the comparison of both electrophysiological markers and behavioral parameters54.
Furthermore, eye-tracking has made significant contributions to cognitive neuroscience studies by providing a non-invasive method for measuring and analyzing eye movements, which can provide valuable insights into the cognitive processes and neural mechanisms underlying perception, attention, and cognitive functions. Measuring different types of eye movements with an eye-tracking system can provide valuable information about the cognitive processes and neural mechanisms involved in planning. For example, the following aspects can be measured: fixations, which are the periods of stable gaze during which visual information is acquired70; saccades, which are the rapid eye movements that are used to shift the gaze from one location to another71; smooth pursuit, which is a type of eye movement that allows the eyes to follow a moving object smoothly72; microsaccades, which are small, rapid eye movements that occur even during fixations73; and blinks, which are a reflex action that helps to keep the eyes lubricated and protect them from foreign objects74. These eye movements can provide insights into the cognitive processes involved in visual search, attention allocation70, visual tracking72, perception73, and working memory74, which are important components for planning and cognitive control.
On the other hand, recent studies on the locus coeruleus-norepinephrine (LC-NE) system have shown its relevant role in cognitive control75. The locus coeruleus (LC) projects to several brain regions, such as the cerebral cortex, hippocampus, thalamus, midbrain, brainstem, cerebellum, and spinal cord76,77,61. Particularly dense LC-NE innervations receive PFC brain areas associated with cognitive control75. Besides, some studies indicate that chronic hyperactivity of the LC system may contribute to symptoms of manic-depressive disorder, such as impulsivity and sleeplessness. In contrast, a chronic decrease in LC function has been linked to reduced emotional expression, a prevalent characteristic among patients suffering from depression78. An overactive response of the locus coeruleus to stimuli may lead to an excessive response in individuals with stress or anxiety disorders79. Therefore, alterations in the LC-NE system may contribute to the symptoms of cognitive and/or emotional dysregulation. Non-invasive techniques can be used to examine locus coeruleus activity, one of which is pupil diameter changes, which are mostly controlled by noradrenaline released from the locus coeruleus. Noradrenaline acts on the iris dilator muscle by stimulating the alpha-adrenoceptors and on the Edinger-Westphal nucleus, which sends signals to the ciliary ganglion and controls iris dilation through the activation of postsynaptic alpha-2 adrenoceptors66,80,81,82. Direct LC neuronal recordings from monkeys have confirmed the relationship between LC-NE activity, pupil diameter, and cognitive performance83. Pupil dilation has been repeatedly observed in response to enhanced processing demands in several cognitive tasks71,84,85,86,87.
Electrophysiological markers of cognitive control combined with eye tracking and pupillary recordings might disentangle crucial questions about how cognitive control and planning are implemented in the brain. The importance of using our protocol combining EEG and eye-tracker systems is two-fold. On the one hand, cognitive control seems to require the participation of distributed brain activity in precise temporal relationships, which constitute ideal candidates for studying brain network function. On the other hand, abnormalities in any of these capacities have a severe impact on normal behavior, as might be in the case of a variety of cognitive and neuropsychiatric disorders, such as attention-deficit/hyperactivity disorder88,89, major depressive disorder90,91, bipolar disorder91, schizophrenia92, frontotemporal dementia93, as well as disorders due to frontal lesions94. Additionally, the current protocol allows for using pupillometry as a parameter to compare LC-NE activity and oscillations using eye-tracking and electroencephalography. This might not only provide evidence for the theoretical relationship between LC-NE, pupillometry, and neural markers in humans but could also permit the tracking of the developmental trajectory of characteristics related to the LC-NE system during cognitive planning. However, in our model, we focused on testing whether there was a specific pattern of saccades during planning that could potentially result in specific oscillation changes95. Additionally, we used an eye-tracker system as an important part of examining the behavioral execution of a plan in the execution phase of our behavioral paradigm.
To sum up, this protocol might produce testable models of brain network dynamics that could serve as a platform for both further basic research and eventual clinical and therapeutic applications.
The protocol described here offers an innovative approach for assessing cognitive planning and cognitive control during a novel and ecological planning task in association with pertinent and complementary behavioral and physiological measurements, such as oscillatory and pupillary dynamics. During our experiments, EEG activity was recorded while the participants performed the planning task, in which the participants were instructed to first elaborate and then execute a plan. The control condition, which involved evaluating a pre-drawn path on the zoo map, was established to eliminate the cognitive planning aspect while maintaining a similar setting and structure. This approach enables the assessment of whether cognitive planning, as a cognitive control function, leads to the generation of frontal theta activity from PFC regions and whether different PFC theta oscillation sources are linked to different aspects of planning performance. Another aspect that might be evaluated using this protocol is the differentiation of the various cognitive processes involved during phases of planning, such as plan elaboration, plan execution, plan achievement, and feedback processing. We found that planning induced a canonical frontal theta activity associated with cognitive control, which contributed to efficiently achieving a goal. These results underpin the interest of this protocol.
Despite the vast developments in cognitive neuroscience, most neuroimaging experiments examine isolated cognitive functions using artificial tasks in sensorimotor-deprived environments and oversimplified stimuli to control confounds. Problematically, those experiments might not be able to identify the real brain mechanisms involved when a cognitive function is implemented in an everyday situation (during ecological situations)138,140. In particular, the capacities for formulating goals, planning, and executing plans effectively are difficult to assess since they require various cognitive functions (working memory, inhibitory control, cognitive flexibility, etc.)104,144. Thus, designing an ecological behavioral task is encouraged and suggested based on the current trends in cognitive neuroscience142,143,145,146.
Our planning task, despite taking place in a laboratory environment (inside a room with the stimuli displayed on a screen), was made engaging and interactive for participants through the use of meaningful stimuli and goals that they could interact with on the screen. Additionally, the task requires participants to engage in a real-life situation-planning a path to visit various locations. To have an ecological task design, the paradigm must challenge the subjects to perform a specific behavior or cognitive function in a manner similar to what they would have to do in everyday life62,63,147. To develop an ecological task design, the planning task used here involves planning a path to visit different places in several stages56. The first stage involves the participants creating a plan while ensuring it follows a set of rules. The second stage involves maintaining the plan in working memory, while the third stage involves executing the plan and monitoring its adherence to the rules. These stages represent different phases of planning and the orchestration of other executive functions, such as cognitive flexibility, inhibitory control, and working memory. In order to have a valid ecological cognitive task design, the task should be able to detect specific cognitive impairments in patients with psychiatric or cognitive disorders who have difficulty performing that specific cognitive function in their daily life105. This can be achieved through future research using this protocol.
The behavioral results obtained through the use of this protocol were aligned with the experimental predictions. A significant difference in behavior was observed when the planning component was removed from the control task to form a control condition, thus facilitating further comparisons. The planning condition was found to be cognitively more demanding than the control condition, as evidenced by parameters such as the reaction time and accuracy. This may reflect the increased involvement of high cognitive functions in the implementation of planning23,55,56,57,148,149,150.
Considering that the control condition involved less complex cognition, as evidenced by the faster reaction times, better performances, and different cognitive processes required (the evaluation of rules), a possible alternative modification is to exploit the complexity levels present in the planning task, manipulate them, and analyze the planning function parametrically according to different levels of complexity (for instance, increasing the number of trials, and creating difficult, medium, and easy trial conditions). However, the results from our protocol showed that while it was possible to distinguish between "easy" and "difficult" trials based on behavioral measures, there were no differences detected in electrophysiological measures. This suggests that the results in our protocol more accurately reflect the intrinsic characteristics of the planning function rather than broader aspects of cognitive control, such as attention, mental effort, level of difficulty, or a high level of cognitive demand54. Nevertheless, further research may consider other types of control conditions, such as following a marked path that visits the four animal locations but also remembering the sequence order. This way, the level of difficulty could be better controlled, and planning could be distinguished from working memory, but one possible disadvantage of this is fatigue, because the subjects would have to perform two highly demanding tasks.
Several studies have linked different eye movement parameters with specific cognitive events. On the one hand, certain studies have found correlations between theta oscillations and pupil diameter during cognitive tasks, suggesting a relationship between these two measures of cognitive function. For example, Lin et al.152 found a correlation between midfrontal theta activity and changes in pupil size, reflecting varying degrees of subjective conflict. Their findings suggest that these signals represent conflict processing, increased attention, and flexible behavioral responses. Hence, the relationship between midfrontal theta activity and pupil responses seems to play a role in weighing costs and benefits during decision-making processes. In another study, Yu et al.153 examined how time-on-task engagement neurophysiologically affects cognitive control through a working memory task that modulated control inhibitory responses. They studied the relationship between pupil diameter data and frontal theta activity and showed that as the task duration increased, the performance decreased, and this was accompanied by a decrease in pupil dilation modulation and frontal theta activity. At the beginning of the task, they found a strong correlation between task engagement, theta activity, and cognitive control, as indicated by pupil dilation modulation, principally for demanding tasks that required high working memory and inhibitory control. However, this relationship dissipated toward the end, signaling a disconnection between the effort invested and the cognitive control used for performing the task, which is a hallmark of prefrontal time-on-task effects153. On the other hand, other studies have investigated saccades and oscillations. For example, Nakatani et al.154 revealed that, in a perceptual task, the alpha band amplitude from occipital regions predicted blinks and saccade effects. Moreover, Velasques et al.155 showed that, during a prosaccadic attention task, the saccade amplitude was associated with frontal gamma changes. Furthermore, Bodala et al.156 found that decreases in frontal midline theta were accompanied by decreases in sustained attention, as well as the amplitude and velocity of saccades. These findings suggest that eye movements, especially saccades, may reflect cognitive processes rather than just contributing to the background noise in EEG signals. In the current study, we enhanced the elimination of eye movement-related artifacts using the ICA algorithm with the saccade-to-fixation variance ratio criterion126. This criterion enhances artifact removal for free viewing tasks157. In our study, no differences were observed in saccade peak velocity and the coherence between theta power and saccade rate across conditions. However, more studies are needed to address these questions.
A critical step in using this protocol is constantly calibrating the eye tracker during the experiment, as the loss of gaze data from the camera could result in errors that would contaminate the task and make it difficult to obtain accurate responses. Therefore, it is crucial to calibrate as often as possible. However, there is a trade-off between the number of trials with a calibrated eye tracker and the length of the experiment. In our study, we decided to calibrate every five trials.
Future research exploring the relationship between theta oscillations and pupil diameter during this planning task should be conducted. Planning is a critical aspect of executive control that requires the allocation of attentional resources and the coordination of multiple cognitive processes. Understanding the relationship between theta oscillations and pupil diameter during planning tasks could provide valuable insights into the underlying neural mechanisms of executive control and how they change over time. Furthermore, such studies could lead to a deeper understanding of how changes in cognitive function, such as fatigue or attentional lapses, affect performance on planning tasks and the ability to allocate resources effectively. This information could have important implications for developing interventions aimed at improving planning performance, such as cognitive training programs or treatments for conditions such as attention-deficit/hyperactivity disorder (ADHD), among others.
Previous research has indicated that the PFC plays a crucial role in cognitive planning, as confirmed by our results. These results demonstrate that cognitive planning induces FMθ activity in the PFC, particularly in the anterior cingulate cortex, mid-cingulate cortex, and superior frontal regions54. These findings align with prior research on executive functions. There is substantial evidence to support the idea that FMθ activity acts as a top-down process for initiating control and facilitating communication between brain regions during demanding tasks13. While only a few studies have examined the temporal dynamics of FMθ activity associated with cognitive control, there is widespread agreement that the time profile of FMθ can provide information about various facets of cognitive control and the involvement of distinct PFC regions. Using our protocol for evaluating cognitive planning allowed us to characterize the time profile of FMθ activity during planning. Specifically, the FMθ activity during the planning condition showed a progressive increase. By implementing this protocol, for the first time, it was demonstrated that FMθ, is also present during planning implementation, as in other higher-order cognitive functions, and its temporal dynamics may serve as an indicator of cognitive control.
Our results and protocol have potential applications in the field of neuroscience, including for improving virtual neuropsychological assessments and the treatment of neuropsychiatric disorders with associated cognitive planning issues, such as depression and attention-deficit/hyperactivity disorder. For example, assessments could involve examining different patterns of errors at the behavioral performance level, different oscillatory patterns at the electrophysiological level, and different eye movements. Additionally, the results of this work may inform the development of brain-computer interfaces and cognitive training programs that aim to enhance cognitive planning abilities.
The present protocol may contribute new evidence to understanding the neural mechanisms that underpin the elusive cognitive control function of cognitive planning in neurotypical and neuropsychiatric populations. Moreover, our behavioral paradigm might offer insights into the neurobiology of cognitive control and planning through the examination of electrophysiological, pupillometry, and behavioral measurements with a practical planning task that examines intrinsic aspects of planning rather than the general cognitive demands typically present in cognitive control tasks, as reflected in the changes in theta oscillations.
The authors have nothing to disclose.
This research was financially supported by the doctoral scholarship program Becas de Doctorado Nacional año 2015 of ANID 21150295, FONDECYT regular grant 1180932, FONDECYT regular grant 1230383, FONDECYT de Iniciación grant 11220009, Postdoc grant Universidad de O'Higgins, and the Institut Universitaire de France (IUF). We want to thank Professor Pablo Billeke for his feedback on the paradigm design. We thank Professor Eugenio Rodríguez for kindly sharing his time-frequency analysis codes. Finally, we thank Milan Domic, Vicente Medel, Josefina Ihnen, Andrea Sánchez, Gonzalo Boncompte, Catalina Fabar, and Daniela Santander for their feedback.
EEG System | Biosemi | ActiveTwo Base system, 64 channels | |
Eye-tracker System | Eyelink SR Research | EyeLink 1000 Plus Core Unit, High-speed camera, Host PC/Monitor, | |
CPU display | Intel | Hard drive 221 GB, Processor Intel Core i7-4790 3.60 Hz, OS Windows 7, 4GB RAM | |
CPU EEG | Intel | Hard drive 223 GB, Processor Intel Core i7-4790 3.60 Hz, OS Windows 7, 4GB RAM | |
Monitor | ASUS | ASUS VG248QE 24" LCD monitor | |
Joytsick | Logitech | Model F310 | |
Luxmeter | Focket | LCD screen (0-200.000 lux) model Liebe WH LX1330B | |
Statistics software | GraphPad Prism | GraphPad Prism version 8 for Windows | |
MATLAB Programming Software | The MathWorks | MATLAB R2014a and R2018b | |
SVG tool Inkscape | Inkscape Project | vector graphic editor software | |
Presentation Software | Neurobehavioral Systems | stimulus delivery and experiment control program for neuroscience |
.