The present protocol describes how to perform concurrent EEG and fNIRS recordings and how to inspect the relationship between the EEG and fNIRS data.
Concurrent EEG and fNIRS recordings offer an excellent opportunity to gain a full understanding of the neural mechanism of cognitive processing by inspecting the relationship between the neural and hemodynamic signals. EEG is an electrophysiological technology that can measure the rapid neuronal activity of the cortex, whereas fNIRS relies on the hemodynamic responses to infer brain activation. The combination of EEG and fNIRS neuroimaging techniques can identify more features and reveal more information associated with the functioning of the brain. In this protocol, fused EEG-fNIRS measurements were performed for concurrent recordings of evoked-electrical potentials and hemodynamic responses during a Flanker task. In addition, the critical steps for setting up the hardware and software system as well as the procedures for data acquisition and analysis were provided and discussed in detail. It is expected that the present protocol can pave a new avenue for improving the understanding of the neural mechanisms underlying various cognitive processes by using the EEG and fNIRS signals.
This study aims to develop a working protocol to reveal the neural activation pattern underlying the Flanker task by using fused EEG and fNIRS neuroimaging techniques. Interestingly, the concurrent fNIRS-EEG recordings allow for the inspection of the relationship between the hemodynamic signals in the prefrontal cortex and various event-related potential (ERP) components of the whole brain associated with the Flanker task.
The integration of various noninvasive neuroimaging modalities including functional near-infrared spectroscopy (fNIRS), electroencephalography (EEG), and functional magnetic resonance imaging (fMRI) is essential to improve the understanding of where and when information processing is taking place in the brain1,2,3. Additionally, there is the potential to combine fNIRS and EEG to examine the relationship between local neural activity and subsequent changes in hemodynamic responses, in which EEG and fNIRS can be complementary in revealing the neural mechanism of human brain cognitive function. fNIRS is a vascular-based functional neuroimaging technique that relies on the hemodynamic responses to infer brain activation. fNIRS measures the relative oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) concentration changes in the cerebral cortex, which plays an important role in the study of cognitive processing3,4,5,6,7. According to the neurovascular and neurometabolic coupling mechanism8, the change of local neural activity associated with cognitive processing is generally accompanied by subsequent alterations in the local blood flow and blood oxygen with a delay of 4-7 seconds. It is shown that the neurovascular coupling is likely a power transducer, which integrates the fast dynamics of neural activity into the vascular input of slow hemodynamics9. Specifically, fNIRS is mostly used for inspecting the neurovascular activity in the frontal lobe, especially the prefrontal cortex that is responsible for high cognitive functions, such as executive functions10,11,12, reasoning and planning13, decision making14, and social cognition and moral judgment15. However, the hemodynamic responses measured by fNIRS only indirectly capture the neural activity with a low temporal resolution, whereas EEG can offer temporally fine and direct measures of neural activities. Consequently, the combination of EEG and fNIRS recording can identify more features and reveal more information associated with the functioning of the brain.
More importantly, the multi-modal acquisition of EEG and fNIRS signals has been conducted to inspect the brain activation underlying various cognitive tasks16,17,18,19,20,21,22 or brain-computer interface23,24. In particular, concurrent ERP (event-related potential) and fNIRS recordings were carried out based on the event-related auditory oddball paradigm1, in which fNIRS can identify the hemodynamic changes in the frontotemporal cortex several seconds after the appearance of P300 component. Horovitz et al. also demonstrated the simultaneous measurements of fNIRS signals and the P300 component during a semantic processing task25. Interestingly, previous studies based on simultaneous EEG and fNIRS recordings showed that P300 during oddball stimuli exhibited a significant correlation with fNIRS signals26. It was discovered that the multi-modal measures have the potential to reveal the comprehensive cognitive neural mechanism based on the event-related paradigm26. Besides the oddball task, the Flanker task associated with ERP component N200 is also an important paradigm, which can be used for the investigation of cognitive ability detection and evaluation with healthy controls and patients with various disorders. Specifically, N200 was a negative component that peaks 200-350 ms from the anterior cingulated cortex frontal27 and superior temporal cortex28. Although previous studies examined the relationship between the superior frontal cortex and alpha oscillation in the Flanker task29, the correlation between the N200 amplitude and the hemodynamic responses during the Flanker task has not been explored.
In this protocol, a home-made EEG/fNIRS patch based on standard EEG cap was utilized for the concurrent EEG and fNIRS recordings. The arrangements of optodes/electrodes with support were achieved through the placement of fNIRS optodes fused into the EEG cap. The simultaneous EEG and fNIRS data acquisitions were carried out with the same stimuli tasks generated by E-prime software. We hypothesize that ERP components associated with the Flanker task can exhibit a significant correlation with the hemodynamic responses in the prefrontal cortex. Meanwhile, the combined ERP and fNIRS recordings can extract multiple signal indicators to identify the brain activation patterns with enhanced accuracy. To test the hypothesis, the fNIRS setup and EEG machine were integrated to reveal the complex neural cognition mechanism corresponding to the event-related Flanker task.
Prior to the experimental tests, all participants signed informed consent documents. The protocol for the present study was approved by the Ethics Committee of the University of Macau.
1. Hardware and software setting for concurrent EEG and fNIRS recordings
2. Experimental preparation
3. Run the experiment
4. Measurement of three-dimensional (3D) MNI coordinates of fNIRS optodes with 3D digitizer
5. Data analysis
Figure 2 shows the HbO signals for all channels while Figure 3 displays the ERPs at Fz and FCz for the two conditions of the Flanker task. Figure 4 illustrated the Pearson correlation analysis results showed that the fNIRS signals in SFC exhibited a significant correlation with the ERP N200 component at Fz for the incongruent condition (P<0.05). However, this is not the case for the congruent conditions (P>0.05).
Figure 1. fNIRS headset placement and channel configuration. The digitized optodes layout are converted into the MNI coordinate system and then overlapped along the brain cortex Please click here to view a larger version of this figure.
Figure 2. HbO signals for all channels associated with the Flanker task. The pink curves denote the incongruent condition while the green ones indicate the congruent condition. Please click here to view a larger version of this figure.
Figure 3. ERP signals for Fz and FCz electrodes. The black curves define the incongruent condition while the red ones denote the congruent condition. Please click here to view a larger version of this figure.
Figure 4: Correlation between the ERP N200 and HbO signals along the superior frontal cortex (SFC) for the incongruent condition. The regression coefficient between the two measurements is 0.59, p = 0.027. Please click here to view a larger version of this figure.
In this protocol, combined EEG and fNIRS recordings were performed to examine the brain activation patterns involving an event-related Flanker paradigm by recording the neural signals of the whole brain and concurrent hemodynamic responses of the prefrontal cortex. The ERP results showed that N200 at Fz was able to significantly distinguish the congruent and incongruent conditions (P=0.037). Meanwhile, the HbO signals in SFC (channels 21) also exhibited a significant difference between the congruent and incongruent conditions, which demonstrated the important role of the ability to suppress responses that involved the brain cognitive function associated with the Flanker task (PFDR = 0.041).
In addition, N200 at Fz showed a significant correlation with the hemodynamic response in the SFC (channel 21) for the incongruent condition although this was not the case for the congruent one. The brain activation in the prefrontal cortex is strongly correlated with high cognitive functions, which can be easily identified by fNIRS with the high signal-to-noise ratio in the spatial domain. However, the neural activity (N200) detected by EEG associated with the same Flanker task is mostly revealed in the parietal cortex with high sensitivity and high temporal resolution. N200 at Fz exhibited the cognitive difference between the two conditions, whereas fNIRS signals illustrated the difference of suppression function in the prefrontal region between the two conditions. It was discovered that the cognition showed a significant relationship with executive control during the Flanker task. This might be the main reason why the N200 at Fz exhibited a significant correlation with the hemodynamic response in SFC.
In this protocol, we described how to conduct fused EEG and fNIRS recordings and how to analyze the event-related potential and measure the hemoglobin concentration changes in the prefrontal cortex. The synchronization of different setups is an essential concern for the fusion of two hardware systems. Meanwhile, the event-related trigger is also the crucial mark for the task design of concurrent EEG and fNIRS recordings.
Combined EEG and fNIRS recordings are promising techniques for the investigation of the neural mechanisms underlying various cognitive tasks. In summary, we successfully acquired concurrent EEG and fNIRS data during a Flanker task. The findings indicated that the fNIRS hemodynamic response and ERP component N200 were significantly correlated, which exhibited different perspectives of the cognitive mechanism associated with the Flanker task. The multi-modal neuroimaging results support an essential role of combined EEG and fNIRS technique in contributing to brain cognition with different latencies and activation regions, which paves a new avenue for improving the understanding of the neural mechanisms of Flanker task.
The authors have nothing to disclose.
This work was performed in part at the high performance computing cluster (HPCC), which is supported by information and communication technology office (ICTO) of the University of Macau. This study was supported by MYRG2019-00082-FHS and MYRG 2018-00081-FHS grants from the University of Macau in Macau, and also funded by The Science and Technology Development Fund, Macau SAR (FDCT 0011/2018/A1 and FDCT 025/2015/A1).
EEG cap | EASYCAP GmbH | – | – |
EEG system | BioSemi | – | – |
fNIRS system | TechEn | – | CW6 System |