The present protocol describes how to conduct fNIRS hyperscanning experiments and analyze brain-to-brain synchrony. Further, we discuss challenges and possible solutions.
Concurrent brain recordings of two or more interacting persons, an approach termed hyperscanning, are gaining increasing importance for our understanding of the neurobiological underpinnings of social interactions, and possibly interpersonal relationships. Functional near-infrared spectroscopy (fNIRS) is well suited for conducting hyperscanning experiments because it measures local hemodynamic effects with a high sampling rate and, importantly, it can be applied in natural settings, not requiring strict motion restrictions. In this article, we present a protocol for conducting fNIRS hyperscanning experiments with parent-child dyads and for analyzing brain-to-brain synchrony. Furthermore, we discuss critical issues and future directions, regarding the experimental design, spatial registration of the fNIRS channels, physiological influences and data analysis methods. The described protocol is not specific to parent-child dyads, but can be applied to a variety of different dyadic constellations, such as adult strangers, romantic partners or siblings. To conclude, fNIRS hyperscanning has the potential to yield new insights into the dynamics of the ongoing social interaction, which possibly go beyond what can be studied by examining the activities of individual brains.
In recent years, neuroscientists have started to study social interactions by recording the brain activities of two or more persons simultaneously, an approach termed hyperscanning1. This technique opens new opportunities to elucidate the neurobiological mechanisms underlying these interactions. To fully understand social interactions, it may not be sufficient to study single brains in isolation but rather the joint activities of brains of interacting persons2. Using different neuroimaging techniques, hyperscanning studies have shown that brain activities of interacting persons or groups synchronize, e.g., while they coordinate their actions3, make music4, communicate5, engage in classroom activities6 or cooperate7.
The article presents a protocol for conducting simultaneous recordings with functional near-infrared spectroscopy (fNIRS). Similar to functional magnetic resonance imaging (fMRI), fNIRS measures the hemodynamic response to brain activation. Changes in oxygenated and deoxygenated hemoglobin (oxy-Hb and deoxy-Hb) are calculated based on the amount of diffusively transmitted near-infrared light through tissue8. fNIRS is well suited for conducting hyperscanning experiments, especially with children, because it can be applied in less constrained and more natural settings than fMRI. Moreover, it is less prone to movement artifacts than both, fMRI and EEG9. In addition, fNIRS data can be acquired at high sampling frequencies (e.g., 10 Hz), thus it highly oversamples the relatively slow hemodynamic response and thereby potentially provides a more complete temporal picture of the brain hemodynamics10.
This protocol was developed within the study of Reindl et al.11 and has been slightly modified (in particular with respect to the channel placement and bad channel identification) more recently. The aim of the study was to examine synchronized brain activity of parent-child dyads. Using fNIRS hyperscanning, we assessed brain-to-brain synchrony in prefrontal brain areas of children (aged five to nine years) and their parents, mostly mothers, during a cooperative and a competitive computer task. Prefrontal brain regions were targeted as they had been identified as important regions for social interactive processes in previous hyperscanning studies1. The cooperative and competitive task were originally developed by Cui et al.12 and recently employed by several previous studies13,14,15. For the study of Reindl et al.11, the tasks were modified to be suitable for children. Participants were instructed to either respond jointly via button presses in response to a target (cooperation) or to respond faster than the other player (competition). Each child performed each task once with the parent and once with an adult stranger of the same sex as the parent. Within each child-adult dyad, wavelet coherence was calculated for the oxy-Hb signals of corresponding channels as a measure of brain-to-brain synchrony.
This protocol describes the procedures to collect fNIRS hyperscanning data of parent and child during the cooperative and competitive game. The overall procedure, however, is not specific to this research design but is appropriate for different populations (e.g., adult strangers, romantic partners, siblings, etc.) and can be adapted for a number of different experimental tasks. This protocol also outlines one possible analytical procedure, which covers necessary and optional data analysis steps, including fNIRS data preprocessing, bad channel detection, wavelet coherence analysis and validation by random pair analysis.
Prior to participation, all parents / children provided informed consent / assent. The study was approved by the ethics committee of the Medical Faculty of RWTH Aachen University.
1. Preparation before the Participant Arrives
2. Participant Arrival in the Laboratory
3. Data Analysis
Representative data of one parent-child dyad during the cooperative condition are shown in Figure 1. The cooperative task consists of three 30 s rest blocks and two task blocks, with 20 trials each, presented in alternating order. In each trial, participants have to react as simultaneously as possible to a signal to earn a point11.
Figure 1: Hyperscanning data analysis and representative results. Light intensity data is collected in 22 channels (CHs) of two participants. First, bad channels are detected and excluded from further analyses. Afterwards, light intensity data is converted to changes in oxy-hemoglobin (Δ Oxy-Hb) and deoxy-hemoglobin (Δ Deoxy-Hb). Signals are shown for one exemplary parent-child dyad in CH 8 during the cooperative condition. Data is preprocessed by reducing motion artifacts and slow drifts. Afterwards, the wavelet coherence is calculated from the preprocessed oxy-Hb signals. To estimate the significance of each wavelet coherence value, 300 surrogate time series are generated. If the observed wavelet coherence value is higher than 95% of the wavelet coherence values obtained from the surrogate time series at the same point in time and scale, it is regarded as significant. Significant wavelet coherence values are marked by solid lines surrounding the respective areas in the plot. Coherence in the task-related frequency band is depicted within the black box. Please note that the analysis steps and the parameterization depicted in the figure should be understood as an example. The optimal parameterization depends on the data, e.g., different parameters of the MARA algorithm might work best for different types of artifacts20, and there is no gold standard for any of the analysis steps yet. Please click here to view a larger version of this figure.
The results are exemplified for the fNIRS data of channel 8 of both participants of a parent-child dyad. Before preprocessing, raw light attenuation data, received from the fNIRS device, are converted to changes in oxy-Hb and deoxy-Hb for both participants. Next, fNIRS time series are preprocessed to reduce motion artifacts and drifts. Finally, the significant WTC is calculated from the preprocessed oxy-Hb signals of both participants.
Figure 1 illustrates a real valued WTC matrix, which is composed of the coherence coefficients in time and frequency domain (here in period length). The coefficients can range between 0 and 1, with 1 indicating a perfect relationship at a specific time and frequency between both signals24. The coefficients are visualized using a color map ranging from blue (little or no coherence) to red (strong or maximum coherence). Significant coherence values are marked by solid black lines surrounding the respective areas in the plot. The beginning and end of each task block are indicated by vertical dashed lines.
Results show a strong coherence throughout the experiment in a high frequency band, until a period length of ~ 1 s (1 Hz). This likely results from the cardiac rhythms of parent and child. Additionally, results show a strong coherence in a lower frequency band between ~ 2 s and 8 s period length (0.5 – 0.125 Hz). Trial lengths differed due to pseudo-randomized variable cue durations (600 – 1500 ms) and participants' individual reaction times but were around 7 s on average, assuming reaction times of about 1 s. Therefore, coherence in this low frequency range likely reflects a synchronization of brain activities of both subjects during the task.
In this protocol, we show how to conduct fNIRS hyperscanning experiments and one possible way to analyze brain-to-brain synchrony, measuring concentration changes of oxy-Hb and deoxy-Hb at frontal brain regions of two subjects simultaneously. FNIRS hyperscanning is relatively easy to apply: a single NIRS device is sufficient to measure brain activities of both subjects by splitting the optodes between them. Thus, no synchronization between different devices is necessary1. Moreover, since fNIRS does not require strict motion restriction, it is well suited for conducting hyperscanning experiments in a natural environment and in children. In the following, we highlight some critical issues when designing, analyzing and interpreting (fNIRS) hyperscanning experiments, discuss challenges as well as possible solutions.
Experimental Design. One important issue of hyperscanning studies concerns the experimental design. Two participants who complete the same experimental task independently of each other might show similar brain activities, which then might be detected as brain-to-brain synchrony26. To differentiate between brain-to-brain synchrony induced by the experimental task and by the social interaction, appropriate experimental control conditions are necessary. On the one hand, the cooperative and competitive tasks are very well suited because they differ only in the cooperative task component and not in the stimulus material and the participant's motor behavior. On the other hand, less standardized and more natural interactions (e.g., making a puzzle together) might induce more variance in social interactive behavior and might have a greater ecological validity.
Spatial registration of channels. One challenge in fNIRS hyperscanning is measuring hemodynamic activity in corresponding channels. Attaching emitter and detector probes at corresponding locations of two participants' heads does not warrant that activity in two corresponding cortical regions is tapped, as individual brain anatomy is liable to differ across participants. Simultaneously measuring an adult and a child exacerbates this problem by introducing developmental differences on top of anatomical ones. Moreover, with an increasing number of channels, the placement of the channels is less reproducible across subjects because of variability in head shape and size27. One optional accessory to the ETG-4000 is a probe positioning unit which creates probe positions relative to fiducial points on the head in three-dimensional space. These data can then be co-registered to the structural MR image of the participant's brain27. Acquiring MR images and using the positioning unit will enable the experimenter to better control whether activity is actually measured in corresponding brain regions across two participants. Additionally, researchers could partly circumvent this problem by calculating an all-to-all connectivity model, measuring the connection between any two channels of the two participants.
Influence of the systemic physiology. Another important issue is that hemodynamic changes are known to be influenced not only by the effect of the neurovascular coupling, thus neuronal activity, but also by systemic changes, such as changes in heart rate, blood pressure, breathing rate, and autonomic nervous system activity28. Therefore, any synchrony detected in the hemodynamic changes of two cooperating participants may also be attributable to a synchrony of those factors. Previous studies have shown that two interacting partners do indeed synchronize their physiological activities29. Note, however, that in tasks with different experimental conditions which are directly compared to each other, this is only a confounder if physiological coupling is more prominent in one but not the other condition. Nevertheless, it can be helpful to acquire physiological data in hyperscanning studies to enable experimental control of these parameters. Another option, as demonstrated recently by Nozawa et al.25, is to add measurement channels with a short source-detector (S-D) separation (e.g., 1 cm), which are sensitive to the superficial skin blood-flow signal. The corresponding component can then be removed from the fNIRS signal obtained from measurement channels with a regular S-D separation (e.g., 3 cm), thus reducing the influence of physiological confounders. Such a dual or multi-distance approach has been shown to improve the sensitivity to task-enhanced (here: communication-enhanced) brain-to-brain synchrony.
Data Analysis. Hyperscanning results depend on an estimator to quantify brain-to-brain synchrony. In the current study, we calculated the WTC of oxy-Hb signals of corresponding channels as a measure of brain-to-brain synchrony. Wavelet-based methods have the advantage that they consider the oscillatory dynamics of time series in the time-frequency space. The WTC is a non-directed measure calculated from wavelet transformed time series, representing the strength of the relationship between two time series. In future studies, it would be interesting to additionally include directed measures, such as Granger causality, in order to examine which participant "leads" the activity (see for instance Pan et al.15). Furthermore, while many previous fNIRS-based hyperscanning studies examine brain-to-brain synchrony in only one signal (e.g., oxy-Hb), it is advisable to consider both oxy-Hb and deoxy-Hb (and possibly total-Hb) in order to take full advantage of the fNIRS technique15.
Limitations. Although fNIRS offers a promising, rapidly growing neuroimaging technique, some technical limitations associated with the device need to be considered when planning such a study (for a recent review see Pinti et al.30). In comparison to EEG and fMRI, fNIRS is more resistant to motion artifacts, yet, it still requires sufficient motion artifact control and detection. There are several potential causes of artifacts. First, some participants tend to move their head abruptly, in particular infants and children, and thereby might pull on the fiber tracts, affecting the optode contact. Developments of new fiberless devices are more robust to movement and thereby allow investigations of active tasks30. The use of a chin-rest can serve as an additional motion artifact control; however, it limits the ability to record brain activities in natural interactions. Second, acquiring an adequate optode contact can be hindered by dark, curly and/ or thick hair of the participant. Placing the optodes can thus be time-consuming and a perfect signal is not always guaranteed. Third, depending on the fNIRS system, wearing optodes for a longer period of time can put pressure on the participant's head, which can be experienced as unpleasant. This does not only limit the recording time of the experiment but might also lead to more movement and artifacts (e.g., smaller children might pull on the cap). In addition to motion artifacts, it is noteworthy that fNIRS provides measures of the cortical surface only. Finally, there are no standardized data analysis guidelines yet. Several toolboxes were developed over the past years and first attempts were made to analyze the effectiveness of various preprocessing techniques (e.g., Brigadoi et al.31 and Cooper et al.32). Moreover, the analytical protocol presented in this article shows one way to analyze fNIRS hyperscanning data. Importantly, the selected parameters of the analysis should be understood as one possible option and not as a standard guideline. Several other analytical protocols for fNIRS hyperscanning have been developed in the last years by different research groups (see for instance Cui et al.12; Hirsch et al.33).
Conclusion. fNIRS hyperscanning is a promising technique to gain further insights into the neurobiological underpinnings of social interactions34. In the future, portable and fiberless NIRS devices may be particularly important when examining brain-to-brain synchrony in natural interactions and moving from the dyad towards larger groups of subjects. Finally, combining different neuroimaging techniques, e.g., EEG-fNIRS, may provide new insights, broadening our understanding of brain-to-brain synchrony.
The authors have nothing to disclose.
This work was funded by the Excellence Initiative of the German federal state and governments (ERS Seed Fund, OPSF449). The Hitachi NIRS system was supported by a funding of the German Research Foundation DFG (INST 948/18-1 FUGG).
NIRS measurement system with probe sets and probe holder grids | Hitachi Medical Corporation, Tokyo, Japan | ETG-4000 Optical Topography System | The current study protocol requires an optional second adult probe set for 52 channels of measurement in total as well as two 3×5 probe holder grids. |
raw EEG caps | EASYCAP GmbH, Herrsching, Germany | C-SCMS-56; C-SCMS-58 | Caps must be provided with holes for NIRS probes by the experimenter. Choose cap size the same size or slightly larger than participant's head circumference. |
Technical computing software | The MathWorks, Inc., Natick, MA | MATLAB R2014a (or later versions) | Serves as base for Psychophysics Toolbox extensions (stimulus presentation), SPM for fNIRS toolbox (fNIRS data analysis), and ASToolbox (WTC computation). |