New Framework for Understanding Cross-Brain Coherence in Functional Near-Infrared Spectroscopy (fNIRS) Hyperscanning Studies

In recent years, there has been a shift in the types of studies conducted to understand the neural bases of social behavior^1,2. Traditionally, studies in social neuroscience have focused on neural activation in one isolated brain during a socially relevant task. However, advances in neuroimaging technology now allow for the examination of neural activation in the brains of one or more individuals during social interaction as it occurs in "real-life" settings³. In "real-life" settings, individuals are able to move freely, and patterns of brain activation are likely to change as information is exchanged and social partners receive feedback from one another⁴.

Hyperscanning is a method that assesses this bidirectional information exchange by measuring the brain activity from two or more individuals simultaneously⁵. An emerging body of research has utilized functional near-infrared spectroscopy (fNIRS), a non-invasive neuroimaging technique that, in comparison to other neuroimaging techniques, is less susceptible to motion artifacts⁶. Hyperscanning via fNIRS allows for the assessment of inter-brain synchronization (IBS) in real-life settings while the interactive partners move freely and naturally. This is particularly relevant for work with infants and young children, who tend to be quite active. IBS has been reported to reflect mutual understanding between interactive partners, which serves as the foundation for effective social interaction and communication and mediates shared intentionality^1,7,8.

Several methods are used to evaluate the IBS of two brains. Such methods include time series correlations, such as cross-correlation and the Pearson correlation coefficient^9,10 (see a review by Scholkmann et al.¹⁰). Other methods involve evaluating the strength of coupling in the frequency domain. Such methods include phase-locking value (PLV) and phase coherence (see a review by Czeszumski et al.¹¹). One of the most common methods in fNIRS studies uses wavelet transform coherence (WTC)-a measure of the cross-correlation of two time series as a function of frequency and time¹⁰.

WTC uses correlational analyses to calculate the coherence and phase lag between two time series in the time-frequency domain. FNIRS hyperscanning studies have used WTC to estimate IBS in many domains of functioning, including action monitoring¹², cooperative and competitive behaviour^5,13,14,15, imitation¹⁶, mother-infant problem solving¹⁷, and teaching-learning behavior^18,19,20,21. Typically, in hyperscanning studies, cross-brain coherence, as measured by WTC, during an experimental task is compared to cross-brain coherence during a control task. These findings are usually presented with a WTC "hot plot", which shows the coherence across the two brains at each time point and frequency (see Figure 1).

As suggested by Czesumaski et al.¹¹, WTC has become the standard analytical approach for analyzing fNIRS hyperscanning. WTC analysis is a flexible, "tool-agnostic" method for data visualization and interpretation²². The coherence coefficient heatmap, which provides a narrative form of analysis allowing for the easy identification of periods of synchronous or asynchronous behavior as well as the intensity of brain activity during the completion of a task, is the main advantage of WTC and makes it a strong tool for applied research²². WTC has an advantage over correlation techniques. Correlations are sensitive to the hemodynamic response function's (HRF) shape, which is thought to differ between individuals (particularly in terms of age) and among different brain areas. In contrast, WTC is unaffected by interregional changes in the (HRF)²³. Researchers have used the wavelet approach to study fMRI time series. Zhang et al.²⁴ compared the commonly used functional connectivity metrics, including the Pearson correlation, partial correlation, mutual information, and wavelet coherence transformation (WTC). They carried out classification experiments using large-scale functional connectivity patterns derived from resting-state fMRI data and natural-stimulus fMRI data of video viewing. Their findings indicated that WTC performed best in classification (specificity, sensitivity, and accuracy), implying that WTC is a preferable functional connectivity metric for studying functional brain networks, at least in classification applications²⁴.

Figure 1: Wavelet transform coherence (WTC). WTC shows the coherence and phase angle between two time series as a function of both time (x-axis) and frequency (y-axis). The coherence increase is depicted by the red color in the graph, and the small arrows in the graph show the phase angle of the two time series. The rightward-pointing arrow represents in-phase synchronization; the downward-pointing and upward-pointing arrows represent lagged synchronization; and the leftward-pointing arrow represents anti-phase synchronization³⁰. This figure was adapted from Pan et al.¹⁹. Please click here to view a larger version of this figure.

Recently, Hamilton²⁵ articulated several limitations to the interpretation of cross-brain coherence data in fNIRS hyperscanning studies. One of Hamilton's primary concerns was that coherence measures (e.g., WTC) only report effects as symmetrical (i.e., two brains are correlated, showing the same pattern of change). However, many social interactions are asymmetric (e.g., information flow between a speaker and a listener) in that two participants may play different roles, and it is not clear that WTC can capture this information. Here, this concern is addressed by a new framework that allows for a straightforward interpretation of the cross-wavelet power by using the cross-wavelet phase to detect directionality. This framework will also allow examination of how the dynamics of interactions develop and change throughout a task.

While WTC and correlation methods assess functional connectivity, other methods assess effective connectivity, attempting to extract the causal influences of one neural element over another. Transfer entropy is a measure from the field of information theory that describes the transfer between jointly dependent processes²⁶. Another related method is Granger causality analysis (GCA), which has been described as equivalent to transfer entropy²⁶.

In the existing literature of fNIRS hyperscanning studies, Granger causality analysis (GCA) has been widely used to estimate the coupling directionality between fNIRS time series data obtained during a variety of different tasks, such as cooperation⁵, teaching¹⁹, and imitation¹⁶. GCA employs vector autoregressive models to assess the directionality of coupling between time series in brain data. Granger causality is based on prediction and precedence: "a variable X is said to 'G-cause' variable Y if the past of X contains the information that helps to predict the future of Y over and above information already in the past of Y"²⁷. Accordingly, the G-causality is analyzed in two directions: 1) from subject A to subject B and 2) from subject B to subject A.

While GCA analysis serves as a complementary analysis aimed at determining whether a high coherence value obtained using a WTC function reflects IBS or lagged synchronization (one signal leading the other), it does not allow for the determination of whether anti-phase synchronization has occurred. In traditional neuroimaging studies, in which only one participant is scanned (i.e., the "single-brain" approach), an anti-phase pattern means that the activity in one brain region is increased while the activity in the other brain region is decreased²⁸. In the hyperscanning literature, the presence of anti-phase synchronization may suggest that neural activation is increased in one subject, and at the same time, neural activation is decreased for the other subject. Therefore, there is a need to provide a comprehensive model that can detect the directionality. More specifically, this model will be able to detect anti-phase synchronization (in which the direction of activity in one individual is opposite to that of their partner) in addition to in-phase synchronization and lagged synchronization.

In an attempt to address the concern that WTC shows only symmetrical effects, where both brains show the same pattern of change²⁵, a new approach to identify the type of interaction by examining the phase of synchronization (i.e., in-phase, lagged, or anti-phase) is presented (see Figure 2). To this end, a toolbox using the WTC method to classify the different types of interactions was developed. The types of interactions are classified by using relative phase data from cross-wavelet transform analysis.

Figure 2: Illustration of the different phase relationships of simple sine waves. (A) When the two signals, Signal 1 (blue lines) and Signal 2 (orange lines), reach their respective maximum, minimum, and zero values at the same time point, they are said to be showing in-phase synchronization³². (B) When one signal reaches its maximum value and the other signal reaches zero value at the same time point, they are said to be showing lagged synchronization (one is leading by 90°)^32,33,34. (C) When two time series shift in opposite directions, meaning one signal reaches the maximum and the other reaches the minimum value at the same time point, this is referred to as anti-phase synchronization²⁸. (D–P) In all other phase relationships between two time series, one signal is leading the other. In all positive phases, Signal 2 is leading Signal 1 (e.g., panels E, F, M, and N), whereas in all negative phases, Signal 1 is leading Signal 2 (e.g., panels D, G, H, O, and P). Notably, when the absolute value of the phase is higher, it becomes more distinctive which time series is leading the other (e.g., the leadership is more distinctive in panel J than in panel I, and in panel K, the leadership is more distinctive than in panel L). Please click here to view a larger version of this figure.