Home-Based EEG Hyperscanning for Infant-Caregiver Social Interactions

Vaidehi Ramanarayanan; Qian Chern Oon; Amritha Varshini Devarajan; Stanimira Georgieva; Vanessa Reindl

doi:10.3791/66655

JoVE Journal > Neuroscience

神经科学

Home-Based EEG Hyperscanning for Infant-Caregiver Social Interactions

Published: May 31, 2024

doi:

10.3791/66655

Vaidehi Ramanarayanan*¹, Qian Chern Oon*¹, Amritha Varshini Devarajan¹, Stanimira Georgieva^1,2, Vanessa Reindl^1,3

¹Department of Psychology, School of Social Sciences,Nanyang Technological University, ²Department of Pediatrics,University of Cambridge, ³Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Medical Faculty,RWTH Aachen University

Summary

This protocol describes how synchronized electroencephalography, electrocardiography, and behavioral recordings were captured from infant-caregiver dyads in a home setting.

Abstract

Prior hyperscanning studies that record the brain activities of caregivers and children concurrently have primarily been conducted within the confines of the laboratory, thus limiting the generalizability of results to real-life settings. Here, a comprehensive protocol for capturing synchronized electroencephalography (EEG), electrocardiography (ECG), and behavioral recordings from infant-caregiver dyads during various interactive tasks at home is proposed. This protocol demonstrates how to synchronize the different data streams and report EEG data retention rates and quality checks. Additionally, critical issues and possible solutions with respect to the experimental setup, tasks, and data collection in home settings are discussed. The protocol is not limited to infant-caregiver dyads but can be applied to various dyadic constellations. Overall, we demonstrate the flexibility of EEG hyperscanning setups, which allow experiments to be conducted outside of the laboratory to capture participants’ brain activities in more ecologically valid environmental settings. Yet, movement and other types of artifacts still constrain the experimental tasks that can be performed in the home setting.

Introduction

With the simultaneous recording of brain activities from two or more interacting subjects, also known as hyperscanning, it has become possible to elucidate the neural basis of social interactions in their complex, bidirectional, and fast-paced dynamics¹. This technique has shifted the focus from studying individuals in isolated, tightly controlled settings to examining more naturalistic interactions, such as parent-child interactions during free play²^,³, puzzle-solving⁴, and cooperative computer games⁵^,⁶. These studies demonstrate that brain activities synchronize during social interactions, i.e., show temporal similarities, a phenomenon termed interpersonal neural synchrony (INS). However, the great majority of hyperscanning studies have been confined to laboratory settings. While this allows for better experimental control, it may come at the expense of losing some ecological validity. Behaviors observed in the laboratory may not be representative of the participants' typical everyday interactive behaviors due to the unfamiliar and artificial setting and the nature of the tasks imposed⁷.

Recent advances in mobile neuroimaging devices, such as electroencephalography (EEG) or functional near-infrared spectroscopy (fNIRS), alleviate these issues by removing the requirement for participants to remain physically connected to the recording computer. Thus, they allow us to measure participants' brain activities while they interact freely in the classroom or in their homes⁸^,⁹. The advantage of EEG compared to other neuroimaging techniques, such as fNIRS, is that it has an excellent temporal resolution, which makes it particularly suitable for investigating fast-paced social dynamics¹⁰. Yet, it comes with the caveat that the EEG signal is highly vulnerable to motion and other physiological and non-physiological artifacts¹¹.

Despite this, the first studies have successfully implemented EEG hyperscanning set-ups in realistic environments and conditions. For instance, Dikker et al.¹² measured the EEG signal of a group of students while they engaged in various classroom activities, including attending lectures, watching videos, and participating in group discussions. This study, along with other studies⁸^,⁹, has predominantly utilized dry EEG electrodes to ease the process of conducting measurements in non-laboratory settings. Compared to wet electrodes, which require the application of conductive gel or paste, dry electrodes offer notable advantages in terms of usability. They have been shown to exhibit comparable performance to wet electrodes in adult populations and stationary conditions; however, their performance may decrease in motion-related scenarios due to increased impedance levels¹³.

Here, we present a working protocol to capture synchronized recordings from a low-density seven-channel liquid gel EEG system with a single lead electrocardiography (ECG) connected to the same wireless amplifier (sampling rate: 500 Hz) of infant-caregiver dyads in a home setting. While active electrodes were used for adults, passive electrodes were used instead for infants since the latter typically comes in the form of ring electrodes, thereby easing the process of gel application. Additionally, EEG-ECG recordings were synchronized to three cameras and microphones to capture the participants' behaviors from different angles. In the study, 8-12-month-old infants and their caregivers engaged in a reading and play task while their EEG, ECG, and behaviors were recorded. To minimize the impact of excessive movement on EEG signal quality, the tasks were conducted in a table-top setting (e.g., utilizing the kitchen table and an infant highchair), requiring participants to remain seated throughout the interaction task. Caregivers were provided with three age-appropriate books and table-top toys (equipped with suction cups to prevent them from falling). They were instructed to read to their child for approximately 5 min, followed by a 10 min play session with the toys.

This protocol details the methods for collecting synchronized EEG-ECG, video, and audio data during the reading and play tasks. The overall procedure, however, is not specific to this research design but is appropriate for different populations (e.g., parent-child dyads, friend dyads) and experimental tasks. The method of synchronization of different data streams will be presented. Further, a basic EEG preprocessing pipeline based on Dikker et al.¹² will be outlined, and EEG data retention rates and quality control metrics will be reported. Since the specific analytical choices depend on a variety of factors (such as task design, research questions, EEG montage), hyperscanning-EEG analysis will not be detailed further, but instead, the reader will be referred to existing guidelines and toolboxes (e.g.,¹⁴ for guidelines;¹⁵^,¹⁶ for hyperscanning analysis toolboxes). Finally, the protocol discusses challenges and potential solutions for EEG-ECG hyperscanning in the home and other real-world settings.

Protocol

The protocol described has been approved by the Institutional Review Board (IRB) of the Nanyang Technological University, Singapore. Informed consent was obtained from all adult participants and from parents on behalf of their infants.

1. Considerations of equipment and space at home sessions

Prepare for differing humidity and temperature conditions depending on country and season. For environments with high temperature and humidity levels, ensure that there is adequate airflow and switch on the air conditioning unit at home if possible.
Maintain distance from WiFi transmitters, Bluetooth equipment (e.g., mobile phones, keyboard, mouse, etc.), and microwave ovens, as nearby machinery operating in the same frequency band can cause interference. Additionally, try to keep the experimental setup away from charging devices in the house, as this could cause significant power-line noise in the data. Note that nearby standing fans or ceiling fans at high speeds could cause the EEG cables to sway and contribute to mechanical artifacts.
Prepare a conducive space for the experiment to be held within the participant's home. Ensure that the testing area has sufficient clear space to accommodate one caregiver and one infant in a highchair seated around a table, as well as three cameras on tripods. Clear all items from the table, if possible, to reduce any potential distractions.
Ensure that there is adequate brightness in the testing area for the camcorders to capture the participant's expressions. If the tables are close to windows, do not position the camcorders facing the window to prevent lens glare.

2. Preparations before the session

Inform the participants that for the EEG measurement, adults are required to wash their hair the day before the session without applying hair products. Ask if they could refrain from wearing makeup on the day of the EEG session.
Ensure that all recording equipment (microphones and camcorders), laptops, amplifiers, and the power bank used by the trigger box are charged the day before the session. Check that all equipment is fully charged a few hours before the session. Carry chargers or preferably portable power banks for unexpected cases of low battery levels in the equipment.
Pack all the necessary forms and recording equipment for the session.

3. Experiment preparation at the participant's home

EEG gelling preparation
1. Put on a pair of gloves when handling the gel. Fill four syringes with gel, two syringes with the blunt tip for the passive electrodes, and two syringes with the finer tip for the active electrodes.
2. Prepare two square pieces of masking tape for the infant and adult ECG electrodes.
Recording equipment preparation
1. Remove gloves when handling electronic equipment.
2. Turn on all the laptops and attach the EEG dongles to the respective laptops. Ensure that the EEG recording software is running.
3. Connect the adult and infant amplifiers to the EEG recording software on the respective laptops via a wireless connection.
4. Check and ensure that both the amplifiers (adult and infant) have the same trigger port settings, i.e., whether the trigger marker is produced at the rising (high-active) or falling edge (low-active) of the input pulse.
5. Mount the three camcorders on each tripod. Ensure that each camcorder captures the caregiver, infant, and both caregiver and infant (combined view), respectively (see Figure 1). Before starting the session, ensure that the camcorder's angles can consistently capture the participants' faces, considering that the caregiver often moves down to the infant's eye during interactions.
Place the infant's highchair and the caregiver's chair at the edge of their table so that they face each other at an angle at which the camcorders can catch their facial expressions (see Figure 1). If there isn't enough space on the table, place the infant's highchair and the caregiver's chair beside one another at a right angle.

Figure 1: Top-down view of set up. (1) Infant-facing camcorder. (2) Combined view camcorder. (3) Caregiver-facing camcorder. Please click here to view a larger version of this figure.

4. EEG and ECG sensor application for the caregiver

Securely attaching the amplifier to the EEG cap
1. For the caregiver, use the back pocket of the EEG cap to store the amplifier in order to reduce cable movement and EEG noise. Before applying the cap to the participant, ensure that the amplifier has been securely stored and attached to the connector.
2. Ensure that there is no contact with water while handling the amplifier (i.e., remove gloves if soaked in gel).
Request the caregiver to remove their glasses, mask, or earrings. If their hair is tied up, ask them to remove the hairband to have their hair down.
While wearing gloves and with the caregiver's permission, clean the forehead using alcohol swabs (70 % isopropyl alcohol (IPA)).
On both sides of the caregiver's head, part the hair at the topmost point of the ear to ensure that the ear is fully visible, following natural hair separation lines.
Measure the caregiver's head circumference by placing the measuring tape around four reference points on the head: the nasion (eyebrow level), over the crest of the inion (the highest point at the back of the head), and between the left and right tragus (the tips of the ears).
Stretch the EEG cap from the inside with a crown-like hand gesture and begin placing the cap from the caregiver's forehead towards the back of their head. Without letting go of the cap, slide hands down to hold the straps, pull them down towards their chin, and fasten the hook and loop ends. Adjust the straps according to the participant's comfort.
1. Ensure that any bangs or strands of hair on the participant's face are removed away from the face to prevent discomfort or obstruction of vision.
2. Gently smooth down the cap to ensure that the electrodes lie in close contact with the scalp. Ensure that there are no wrinkles on the cap.
Place the measuring tape over the electrodes at the midline and measure the distance between the nasion and the inion. Ensure that the Cz electrode of the international 10-20 system is at the midpoint of the measurement. Use the scrunch and release method to shift the positions of the cap back and forth if needed.
Measure the distance between the left and right tragus to ensure that the Cz electrode corresponds to the midpoint of the measurement and adjust if needed.
EEG and ECG gel application
1. Begin by applying gel to the reference and ground electrodes. Proceed to gel the remaining electrodes. Start with the electrodes at the back because it may take longer for the impedance to reduce to an acceptable range (typically < 25 kΩ) since adults typically have more hair at the back.
2. When applying gel on the caregiver, use a syringe with a longer and finer tip. Insert the syringe tip into the electrode opening and part the hair using the bottom curve of the syringe. Squirt small amounts of gel as the syringe is drawn out. If there is a large gap between the scalp and the electrode, firmly press down on the electrode to ensure contact.
3. Use light as an impedance reference guide if the given active electrode setup allows for that. Alternatively, refer to the impedance readings on the EEG recording software to identify which electrodes require contact improvement.
  NOTE: Electrodes with low impedance and good signal quality (ensured by checking for artifacts before starting the session) will be colored green on the cap and/or on the EEG montage. If the impedance for an electrode is high, repeat the effort to separate the hair and move the gel in the electrode well to the point of clear contact with the scalp.
4. Note that the cap usually does not perfectly fit each head, and concave areas of the head may cause a natural gap between the electrode and scalp. If impedance remains high, apply a small mound of gel to ensure a gel bridge between the electrode and the scalp. Be careful not to move the gel sideways after this to avoid bridging with neighboring electrodes underneath the cap.
5. Be mindful not to overfill the wells with gel to prevent bridging across adjacent electrodes over the cap as well.
6. Once all the EEG electrodes have low impedance, clean the soft area under the participant's left collarbone using alcohol wipes and begin to attach the ECG electrode.
  1. Attach a circular tape to the bottom of the ECG electrode and apply enough gel to cover the well of the electrode.
  2. Attach the electrode to the soft area under the left clavicle. Apply white masking tape on the top of the sensor.

5. EEG and ECG sensor application for the infant

Ask the caregiver to help the infant wear a vest, which is provided by the experimenters, with a back pocket, which will later store the amplifier.
With the caregiver's permission, clean the forehead using alcohol wipes. Measure the infant's head circumference by placing the measuring tape around the four reference points.
If the infant is in an apparent good mood and happily playing by themselves or with a caregiver, proceed with putting the EEG cap on. Consult the caregiver as to the infant's mood and predisposition to headwear. Prepare dry food or toys to occupy the infant's hands before capping, if agreeable with the parent. Additionally, place the infant on the caregiver's lap so that they feel comforted during the gelling process.
Place the cap on the infant's head with the same crown-like stretching motion, and fasten the hook and loop ends under the chin. Have the second experimenter or caregiver prompt the infant to look up, using a rattle, so that it is easier to fasten the hook and loop ends.
1. Ensure that any bangs or stray hairs on the infant's face are neatly tucked underneath the cap to prevent discomfort or obstruction of vision.
2. Gently smooth down the cap to ensure that the electrodes lie in close contact with the scalp. Ensure that there are no wrinkles on the cap.
Use the measuring tape to ensure that the cap is positioned correctly, with the Cz electrode at the midpoint or top of the head and use the scrunch and release method to adjust if needed (see step 4.7).
Securely attach the amplifier to the cap. Once connected, display the impedance readings.
Place the connected amplifier in the pocket at the back of the infant's vest.
EEG and ECG gel application
1. Begin by applying gel to the reference and ground electrodes. Proceed with gelling the remaining electrodes starting from the back.
  NOTE: Infants might be a little fussy with the experimenters touching their heads or from the cooling sensation of the gel. If this is the case, allow some time for the caregiver to soothe the infant before continuing. The second experimenter can distract the infant (e.g., by blowing bubbles, using toys, or giving some food).
2. Fill all electrode openings with gel and then proceed to use the cotton tips to part the hair in a gentle left-to-right motion if using passive electrodes (as here); refer to the montage on the EEG recording software to check for impedance while gelling.
3. Ensure that all the electrodes demonstrate low impedance on the EEG recording software, typically < 50 kΩ, since infants are less tolerant to the gelling process compared to adults.
4. Clean the soft area under the infant's left collarbone with baby wipes and attach the ECG below the infant's left clavicle following the same procedure as the adult ECG.

6. Creating a trigger box for multi-modal data synchronization

NOTE: Since different sensor data streams (i.e., EEG, ECG, video, and audio) will start recording at different time points during the session, they need to be manually synchronized to create a single timeline of events. Thus, a common event is needed that can be captured by both the camcorder (i.e., LED light) and the amplifier (i.e., digital or analog signal). To achieve this, an in-house synchronization trigger box is used, which can be built using a simple microcontroller unit program, as detailed below.

To build the trigger box, use a microcontroller development board, LED, BNC connector, power bank, a digital input device (i.e., a push-button), and electrical pulse/signal output (i.e., trigger wires) that connect to the amplifier's trigger port (see Figure 2B).
1. Connect the female BNC connector to the push-button, which serves as the input (e.g., input pin: 8), and the LED light and the electric pulse signal act as the output (e.g., output pin: 12; see Figure 2A).
2. Connect the BNC connector to the two amplifiers via the 2.5 mm electric wire cables, producing one-bit triggers marking the samples in the EEG-ECG recording when it reads the digital TTL signal from the push button (see Figure 2C).
Configure the trigger port settings of the adult and the infant amplifiers in an identical manner to ensure tight and accurate EEG-EEG synchronization.
1. Design the trigger system such that trigger markers are produced in both amplifier recordings when the trigger push button is released (after being pushed) as opposed to when it is being pushed.
2. Identify the current state of the trigger port in the amplifiers. If this is initially at 0 or LOW, set the port to low-active to produce a marker when the push button is released. Alternatively, if the initial state of the port is 1 or HIGH, set the port to be high-active to produce a marker when the push button is released (see Figure 3).

Figure 2: Building of Trigger Box. (A) Microcontroller circuit diagram for trigger box; (B) Interiors of the built trigger box; (C) Trigger box connected to the adult and infant EEG-ECG amplifiers, the trigger push button, and the power bank. Please click here to view a larger version of this figure.

Figure 3: High active and low active trigger port settings. Depending on the initial state of the trigger pin (0 or 1), the trigger port setting (High Active, HA or Low Active, LA) is chosen so that the marker is produced at the end of the pulse (when the trigger push button is released). Please click here to view a larger version of this figure.

7. Sensor streams synchronization

To facilitate ease of synchronization, attach the microphone receiver to its respective camcorder, thereby automatically synchronizing video (camcorder) with audio (microphone).
NOTE: Since the microphones are attached to the camcorders and the EEG and ECG electrodes come from the EEG electrode branch set connected to the same amplifier, these data streams are automatically pre-synchronized.
Attach the respective microphone transmitter to the collar of the caregiver's top and to the infant's vest.
Start the camcorder recordings for all three camcorders after ensuring that their positioning can capture the LED light signal from the Trigger Box in the room.
Start the EEG recording on the laptop and the amplifier (SD card version) for both the infant and caregiver. To ease post-session synchronization, start the cameras first, followed by the EEG sensors in order adult-infant since the infant EEG is the Master sensor to which every other sensor is synchronized (to avoid negative video-EEG offsets).
Attach both amplifiers to the trigger box setup using the 2.5 mm jack tightly to avoid spurious markers (Figure 2C).
NOTE: A trigger marker should be seen on the EEG continuous stream as the jack is attached to the amplifier.
Synchronize the data streams using the Trigger Box.
1. Perform long push-button presses as the difference in marker appearance between the two configurations (start vs end of button press) is only noticeable when the presses are considerably long (at least 1 – 2 s long).
2. Perform multiple long presses to produce multiple triggers to increase the accuracy and reliability of the estimated offsets between sensors and aid post-hoc synchronization in situations where some triggers are not registered by some of the sensors.
3. Check all cameras to verify whether the LED signal from the Trigger Box is visible in the recordings and check whether there are simultaneous markers displayed in the ongoing EEG recordings on the laptop.
Remove the amplifiers from the trigger box setup and ensure that the amplifiers are securely attached to respective EEG caps.
Perform this synchronization procedure at the start and at the end of the experimental session, particularly if web cameras are used to monitor dropped video frames, which causes the synchronization to drift.

8. Parent-infant interaction experiment

Before starting the parent-infant interaction task, remove all equipment from the table to remove potential distractions for the caregiver-infant interaction experiments.
Conduct the experimental tasks.
NOTE: The tasks were chosen to emphasize naturalistic interactions between the dyad with minimal instructions or involvement from the experimenter (e.g., read with your baby for 5 min as you would typically do at home). The length of the task is contingent on participants' comfort and the limitations of the equipment, e.g., the battery life of the amplifier.
If possible, ensure that all experimenters are hidden from the infant's view throughout the experiment to avoid distractions. One experimenter is to remain nearby with the recording laptops to be able to intervene in case any sensor streaming issues arise. Make sure that the recording laptops are in close proximity to the amplifiers (< 10 m) to avoid lost samples due to wireless (Wi-Fi, Bluetooth) degradation.

9. Clearing up at the end of the experiment

Stop the recordings on camcorders and the EEG recording software on the laptop and the amplifier. Turn off the amplifier and detach it from the cap.
Removing EEG caps
1. Gently remove the ECG tapes and electrodes starting from the caregiver or ask the caregiver to remove the infant's ECG if needed. As the infant's skin is sensitive, use baby wipes or warm water to wet the tape for easier removal.
2. Start by removing the hook and loop ends of the cap, then peel the cap off backward and turn it inside out. Use baby wipes to clean off the gel residue from the infant's head and proceed to remove the caregiver's cap.
3. Advise the caregiver that excess gel washes out easily in the shower.
Preparations for transportation
1. Place the EEG cap along with the EEG and ECG electrodes into a plastic bag. Ensure that the gel on the cap does not come in contact with the cap's splitter box. Keep the wires and the splitter box in a structured box during transport.
2. To prevent mechanical damage during transport, pack the amplifier in a padded box to minimize vibration.
Post-cap cleaning in the lab
1. Begin cleaning the EEG caps as soon as possible.
2. Put the EEG caps into a cleaning tub and sandwich the wires with dry cloths away from the water source. Ensure that the splitter box does not come in contact with water.
3. Pour approximately 1 L of water into the tub and add 10 mL of aldehyde-based disinfectants. Allow the cap to sit within the solution for 10 min. Do not keep the cap in the solution for too long or put too much disinfectant, as it will speed up the deterioration of the caps over time.
4. After 10 min, use a toothbrush to clean the gel residue from the electrodes under running water. Be particularly cautious while cleaning active electrodes since they have small electronic circuitry built into each electrode.
5. Rinse the cap thoroughly with water to remove the disinfectant solution.
6. Using a dry cloth, pat the cap dry and insert a dry cloth within the cap to absorb any remaining moisture. Close the Hook and Loop ends to keep the dry cloth within the cap.
7. Hang the cap to dry and ensure that the wet ends of the cap do not meet the connectors. To achieve this, place the cap at a lower position than the splitter box and connectors while hanging.
Data saving
1. Ensure that the data from the SD cards of the camcorders, amplifiers, and recordings from the laptop are exported and backed up onto the data storing site.

10. Data quality assurance

Video-audio data
1. Check that the sound is on, that no view was obstructed during the tasks, and that the markers are captured.
EEG-ECG data
1. Ensure that SD card recording is present and not corrupted. Check if the markers are captured on both the EEG recordings.
2. Check for technical faults, i.e., amplifier disconnecting and electrical or mechanical interference in the EEG / ECG signal.

11. Data processing

Synchronization of multi-sensor recordings
1. Import the caregiver, infant, and combined view camcorder videos into the video editing software.
2. Go through the video to mark the specific frames in which each LED trigger light first appears. Continue to go through the video and add another marker at the specific frame in which the LED light completely disappears for each trigger.
3. Complete these steps for all three videos. Once completed, write down the frame numbers of the markers from all videos in a spreadsheet.
4. Open the EEG marker files from the amplifier's SD card for the infant and the adult. Write down the marker sample information in a spreadsheet.
5. Since the video is recorded in frames per second (FPS, e.g., 25 FPS) and the EEG-ECG is recorded in samples per second (e.g., 500 Hz), convert the frame and sample numbers into a common measurement unit, such as milliseconds (ms), to be able to create a single timeline.
6. For every sensor (videos and physiology recordings), calculate the times in between triggers, here referred to as inter-trigger intervals (ITI), by subtracting the ms timestamp for every pair of consecutive markers.
  NOTE: Although the starting times of the recording sensors are different, the ITIs in ms for the same set of triggers across the different sensors (videos and EEGs) should still match. Therefore, to validate this, calculate the difference in each ITI (ITI Lag) between every sensor and the master EEG.
7. Check the quality of the video-EEG ITI Lags. Since the sampling rate/FPS is different between the two sensors, typically, the EEG has a much higher sampling rate than the videos, allowing an error with tolerance ± 1 video frame (here, 40 ms) for the computed EEG-video ITI lags.
8. Check the quality of the adult EEG – infant EEG ITI Lags. Since they have the same sampling rate, allow an error tolerance of ± 1 EEG sample (here, 2 ms).
9. Once these checks are complete, calculate the offsets between each sensor and the Master sensor's timeline (in this case, infant EEG).
10. Subtract the ms timestamp of each trigger marker of the Master EEG from the respective markers of each sensor (videos, adult EEG). This produces N [sensor – master EEG] offsets (N = number of triggers) for each sensor.
11. For each sensor, calculate the average of these offsets and round them up to produce the final offset number with respect to the master EEG. Use these offset numbers to cut the video and EEG data to start at the same time as the master EEG.
Simple video coding
1. To identify the different stages of the experiment within the videos (e.g., start and end of the experimental tasks or interruptions), note down the specific timestamps in frames using video editing software.
EEG preprocessing
1. Cut the EEG files of the adults and infants at the beginning and end of the task.
2. Identify and remove bad channels
  1. Note down any channels that look continuously noisy or contain no signal throughout the entirety/majority of the task recording.
    NOTE: For low-density recordings, the aim is to keep as many channels as possible. Thus, for channels that are bad only for short periods of time, it is preferable to remove the corrupted data segment at a later stage of the analysis rather than the channel itself.
  2. Plot and visually inspect the Power Spectral Density (PSD) to identify outlying channels.
3. To remove slow (over 2 s) linear trends, high-pass filter the data using a Basic FIR high-pass filter with a cut-off frequency of 0.5 Hz.
4. To remove high-frequency noise from myogenic and external sources, low-pass filter the data using a Basic FIR low-pass filter with a cut-off frequency of 35 Hz.
5. Segment the data into consecutive, non-overlapping 1 s epochs.
6. Automatically reject all segments with the minimum value below -100 µV (for adults) and -150 (for infants) and/or the maximum value above +100 µV (for adults) and +150 (for infants).
7. Visually inspect all segments that have not been excluded in 11.3.6. Manually reject all segments containing artifacts. If only a single 1s segment is acceptable located between rejected segments, remove the entire period so that no single 1s segments are retained.
8. For INS / dyadic analysis, analyze only accepted segments that are in common for adults and infants. By removing all segments of the adult that are rejected in the infant and vice versa, ensure that the EEG time series of adult and infant remain perfectly aligned.

Representative Results

Participants included in this study were 8- to 12-month-old, typically developing infants and their mother and/or grandmother who spoke English or English and a second language at home. The 7-electrode EEGs and a single-lead ECG of adults and infants, as well as video and audio recordings from three cameras and microphones, were acquired simultaneously during the tasks. Neural activities were measured over F3, F4, C3, Cz, C4, P3, and P4 according to the international 10-20 system. The different data streams were temporally aligned and cut at the beginning and end of the experiment so that all recordings started at timepoint t = 0 (Figure 4).

Figure 4: Synchronization of data streams. Three cameras (infant-view, combined-view, and caregiver-view), caregiver and infant raw ECG, as well as caregiver and infant raw EEG, are synchronized to the same timeline. Please click here to view a larger version of this figure.

EEG data retention rates and quality metrics for the first 5 dyads of the data set with a total of 10 participants are presented in Table 1. After bad channel rejection (Figure 5), data segments containing artifacts were rejected using an automated voltage threshold followed by visual inspection of the remaining segments (Figure 6). Results showed that EEG recordings had an average length of M = 562.96 s (SD ± 148.94 s). From these, M = 34.30% (SD ± 13.00%) of the adult data and M = 46.32% (SD ± 16.63%) of the infant data were accepted following automatic and manual rejection. If considering only matched data between adult and infant, the retention rate dropped to M = 20.58% (SD ± 9.51%), leaving M = 215.00 s (SD ± 117.54 s) of matched task data. Further, a total of 0 to 2 channels per dyad were excluded due to consistently poor data quality.

Figure 5: Identifying bad channels. An infant EEG data scroll and Power Spectral Density (PSD) plot for 7 EEG channels in which either channel Cz is observed to be a flatline (A: data scroll, B: PSD plot), or channel F3 is excessively noisy (C: data scroll, D: PSD plot). Bad channel detection was performed in EEGLAB¹⁷. Please click here to view a larger version of this figure.

Figure 6: Artifact rejection. Epochs with artifacts were rejected automatically according to a (A) rejection threshold, (B) followed by manual rejection through visual inspection. The highlighted portion of the plot shows rejected segments according to the rejection threshold (A) or manual rejection (B), respectively. Data were visualized in EEGLAB¹⁷. Please click here to view a larger version of this figure.

		Adult		Child		Matched
ID	Recording length (s)	Bad channels	% accepted epochs	Bad channels	% accepted epochs	Bad channels	% accepted epochs (matched)
1	898	NA	35.7	Cz	25.2	Cz	15.3
2	1234	NA	38.2	Cz, F3	61.8	Cz, F3	21.2
3	1088	F3, F4	52.4	F3, F4	63.1	F3, F4	36.7
4	873	NA	27.9	P3	34.6	P3	12.8
5	975	NA	17.2	NA	47.0	NA	16.9

Table 1. EEG data quality report for 5 dyads during the experimental tasks.

Discussion

In this protocol, we conduct measurements in the participants' homes where infants and caregivers may feel more comfortable and their behaviors may be more representative of their real-life interactions as opposed to a laboratory setting, thus, increasing ecological validity⁷. Further, recordings in the home environment may ease the burden on the participants, e.g., with respect to travel times, and may thus make certain participant groups more accessible. However, along with these advantages, naturalistic EEG hyperscanning recordings in real-world contexts pose their own set of challenges and limitations with regard to experimental design and protocol as well as data artifacts. In the following, challenges and possible solutions for home recordings are discussed.

The naturalistic environment may introduce a set of confounding variables such as space, temperature, and interruptions, which may differ between participant groups at home but stay constant in a controlled laboratory setting. The EEG hyperscanning protocol requires a lot of technical equipment, e.g., several cameras, microphones, and recording laptops, and therefore, lack of sufficient space in participant homes may sometimes be an issue. Researchers must be aware not to set up equipment haphazardly or somewhere surrounded by clutter. For example, it is important to be mindful not to set up devices on tables with food or drink items and to make sure camera tripods are not blocking the way in narrow spaces. One way to prevent issues with space would be to visit the participant's home beforehand to appropriately plan ahead of time for any space constraints. It is also helpful to send reminders to the participants to have the required space cleared of items. Cameras and tripods should be placed out of the way as much as possible, especially when out of reach from where the infant is sitting during the session. Most of all, all parties' safety must be considered at all stages of the set-up. Another factor that researchers may encounter in naturalistic settings is varying temperatures. In Singapore, where temperatures are high throughout the day and year, sweat artifacts may occur in the EEG data, which can be better controlled in the laboratory environment with appropriate air conditioning. Using fans to keep participants cool also introduces other artifacts due to having electrical appliances in proximity, and the blowing air may move participants' hair, as well as the EEG wires, resulting in poor data quality. Ideally, air conditioning should be used during the session as it will keep participants cool. Still, if this is not possible, an overhead fan or standing fan may be used instead while making sure it is not placed too close to the participants to avoid creating noise in the EEG data. Other alternatives would be to schedule the session during a cooler time of the day if possible so that sweat artifacts can be avoided. Finally, researchers also need to be wary that interruptions may occur in a naturalistic setting, especially if conducting the session at participant's homes. Family members may be in the vicinity, which can cause a violation of privacy when filming the session in a common room where they may be walking by. It can also be a distraction for the infant to see other caregivers or family members during the task, which may bias the EEG measurements. It would be best to remind participants that for the session to run smoothly, it would be ideal to have other family members in a different room. Researchers can also try to conduct the session as efficiently as possible so as not to inconvenience the other members of the household too much. Lastly, researchers must ensure that all data is collected and that the necessary items are completed before leaving the participant's home. Having a clear and organized checklist of documents and items to be completed can help avoid missing any important steps and also help complete them efficiently and in a timely manner.

Apart from the confounding variables found in a naturalistic environment, there are also some aspects of the protocol that will need to be adjusted for each session in a natural setting that are otherwise controlled in a laboratory environment. Standardization will not be possible for certain aspects, such as camera angles and lighting. Flexibility in set-up while also ensuring high-quality and comparable data is crucial. Camera angles may change with each participant's home as a result of differences in the layout and space, which may make it more difficult for later annotations of videos for specific events and behavioral metrics. Similarly, the lighting will also differ in each home, which can affect the quality of the video. Researchers can be adequately prepared by creating a general set of standards that can be adapted, such as making sure the participants are not seated against a main source of light and knowing what camera angles to prioritize. Another varying factor would be the furniture available to use in each session. Since researchers most likely cannot bring furniture to the participants' homes, they will have to rely on furniture that the participants already have. Due to this, the different furniture used can change the physical dynamic between the caregiver and the infant. For example, various kinds of baby chairs will change the height and position at which the infant is seated during the task. This may affect the way the caregiver interacts with the child and also affect the EEG data due to potential muscle movement artifacts or other factors. During the preprocessing stage of data analysis, researchers may be able to identify the EEG artifacts caused by specific movements by seeking guidance from the synchronized videos. Furthermore, having a general idea of what kinds of behaviors are going to be observed or analyzed can help to ensure that the necessary data is captured despite the varying physical dynamics.

A further implication of the home-environment naturalistic setup of EEG experiments concerns the quality and usability of physiological sensor data. EEG recordings are prone to artifact interference from environmental (non-physiological, such as line noise¹⁸) and physiological sources (ocular, sweat, myogenic)¹⁹. Although wireless EEG is generally less vulnerable to line noise, electrical devices in the home, e.g., fans, TV screens, and aircon, will introduce noise artifacts. Movement artifacts, on the other hand, are even more prominent in a naturalistic setting and contribute to lower data retention¹¹^,²⁰, reduction in signal-to-noise ratio²¹, and vulnerability in data analysis in interpretation¹¹. Dyadic EEG and infant EEG present an additional challenge in data retention due to lower recording durations, less stereotypical artifact presentations, and, in the case of hyperscanning, the necessity of clean analyzable segments to be matched in time¹⁴^,²²^,²³. Mitigation of these factors relies on thoughtful experimental design and well-calibrated experimental setup²². Although high-density EEG compositions allow for some artifact correction and data augmentation techniques, such as independent component analysis (ICA) removal of canonical noise components, this is not recommended with low-density setups. In contrast, relying on hand annotation of artifacts and removal of affected EEG channels and segments leads to greater data loss. The proposed protocol can also be performed with more EEG channels but at the cost of a longer preparation time. These advantages of shorter acquisition time versus richer EEG data must be carefully weighed against each other. Here, a realistic estimate of data retention rates from the naturalistic home recordings is reported, adhering to strict quality standards using a combination of automated voltage spikes labeling and manual artifact rejection. Although the retention rates were low (M = 34% for adults and M = 46% for infants), they are within the excepted range for naturalistic infant-adult EEG recordings, e.g., as a comparison, Dikker et al.¹² reported a retention rate of 38% during the discussion task in adult EEG using dry electrodes. The amount of clean data recovered from the paradigm can be fed into further analyses, such as time-frequency-based connectivity analyses. Alternative semi-automated pipelines for artifact correction of low-density EEG recordings (e.g., HAPPILEE²⁴), albeit out of the scope of the current paper, may help remove artifacts without the use of ICA and thus significantly reduce data loss.

To ensure high-quality EEG but feasible data collection, researchers will need to consider how the naturalistic setting affects the tasks that are chosen for the experimental session. For example, the choice of tasks can be based on what would be commonly found in participant homes, such as a dining table, chairs, baby chairs, playmat, etc. This would allow for less bulky equipment or furniture that needs to be transported back and forth and would also reduce the set-up and clean-up time. In this experiment, books and toys that are suitable for tabletop play were used, allowing caregiver and child to maintain a naturalistic play dynamic while also limiting free movement so that muscle movement EEG artifacts can be reduced. As a result, in the current protocol, the toys were chosen based on what would reflect natural interactions. For example, toys with suction that can be placed in a stationary position for the caregiver and child to engage with on the table have the advantage that they cannot fall off the table, which may cause motion artifacts when the caregiver tries to pick them up. Researchers also need to be wary of preparation and clean-up time to reduce participant burden.

Although choosing to conduct EEG hyperscanning measurements in a naturalistic environment has many benefits for more ecologically valid data, researchers should be aware of the limitations and challenges that may arise from the experimental design and implement sufficient steps to mitigate the effects as much as possible. Researchers must strive to strike a balance between an ecological design and experimental control when optimizing their paradigm and planning their visits. As described above, some flexibility with respect to the experimental set-up is needed, which, however, introduces more variability between participants. While this is undesirable from an experimental perspective, it may be more reflective of the participants' real-world environments. Additionally, the naturalistic setup may introduce more and other types of artifacts to the EEG data, as discussed above. These can, to some extent, be mitigated by appropriate EEG preprocessing and analysis techniques but generally can lead to a higher loss and lower quality of data. Further, the equipment used, in particular the cameras and tripods, comes with the disadvantages of being relatively bulky and heavy, thus making it difficult to transport and less suitable for confined spaces. Finally, the wet electrode system needs additional experimental materials (e.g., gel, syringes, gloves, wipes) and longer preparation times. Experimenters must be very careful not to leave a mess in the participants' homes, e.g., getting gel on parts of the furniture, and explain in advance that there is a risk that the infant may do so. Dry electrodes can be a good alternative to circumvent these issues and save set-up time. Thus, for hyperscanning recordings in larger groups (e.g., classrooms), these may be the method of choice (e.g., see ¹²). Therefore, by refining and adapting this protocol to the circumstances at hand, it has the potential to be applied in many different types of naturalistic settings, such as schools and workplaces, to capture a larger variety of hyperscanning and behavioral data.

Disclosures

The authors have nothing to disclose.

Acknowledgements

The work was funded by a Presidential Postdoctoral Fellowship Grant from Nanyang Technological University that was awarded to VR.

Materials

10 cc Luer Lock Tip syringe without Needle	Terumo Corporation
actiCAP slim 8-channel electrode set (LiveAMP8)	Brain Products GmbH
Arduino Software (IDE)	Arduino	Arduino IDE 1.8.19	The software used to write the code for the Arduino microcontroller. Alternate programming software may be used to accompany the chosen microcontroller unit.
Arduino Uno board	Arduino		Used for building the circuit of the trigger box. Alternate microcontroller boards may be used.
BNC connectors			BNC connectors to connect the various parts of the trigger box setup.
BNC Push button	Brain Products GmbH	BP-345-9000	BNC trigger push button to send triggers.
BNC to 2.5 mm jack trigger cable (80 cm)	Brain Products GmbH	BP-245-1200	BNC cables connecting the 2 LiveAmps to the trigger box.
BrainVision Analyzer Version 2.2.0.7383	Brain Products GmbH		EEG analysis software.
BrainVision Recorder License with dongle	Brain Products GmbH	S-BP-170-3000
BrainVision Recorder Version 1.23.0003	Brain Products GmbH		EEG recording software.
Custom 8Ch LiveAmp Cap passive (infant EEG caps)	Brain Products GmbH	LC-X6-SAHS-44, LC-X6-SAHS-46, LC-X6-SAHS-48	For infant head sizes 44, 46, 48 . Alternate EEG caps may be used.
Dell Latitude 3520 Laptops	Dell		Two laptops, one for adult EEG recording and one for infant EEG recording. Alternate computers may be used.
Dental Irrigation Syringes
LiveAmp 8-CH wireless amplifier	BrainProducts GmbH	BP-200-3020	Two LiveAmps, one for adult EEG and one for infant EEG. Alternate amplifier may be used.
Manfrotto MT190X3 Tripod with 128RC Micro Fluid Video Head	Manfrotto	MT190X3	Alternate tripods may be used.
Matlab Software	The MathWorks, Inc.	R2023a	Alternate analysis and presentation software may be used.
Power bank (10000 mAh)	Philips	DLP6715NB/69	Alternate power banks may be used.
Raw EEG caps	EASYCAP GmbH		For Adult head sizes 52, 54, 56, 58. Alternate EEG caps may be used.
Rode Wireless Go II Single Set	Røde Microphones		Alternate microphones may be used.
Sony FDR-AX700 Camcorder	Sony	FDR-AX700	Alternate camcorders or webcams may be used.
SuperVisc High-Viscosity Gel	EASYCAP GmbH	NS-7907

References

Hari, R., Henriksson, L., Malinen, S., Parkkonen, L. Centrality of social interaction in human brain function. Neuron. 88 (1), 181-193 (2015).
Endevelt-Shapira, Y., Djalovski, A., Dumas, G., Feldman, R. Maternal chemosignals enhance infant-adult brain-to-brain synchrony. Sci Adv. 7 (50), (2021).
Santamaria, L., et al. Emotional valence modulates the topology of the parent-infant inter-brain network. NeuroImage. 207, 116341 (2020).
Nguyen, T., et al. The effects of interaction quality on neural synchrony during mother-child problem solving. Cortex. 124, 235-249 (2020).
Reindl, V., Gerloff, C., Scharke, W., Konrad, K. Brain-to-brain synchrony in parent-child dyads and the relationship with emotion regulation revealed by fNIRS-based hyperscanning. NeuroImage. 178, 493-502 (2018).
Reindl, V., et al. Conducting hyperscanning experiments with functional near-infrared spectroscopy. J Vis Exp. (143), e58807 (2019).
Gardner, F. Methodological issues in the direct observation of parent-child interaction: Do observational findings reflect the natural behavior of participants. Clin Child Fam Psychol Rev. 3, 185-198 (2000).
Xu, J., Zhong, B. Review on portable EEG technology in educational research. Comput Hum Behav. 81, 340-349 (2018).
Troller-Renfree, S. V., et al. Feasibility of assessing brain activity using mobile, in-home collection of electroencephalography: methods and analysis. Dev Psychobiol. 63 (6), e22128 (2021).
Bögels, S., Levinson, S. C. The brain behind the response: Insights into turn-taking in conversation from neuroimaging. Res Lang Soc. 50 (1), 71-89 (2017).
Georgieva, S., et al. Toward the understanding of topographical and spectral signatures of infant movement artifacts in naturalistic EEG. Front Neurosci. 14, 452947 (2020).
Dikker, S., et al. Brain-to-brain synchrony tracks real-world dynamic group interactions in the classroom. Curr Biol. 27 (9), 1375-1380 (2017).
Oliveira, A. S., Bryan, R. S., Hairston, W. D., Peter, K., Daniel, P. F. Proposing metrics for benchmarking novel EEG technologies towards real-world measurements. Front Hum Neurosci. 10, 188 (2016).
Turk, E., Endevelt-Shapira, Y., Feldman, R., vanden Heuvel, M. I., Levy, J. Brains in sync: Practical guideline for parent-infant EEG during natural interaction. Front Psychol. 13, 833112 (2022).
Kayhan, E., et al. A dual EEG pipeline for developmental hyperscanning studies. Dev Cogn Neurosci. 54, 101104 (2022).
Ayrolles, A., et al. HyPyP: a Hyperscanning Python pipeline for inter-brain connectivity analysis. Soc Cogn Affect Neurosci. 16 (1-2), 72-83 (2021).
Delorme, S., Makeig, S. EEGLAB: an open-source toolbox for analysis of single-trial EEG dynamics. J Neurosci Meth. 134, 9-21 (2004).
Nathan, K., Contreras-Vidal, J. L. Negligible motion artifacts in scalp electroencephalography (EEG) during treadmill walking. Front Hum Neurosci. 9, 708 (2016).
Stone, D. B., Tamburro, G., Fiedler, P., Haueisen, J., Comani, S. Automatic removal of physiological artifacts in EEG: The optimized fingerprint method for sports science applications. Front Hum Neurosci. 12, 96 (2018).
Noreika, V., Georgieva, S., Wass, S., Leong, V. 14 challenges and their solutions for conducting social neuroscience and longitudinal EEG research with infants. Infant Behav Dev. 58, 101393 (2020).
Ng, B., Reh, R. K., Mostafavi, S. A practical guide to applying machine learning to infant EEG data. Dev Cogn Neurosci. 54, 101096 (2022).
vander Velde, B., Junge, C. Limiting data loss in infant EEG: putting hunches to the test. Dev Cogn Neurosci. 45, 100809 (2020).
Bell, M. A., Cuevas, K. Using EEG to study cognitive development: Issues and practices. J Cogn Dev. 13 (3), 281-294 (2012).
Lopez, K. L., et al. HAPPILEE: HAPPE in low electrode electroencephalography, a standardized pre-processing software for lower density recordings. NeuroImage. 260, 119390 (2022).

Play Video

PDF

DOI

DOWNLOAD MATERIALS LIST

Cite This Article

Ramanarayanan, V., Oon, Q. C., Devarajan, A. V., Georgieva, S., Reindl, V. Home-Based EEG Hyperscanning for Infant-Caregiver Social Interactions. J. Vis. Exp. (207), e66655, doi:10.3791/66655 (2024).