This article demonstrates an experimental design in which whole-body animated characters are used in conjunction with functional magnetic resonance imaging (fMRI) to investigate the neural correlates of observing virtual social interactions.
The ability to gauge social interactions is crucial in the assessment of others’ intentions. Factors such as facial expressions and body language affect our decisions in personal and professional life alike 1. These “friend or foe” judgements are often based on first impressions, which in turn may affect our decisions to “approach or avoid“. Previous studies investigating the neural correlates of social cognition tended to use static facial stimuli 2. Here, we illustrate an experimental design in which whole-body animated characters were used in conjunction with functional magnetic resonance imaging (fMRI) recordings. Fifteen participants were presented with short movie-clips of guest-host interactions in a business setting, while fMRI data were recorded; at the end of each movie, participants also provided ratings of the host behaviour. This design mimics more closely real-life situations, and hence may contribute to better understanding of the neural mechanisms of social interactions in healthy behaviour, and to gaining insight into possible causes of deficits in social behaviour in such clinical conditions as social anxiety and autism 3.
1. Stimuli, Task Design, and Experimental Protocol
Our stimuli are created using Poser 7.0 (http://poser.smithmicro.com/poser.html), and they are presented using CIGAL (http://www.nitrc.org/projects/cigal/).
2. Preparing the Subject for the Scan
Subjects are typically recruited on the basis of their age, health, first language, and individual risk factors for MRI scanning, such as metallic joint replacements. However, depending of the purpose of the study, other factors may also be considered, including race/ethnic background, socio-economic status, and history of drug use. All subjects provide written informed consent prior to running the experimental protocol, which is approved by an Ethics Board.
Prior to Entering the scanning room
Entering the scanning room
3. Data Recording and Processing
Scanning Parameters
We collected MRI data using a 1.5 Tesla Siemens Sonata scanner for MRI recordings. Our anatomical images were 3D MPRAGE anatomical series (repetition time, TR = 1600 ms; echo time TE = 3.82 ms; number of slices = 112; voxel size = 1 x 1 x 1mm), and functional images consisted of series of 28 functional slices (voxel size = 4x4x4 mm), acquired axially using an echoplanar sequence (TR = 2000 ms; TE = 40 ms; field of view FOV = 256 x 256mm), thus allowing for full-brain coverage.
Data Analysis
We use Statistical Parametric Mapping (SPM2/SPM5) in combination with in-house Matlab-based tools. Pre-processing involves typical steps: quality assurance, TR alignment, motion correction, co-registration, normalization, and smoothing (8 mm³ kernel) 12.
4. Representative Results
Figure 1. Increased activity in the social cognition network in response to observing social interactions. Comparison of social interaction vs. no-interaction/control trials revealed activity in typical social cognition brain regions, including the superior temporal sulcus (STS, a), the lateral and medial prefrontal cortex (mPFC, b & d, respectively), and the amygdala (AMY, c). The “activation maps” are superimposed on high resolution brain images displayed in lateral (left- and right-side panels) and coronal (middle panel) views; the color bars indicate the gradient of t values of the activation maps (based on data from 15 participants), reflecting brain activity time-locked to the onsets of approach/avoid behaviours. The line graphs illustrate the time courses of the fMRI signal, extracted from functional ROIs for each trial type and TR (1 TR = 2 seconds). L = Left; R = Right.
The experimental design introduced here allows investigation of the neural correlates of observing and interpreting body language. This design has the potential to advance our knowledge concerning brain mechanisms involved in social interactions, and to extend theoretical models of how we combine perception of different types of body language or social concepts such as trustworthiness to make decisions in interactive social environments 3. Such knowledge can be applied in a variety of personal and business settings, and can improve our understanding of clinical deficits in social interaction. The success of this design depends on proper task manipulation, involvement of ecologically valid stimuli, and careful data collection
The authors have nothing to disclose.
This research was supported by start-up funds to FD. KS was supported by a summer studentship from the Alberta Heritage Foundation for Medical Research. FD was supported by a Young Investigator Award from the National Alliance for Research on Schizophrenia and Depression, and a CPRF Award from the Canadian Psychiatric Research Foundation. The authors wish to thank Peter Seres for assistance with data collection and Kristina Suen for assistance with data analysis.