Physiological Correlates of Emotion Recognition

Physiological measurements can be used to understand psychological functioning, revealing an interactive relationship between the body and mind.

How fast a person’s heart beats can change depend on their surroundings, and—by extension—their emotional state. For example, someone who senses a shadowy figure following them at night may demonstrate an elevated heart rate due to fear, as well as the physical exertion from running away.

In contrast, when the same individual enters a comfortable environment, such as a gathering at a friend’s house, relatively no danger—except maybe spilling wine on the white carpet—is perceived.

Interestingly, although the speed of the heart can change automatically in response to their own feelings, it can also be influenced by the emotions they distinguish in others—like terror in a man who just broke the host’s favorite lamp.

Here, the very observation of signs of dread in the face of the guilty party—wide eyes, bared teeth, and raised eyebrows—can briefly escalate the heart rate of the observer. Throughout all of these experiences, the individual displayed high heart rate variability, or HRV, representing effective regulation.

In this video, we explore how ranges in cardiovascular capacity—low to high HRV—can be used to predict an individual’s ability to recognize feelings in other people.

We demonstrate how to design and perform an experiment to analyze both heart rate and emotional recognition data.

In this experiment, participants are first outfitted with electrodes around the heart, asked to rest while baseline recordings are taken, and then shown pictures of human faces expressing different emotions, which they must identify as their cardiac signals are continuously recorded.

During the task, a small fixation cross is displayed in the center of a computer screen, followed by a picture of a single human face.

The trick is that this facial image can be of either a calm, neutral expression—for example, someone with a relaxed mouth, eyebrows, and forehead—or one of fear.

When each visual is presented, participants must indicate which feeling it depicts. The idea here is that the overall accuracy of their responses provides a measure of emotional recognition ability.

Every trial ends with the presentation of a final fixation symbol to clearly separate the dependent variables: emotional and cardiac responses across trials types that are shown equally but in random order.

While monitoring heart rate, emphasis is placed on a critical component of the cardiac cycle called the R wave, which corresponds to the large peak during every beat. Heart rate variability—the HRV—is calculated, corresponding to the time between each peak and how it varies over a period of time.

The calculations are processed through a number of mathematical operations, such as fast Fourier transformation, and result in frequency information. Within this whole power spectrum, a certain band, the high frequency range, is of interest since it reflects the state of autonomic regulation.

Specifically, high HRV means that a participant’s heart rate changes continuously, which suggests an index of successful control. In contrast, low HRV equates to consistent values and therefore an association with poor regulation.

Importantly, the HRV measures are compared to the emotion recognition results. Based on previous experience, it is expected that accuracy of identifying feelings will correlate with HRV.

In other words, a participant whose heart rate fluctuates as pictures of different emotions are shown is more likely to correctly distinguish them, suggesting that HRV is a physiological correlate of this identification process.

To begin, greet the participant when they arrive. Verify that they have normal vision, have not consumed any substances, such as caffeine or alcohol, in the last 6 hrs, which could affect their heart rate, and have no history of psychiatric or cardiac disorders.

Then, direct the participant to sit in front of a computer monitor and keyboard. Gather three pre-gelled electrodes, and check that they have not dried out, to ensure that electrical signals from the heart are properly conducted.

Now instruct the participant to affix these electrodes on their chest using an established configuration. Attach the leads, and confirm that they are properly connected to the equipment that will amplify and relay information to the recording computer.

At this point, in the software monitor the cardiac signal to ensure that it’s devoid of noise and artifacts. In addition, verify that the acquisition parameters are correct, and start collecting a resting heart rate for 5 min.

Proceed to explain that pictures of human faces will be shown on the monitor. Emphasize that the participant should quickly push 'F' on the keyboard if they interpret an image as being fearful. In contrast, if the face appears neutral and devoid of emotion, stress that the 'J' key must be used.

Also confirm that the very first fixation cross appears onscreen for 500 ms, and for each trial, the facial image for 200 ms, followed by the final fixation symbol for 2 s. Now, leave the room and continue to record their cardiac signal as they perform 120 emotion recognition trials.

With all of the heart-derived data collected, first navigate the graphical user interface and isolate the R-waves to calculate the time between these identified peaks.

Further process these measures to generate the power spectrum of frequencies, and in particular, record the values in the high-frequency range, which represents the degree of fluctuation within 0.15-0.4 Hz.

Finally, compute the emotion recognition ability of each individual. To do this, first import the data recorded from the key presses, and calculate the average percentage of fearful faces that were correctly identified. Then, repeat this process for neutral images.

To analyze the data, begin by creating a scatter plot where accuracy of fearful face identification is graphed on the y-axis, and HRV values on the x-axis. Include data from all participants in this figure. Then, create a similar plot for neutral face data.

In each, determine the line of best fit and the correlation coefficient. Importantly, the nearer the data points are to this line, the stronger the linear correlation is between emotion recognition and HRV.

Notice that data points were more tightly clustered around the best fit line in the fear recognition graph, compared to its neutral counterpart.

This suggests a positive correlation between HRV and an individual’s ability to distinguish fright; however, no correlation existed between HRV and participants’ capacity to recognize neutral faces, indicating that this relationship is emotion-specific.

Collectively, these results imply a link between the physiological phenomenon of heart rate and social behavior. Specifically, an individual with high HRV may be more likely to recognize emotions in others, and thus excel with social interactions.

Now that you understand how to explore the correlation between autonomic self-regulation and social behavior, let’s look at how researchers are investigating the variability of physiological arousal at a young age and in other cognitive and mental health contexts.

Up until now, we’ve focused on the identification of emotions in adults. However, similar work is being performed in children to investigate factors related to early cardiovascular variation that may predict later health conditions.

For example, researchers measured heart rate while subjecting toddlers to a number of behavioral conditions, including a lullaby during rest, an active jack-in-the-box, tasting sour lemon juice, and audio of an infant crying.

Such experiences resulted in different heart rate outcomes, suggesting that variability can be elicited in young children. Additional research is needed to assess the developmental trajectory of cardiovascular measures and ultimately predict future health outcomes.

In terms of the relationship between HRV and working memory, research has shown that a person with low HRV—whose heart beats at a continuous, even pace despite even frightening surroundings—is more likely to demonstrate poor working memory, and forget the address they are told.

Interestingly, repeated aerobic exercise—like running—can actually improve HRV and memory retention, emphasizing the relationship between body and mind.

Lastly, other work has looked at the whether there is an association between HRV and certain mental health disorders, such as depression.

Specifically, an individual with similarly low HRV may be more likely to exhibit depressive symptoms, like disinterest in activities or withdrawal from social settings. Thus, HRV may serve as a useful, non-invasive biomarker for this social disorder.

You’ve just watched JoVE’s video on the physiological correlates of emotion recognition. By now, you should know how to present socially relevant visual stimuli to patients, and collect and interpret both behavioral and HRV data. In addition, you should understand how scientists are applying HRV to additional contexts, such as how it relates to other cognitive processes like memory.

Thanks for watching!