The study proposes an activation-match model to study how loneliness is mitigated when a lonely audience watches barrage videos of rational and emotional appeals. The protocol uses eye tracking to document duration and fixation, accounting for the degree of satisfaction when emotional needs are appeased by content and barrage.
Researchers usually theorize media exposure based on assumptions of legacy media. However, a new interactive video viewing format, in this case barrage video where viewers’ comments are overlaid over visual content, challenges past perspectives. This study proposes an activation and match satisfaction model to study viewing behaviors of lonely people and to challenge previous claims. It presents a protocol to examine the mechanism of how loners use barrage videos by combining eye tracking and self-report measures. Eye tracking documents the audience's conscious and subconscious watching behaviors in real time and allows for inference of the amount of allocated cognitive resources in response to rational and emotional content. The self-report gauges the amount of satisfaction obtained. Overall, results from the measures supported an activation and match satisfaction model regarding loners and their barrage video viewing behaviors. Implications are discussed.
Eye-tracking technique
The eye is often referred to as a window of the mind1,2. Eighty percent of human information intake is obtained visually3. Since the 19th century, people began to study human psychological activities by directly observing eye movements of participants. Miles invented the peephole to make observations when participants read4. In the study, the experimenter sat opposite a participant and observed the participant’s eye movements through a small hole in the middle of the reading material. Since then, technology has vastly improved. Currently, state-of-the-art eye-movement tracking devices mainly focus on electric current-recording, magnetic-induction and optical-recording, which includes corneal reflection and iris-scleral reflection methods5,6,7. The noninvasive features widely prevalent today make eye movement recording more natural, enhancing ecological validity. Today, eye movement techniques generally refer to the use of computer-controlled eye tracking to record and analyze the positioning of the participant's eye and the forms of eye movements during viewing of visual material.
Many theoretical perspectives regarding eye movements have matured over the years. These include the vision buffer processing model, the parafoveally processing model, the E-Z Reader model, the immediately processing model and the eye-mind processing model8,9,10. The immediately processing model holds that the processing of viewing content at all levels is not delayed, but occurs in real-time. The eye-mind processing model focuses on text information and holds that as long as one is processing a word, one would look at it. Put in another way, the word that one processes is exactly the very word one is looking at. The processing time of a word is the total fixation time of the participant’s eye.
There are three basic types of human eye movements: fixation, saccades and pursuit movements8,11,12. The fixation duration and fixation count usually reflect the extent to which the participant exerts cognitive resources to the content being viewed. Saccade refers to the movement from one gaze point to another. Retrospective saccade can be used as an indicator of the processing sophistication in the encoding process. Regression saccade indicates a deeper processing of an area after the first gaze of the key areas, reflecting the difficulty or interest of the content in that area. Pursuit movements are usually performed when there is visual noise, and the eye seeks out a point of interest.
On the other hand, we can also measure pupil size and blink frequency; both reflect people’s psychological activities13,14,15. For example, there is a relationship between pupil size and the specific task difficulty, motivation, interest, attitude, and fatigue. At present, the relationship between pupil size and emotional valence is not clear16. However, if a researcher combines pupil size with other indicators, such as electroencephalogram (EEG), the accuracy would be greatly improved17. For blink indicators, according to the hedonic-blink hypothesis, the decrease of blink frequency is usually associated with happy emotional thoughts, while the increase of blink frequency is associated with unhappy emotional states18.
The application of eye-tracking technology is extremely broad, including reading strategies, visual information processing, compulsive behaviors, to even artistic intentions. The application in the field of reading is the most mature. In communication, eye-tracking is useful in news consumption studies and advertising effectiveness research. For example, a large number of eye movement experiments have explored both exogenous and endogenous factors in advertising19,20, with the former exploring the physical characteristics of advertisements such as size, pattern, color, position, originality and repeated presentations21,22, and the latter exploring individual factors such as product involvement, product motivation, prior knowledge and brand familiarity23,24,25,26,27.
In addition, eye tracking technology is widely used in many other fields, such as human-computer interaction and usability research28,29,30,31; skills transfer32; infant and child development research33,34,35,36; marketing and online consumer behavior research37,38,39; and packaging design40,41, among others.
In addition to being used alone, eye tracking is often combined with other multimodal measurement techniques. For example, researchers can combine eye movement data with other physiological indicators such as EEG, skin electrical response, heart rate, skin temperature, facial expression, etc. In this way, users' emotional responses to different kinds of information can be studied more effectively42.
Eye movement technology can also be integrated into other technologies. For example, it is being used with augmented reality technology30. At present, the integration of eye movement technology and virtual reality (VR) technology is worthy of attention. On the one hand, such integration can promote the rapid development of VR related equipment. For example, the graphics processing unit (GPU) of most VR equipment is overburdened and consumes a lot of energy. Eye tracking technology can detect the audience's fixation point in real time. VR devices only need to focus on rendering this area and ignore other areas. This can significantly reduce power consumption and GPU rendering load. On the other hand, the combination can enhance its interaction features, and improve the immersion and involvement of VR users. For example, players can use eye movement instead of hands to complete operations in a game43. In addition, researchers can implement eye movement testing in a simulated environment. For example, in consumer behavior research, researchers do not need to take subjects to a real shopping mall or use real products, but test them in virtual research scenarios, as shown in Figure 1.
Figure 1: Virtual research scenario Please click here to view a larger version of this figure.
Barrage video
Barrage is a term originating from the military. In order to intercept moving targets, multiple artillery guns are used to fire and to form a curtain composed of high-density projectiles before the moving targets44. The term "barrage" is used in video viewing to describe a format and a phenomenon in which audience members of a particular video comment and displaying such comments on the video screen while watching a video, thus forming a comment wall curtain, as shown in Figure 2. This format of interacting and commenting is also referred to as overlaid comments45,46. Barrage videos first appeared on the "niconico" video website in Japan. The most popular websites in China are AcFun (Station A, www.acfun.cn) and Bilibili (Station B, www.bilibili.com). Many other streaming video sites in China, such as Youku, Tencent, LeTV and iQiyi, have also added the barrage sending and viewing functions. Barrage videos have attracted many Chinese viewers, and the attention of researchers in many disciplines.
Figure 2: Comment wall curtain Please click here to view a larger version of this figure.
Viewers of barrage videos can watch and interact with others by commenting to possibly meet a few psychological needs. The audience of such videos may share similarities in age, background and interests, so they may develop a sense of identity and belonging. There is research, for example, that shows AcFun and Bilibili are home to the “quadratic element” genre enthusiasts, with its audience being mainly between 17 and 25 years old47,48. These two websites have set up an extremely strict membership review system in that membership is only granted after acing an entrance exam by answering questions such as "What is the height of the heroine of the Thousands of Defenders in Hurricane Butler?" "How many cruisers have been dispatched by Gonon in the Battle of Rum?" Such criteria basically weed out most of the "heterogeneous" audience who want to "invade" the group.
Related to this study, barrage video creates a sense of crowd watching, which is particularly meaningful for lonely individuals. Because the content of the barrage is highly related to the plot, and comments are immediate responses, it gives the audience an illusion of watching with others even though they may be watching alone physically. This sense of companionship has been shown to alleviate loneliness49.
Barrage videos can also afford entertainment of a different kind in that viewers are not only consuming but also creating video content by blending sometime serious film making with fun plays. Barrage viewers can also find a sanctuary away from reality50,51 in which they can vent their anxiety and engage in emotional catharsis in a safe environment52, or showcase their personalities, demonstrate narcissism by getting others’ attention, and even bypass conformist norms in the real world53.
Viewing mechanism of lonely audience on barrage video
Barrage video viewing is an ideal platform to study media use and loneliness for its emotional support afforded by the venue. In this study, the researchers find previous conceptualizations of media exposure to be inadequate and therefore offer an activation and match satisfaction model (AMSM) to explain the psychological underpinnings of viewing barrage, especially by loners. In previous research, there are two perspectives that explain media use. The deficiency paradigm holds that loners, for lack of companionship, would devote more cognitive resources to the barrage content during watching, to seek company and to compensate loneliness. The global-use paradigm maintains that media use is prevalent, and that it satisfies generic and interpersonal needs in general. So regardless of emotional state, all viewers would pay attention to barrage, and they obtain different gratifications54. However, AMSM posits that emotional content serves to activate lonely audiences' interpersonal needs, and they would take initiatives to search for interactive and interpersonal communication elements, such as barrage content in the viewing process, and devote more attention to such components. The degree that these elements satisfy their emotional needs determines the degree of satisfaction they obtain.
In order to understand the mechanism of barrage video watching by lonely people, one needs to know the amount of cognitive resources that loners invest in different media content and how they satisfy their needs. However, such data cannot be reliably obtained by traditional participative reporting methods. Cognitive resources allocation operates consciously and subconsciously. It is a tall order that audience members articulate to which parts of content they invest more cognitive resources. To accomplish this, a suitable research methodology is needed to record the viewing process, and to distinguish the amount of attention to different parts of content, in addition to measuring the corresponding satisfaction from the viewing process.
For these reasons, this project tracked participants’ eye movements, as measurements of attention and the degree of cognitive resources allocated. The follow-up Likert scale questions were designed to measure participants' degree of satisfaction from exposure. Eye-tracking is a noninvasive technology that has high temporal and spatial resolutions, allowing recording while participants process continuous visual stimuli without distraction55,56. In this study, duration time and fixation counts are used as measures of attention. Duration time refers to the length of attention, and fixation count refers to the number of gazes at a particular area of the video material. Both of these eye movement indicators have been shown to be valid measures of processing thoroughness, reflecting the cognitive resources allocated by individuals57,58. Results of gaze probability, for example, allow researchers to infer attributes in the video that were important to the participant. For self-reported measures, the researchers use a 7-point Likert scale, asked and answered immediately after watching.
Based on arguments from previous explicated perspectives, the researchers designed a 2 (audience type) x 2 (ad appeal) x 2 (barrage) mixed experimental study and hypothesized the amount of attention that normal and lonely participants would pay to barrage video. Audience type (lonely and normal) was a between factor. Ad appeal had two levels, with either emotional ads or rational ads. Barrage also had two levels, denoting video that either had barrage or not. The last two were within subject factors. The general hypotheses were that lonely audience would pay more attention to emotional ads than to rational ads, and they would pay more attention to barrage than to non-barrage, whereas for normal audience, there was no such difference. Satisfaction ratings followed the same patterns. The Chen et al.59 original paper detailed all these hypotheses.
This protocol adheres to the Jinan University research guidelines. As only the medical school of the university has an IRB board, no other discipline is required to have IRB approvals. However, the researchers confirm that all ethical rules and regulations were followed. The project did not pose any physical or psychological harm to participants.
1. Participant screening for the experiment
2. Stimuli construction
3. Eye tracking protocol
Figure 3: Experimental flowchart Please click here to view a larger version of this figure.
Repeated measures MANOVAs were conducted using duration and fixation as dependent variables, which indicated attention. Results confirmed proposed hypotheses that lonely participants’ gaze stayed on barrage longer than on non-barrage areas when the emotional ads were present. However, when the rational ads were viewed, there was no difference. Such data pattern did not replicate for the low loneliness participants, whose attention remained statistically non-significant when watching emotional ads and rational ads. That is, there was no difference between the gaze probability to the barrage area of interest and the gaze probability of the non-barrage area of interest. The patterns of eye movement data were only in line with the expectations of AMSM. Both duration and fixation count produced significant statistical results59. See Figure 4, credited to CHB, below for the representative fixation count results.
Figure 4: Percentage of fixation count in the barrage Please click here to view a larger version of this figure.
For the significant interaction of loneliness, ad appeal, and barrage, pairwise comparisons were computed. The results showed that lonely subjects gazed at the emotional/barrage area the most, whereas a normal audience exhibited no gaze differences within either emotional or barrage condition.
Self-reported data and analyses
The participants’ satisfaction level was measured by using established satisfaction items from previous research63,64. Satisfaction with watched videos largely replicated the duration and fixation results, and as proposed by AMSM, and as shown in Table 1, credited to CHB59. Results showed that a lonely audience was more satisfied watching emotional ads than rational ads, and with barrage than non-barrage, whereas for a normal audience, there were no statistical differences for both interactions.
Evaluation results of advertising videos (M ± SD) | ||||
Story-base ads | Hard-selling ads | |||
Audience type | Barrage video | Non-barrage video | Barrage video | Non-barrage video |
Loneliness | 4.84±0.69 | 4.45±0.73 | 4.30±0.87 | 4.30±1.06 |
Normal | 4.75±0.67 | 4.62±0.80 | 4.14±0.79 | 4.21±0.87 |
Note: revised from Chen & Zhou (2019) published in CHB |
Table 1: Evaluation results of advertising videos
Video 1: Sample barrage ad Please click here to download this video.
In this study, eye tracking technology and self-reports are combined to test the validity of the proposed AMSM model. Previous studies mainly used self-reports to explore the relationship between loneliness and media use at the mercy of participants’ ability to articulate. These offline methods fail to understand the psychological processes while watching barrage videos. In this study, eye tracking technology was used to record duration and fixation so researchers could reliably infer how much cognitive resources the participants invested in different video content. The technology is noninvasive, so participants watched the videos without interruption. Results supported the proposed AMSM.
Specifically, this study provided evidence that lonely people gravitated toward emotional content. When emotional content was watched and being discussed (i.e., with barrage), it was more attention getting as measured in gaze duration and fixation counts. In contract, data did not show such effects on people who were not lonely. Equally important, satisfaction evaluation after exposure showed that lonely people were more satisfied with emotional, barrage video, whereas no such conclusion could be drawn for people low in loneliness.
Theoretically, data from this study did not support traditional models of media consumption, claiming either that people used media for the sake of compensating emotional needs, according to the deficiency model, or viewers consumed for broader, more general purposes such as learning and diversion, according to the general use model. Instead, data supported the proposed AMSM model, which posited that emotional content activated viewers’ affective needs, encouraging them to seek out interactive features such as barrage in media to fulfill such needs.
Methodologically, even though eye tracking has been used in communication in a variety of areas such as news consumption and advertising effectiveness, most of these studies explore physical attributes of media such as story placement, layout designs, and product features to explore attention-getting attributes. This study used eye-tracking creatively in the area of affect, by measuring personality variables related to loneliness, and manipulating media variables of emotional content and interactive barrage. It is the researchers’ belief that eye tracking did not have to focus on the obvious, it can also be used to investigate psychological processes. The new barrage format also offers a new and interesting arena for research.
However, this study also has some limitations. For the sake of manipulation convenience, ads were used instead of the usual entertainment content for barrage. Normally, barrage viewers were highly involved in story plots to engage in barrage viewing and participation. Viewing a number of ads seemed artificial. Also, because barrage was pre-produced before the experiment, participants of this study did not have a chance to really engage in barrage activities. As such, the activation process as proposed in AMSA was not specifically investigated. Future research may want to focus on the before and after emotional states to infer whether activation occurs. Also in this study, barrage appeared in the top part of the screen and its crawling direction was predetermined to be from left to right. In real life, barrage texts appear in different styles. Future research may need to allow natural barrage activity to increase external validity.
In sum, the combination of this approach—objective eye tracking data and subjective self-reporting data—allows the researchers to not only untangle the viewing mechanism of lonely on barrage videos, but also to identify the underlying mechanisms of attention allocation. Our hope is that it can serves as a starting point for more such studies.
The authors have nothing to disclose.
This study was supported by the National Social Science Fund of China for the project (19ZDA332) entitled “Deep Convergence of Media and Innovation Model of Social Governance in the New Era;” a PhD Start-up Fund of the Natural Science Foundation of Guangdong Province, China (2017A030310536); and the Fundamental Research Funds for Central Universities (19JNQM04). The authors would like to thank Dean/Professor Zhi Tingrong for support and investment in the eye tracking laboratory.