Loneliness Assuaged: Eye-Tracking an Audience Watching Barrage Videos

Eye-tracking technique
The eye is often referred to as a window of the mind^1,2. Eighty percent of human information intake is obtained visually³. Since the 19th century, people began to study human psychological activities by directly observing eye movements of participants. Miles invented the peephole to make observations when participants read⁴. In the study, the experimenter sat opposite a participant and observed the participant’s eye movements through a small hole in the middle of the reading material. Since then, technology has vastly improved. Currently, state-of-the-art eye-movement tracking devices mainly focus on electric current-recording, magnetic-induction and optical-recording, which includes corneal reflection and iris-scleral reflection methods^5,6,7. The noninvasive features widely prevalent today make eye movement recording more natural, enhancing ecological validity. Today, eye movement techniques generally refer to the use of computer-controlled eye tracking to record and analyze the positioning of the participant's eye and the forms of eye movements during viewing of visual material.

Many theoretical perspectives regarding eye movements have matured over the years. These include the vision buffer processing model, the parafoveally processing model, the E-Z Reader model, the immediately processing model and the eye-mind processing model^8,9,10. The immediately processing model holds that the processing of viewing content at all levels is not delayed, but occurs in real-time. The eye-mind processing model focuses on text information and holds that as long as one is processing a word, one would look at it. Put in another way, the word that one processes is exactly the very word one is looking at. The processing time of a word is the total fixation time of the participant’s eye.

There are three basic types of human eye movements: fixation, saccades and pursuit movements^8,11,12. The fixation duration and fixation count usually reflect the extent to which the participant exerts cognitive resources to the content being viewed. Saccade refers to the movement from one gaze point to another. Retrospective saccade can be used as an indicator of the processing sophistication in the encoding process. Regression saccade indicates a deeper processing of an area after the first gaze of the key areas, reflecting the difficulty or interest of the content in that area. Pursuit movements are usually performed when there is visual noise, and the eye seeks out a point of interest.

On the other hand, we can also measure pupil size and blink frequency; both reflect people’s psychological activities^13,14,15. For example, there is a relationship between pupil size and the specific task difficulty, motivation, interest, attitude, and fatigue. At present, the relationship between pupil size and emotional valence is not clear¹⁶. However, if a researcher combines pupil size with other indicators, such as electroencephalogram (EEG), the accuracy would be greatly improved¹⁷. For blink indicators, according to the hedonic-blink hypothesis, the decrease of blink frequency is usually associated with happy emotional thoughts, while the increase of blink frequency is associated with unhappy emotional states¹⁸.

The application of eye-tracking technology is extremely broad, including reading strategies, visual information processing, compulsive behaviors, to even artistic intentions. The application in the field of reading is the most mature. In communication, eye-tracking is useful in news consumption studies and advertising effectiveness research. For example, a large number of eye movement experiments have explored both exogenous and endogenous factors in advertising^19,20, with the former exploring the physical characteristics of advertisements such as size, pattern, color, position, originality and repeated presentations^21,22, and the latter exploring individual factors such as product involvement, product motivation, prior knowledge and brand familiarity^{23,24,25,26,27}.

In addition, eye tracking technology is widely used in many other fields, such as human-computer interaction and usability research^28,29,30,31; skills transfer³²; infant and child development research^33,34,35,36; marketing and online consumer behavior research^37,38,39; and packaging design^40,41, among others.

In addition to being used alone, eye tracking is often combined with other multimodal measurement techniques. For example, researchers can combine eye movement data with other physiological indicators such as EEG, skin electrical response, heart rate, skin temperature, facial expression, etc. In this way, users' emotional responses to different kinds of information can be studied more effectively⁴².

Eye movement technology can also be integrated into other technologies. For example, it is being used with augmented reality technology³⁰. At present, the integration of eye movement technology and virtual reality (VR) technology is worthy of attention. On the one hand, such integration can promote the rapid development of VR related equipment. For example, the graphics processing unit (GPU) of most VR equipment is overburdened and consumes a lot of energy. Eye tracking technology can detect the audience's fixation point in real time. VR devices only need to focus on rendering this area and ignore other areas. This can significantly reduce power consumption and GPU rendering load. On the other hand, the combination can enhance its interaction features, and improve the immersion and involvement of VR users. For example, players can use eye movement instead of hands to complete operations in a game⁴³. In addition, researchers can implement eye movement testing in a simulated environment. For example, in consumer behavior research, researchers do not need to take subjects to a real shopping mall or use real products, but test them in virtual research scenarios, as shown in Figure 1.

Figure 1: Virtual research scenario Please click here to view a larger version of this figure.

Barrage video
Barrage is a term originating from the military. In order to intercept moving targets, multiple artillery guns are used to fire and to form a curtain composed of high-density projectiles before the moving targets⁴⁴. The term "barrage" is used in video viewing to describe a format and a phenomenon in which audience members of a particular video comment and displaying such comments on the video screen while watching a video, thus forming a comment wall curtain, as shown in Figure 2. This format of interacting and commenting is also referred to as overlaid comments^45,46. Barrage videos first appeared on the "niconico" video website in Japan. The most popular websites in China are AcFun (Station A, www.acfun.cn) and Bilibili (Station B, www.bilibili.com). Many other streaming video sites in China, such as Youku, Tencent, LeTV and iQiyi, have also added the barrage sending and viewing functions. Barrage videos have attracted many Chinese viewers, and the attention of researchers in many disciplines.

Figure 2: Comment wall curtain Please click here to view a larger version of this figure.

Viewers of barrage videos can watch and interact with others by commenting to possibly meet a few psychological needs. The audience of such videos may share similarities in age, background and interests, so they may develop a sense of identity and belonging. There is research, for example, that shows AcFun and Bilibili are home to the “quadratic element” genre enthusiasts, with its audience being mainly between 17 and 25 years old^47,48. These two websites have set up an extremely strict membership review system in that membership is only granted after acing an entrance exam by answering questions such as "What is the height of the heroine of the Thousands of Defenders in Hurricane Butler?" "How many cruisers have been dispatched by Gonon in the Battle of Rum?" Such criteria basically weed out most of the "heterogeneous" audience who want to "invade" the group.

Related to this study, barrage video creates a sense of crowd watching, which is particularly meaningful for lonely individuals. Because the content of the barrage is highly related to the plot, and comments are immediate responses, it gives the audience an illusion of watching with others even though they may be watching alone physically. This sense of companionship has been shown to alleviate loneliness⁴⁹.

Barrage videos can also afford entertainment of a different kind in that viewers are not only consuming but also creating video content by blending sometime serious film making with fun plays. Barrage viewers can also find a sanctuary away from reality^50,51 in which they can vent their anxiety and engage in emotional catharsis in a safe environment⁵², or showcase their personalities, demonstrate narcissism by getting others’ attention, and even bypass conformist norms in the real world⁵³.

Viewing mechanism of lonely audience on barrage video
Barrage video viewing is an ideal platform to study media use and loneliness for its emotional support afforded by the venue. In this study, the researchers find previous conceptualizations of media exposure to be inadequate and therefore offer an activation and match satisfaction model (AMSM) to explain the psychological underpinnings of viewing barrage, especially by loners. In previous research, there are two perspectives that explain media use. The deficiency paradigm holds that loners, for lack of companionship, would devote more cognitive resources to the barrage content during watching, to seek company and to compensate loneliness. The global-use paradigm maintains that media use is prevalent, and that it satisfies generic and interpersonal needs in general. So regardless of emotional state, all viewers would pay attention to barrage, and they obtain different gratifications⁵⁴. However, AMSM posits that emotional content serves to activate lonely audiences' interpersonal needs, and they would take initiatives to search for interactive and interpersonal communication elements, such as barrage content in the viewing process, and devote more attention to such components. The degree that these elements satisfy their emotional needs determines the degree of satisfaction they obtain.

In order to understand the mechanism of barrage video watching by lonely people, one needs to know the amount of cognitive resources that loners invest in different media content and how they satisfy their needs. However, such data cannot be reliably obtained by traditional participative reporting methods. Cognitive resources allocation operates consciously and subconsciously. It is a tall order that audience members articulate to which parts of content they invest more cognitive resources. To accomplish this, a suitable research methodology is needed to record the viewing process, and to distinguish the amount of attention to different parts of content, in addition to measuring the corresponding satisfaction from the viewing process.

For these reasons, this project tracked participants’ eye movements, as measurements of attention and the degree of cognitive resources allocated. The follow-up Likert scale questions were designed to measure participants' degree of satisfaction from exposure. Eye-tracking is a noninvasive technology that has high temporal and spatial resolutions, allowing recording while participants process continuous visual stimuli without distraction^55,56. In this study, duration time and fixation counts are used as measures of attention. Duration time refers to the length of attention, and fixation count refers to the number of gazes at a particular area of the video material. Both of these eye movement indicators have been shown to be valid measures of processing thoroughness, reflecting the cognitive resources allocated by individuals^57,58. Results of gaze probability, for example, allow researchers to infer attributes in the video that were important to the participant. For self-reported measures, the researchers use a 7-point Likert scale, asked and answered immediately after watching.

Based on arguments from previous explicated perspectives, the researchers designed a 2 (audience type) x 2 (ad appeal) x 2 (barrage) mixed experimental study and hypothesized the amount of attention that normal and lonely participants would pay to barrage video. Audience type (lonely and normal) was a between factor. Ad appeal had two levels, with either emotional ads or rational ads. Barrage also had two levels, denoting video that either had barrage or not. The last two were within subject factors. The general hypotheses were that lonely audience would pay more attention to emotional ads than to rational ads, and they would pay more attention to barrage than to non-barrage, whereas for normal audience, there was no such difference. Satisfaction ratings followed the same patterns. The Chen et al.⁵⁹ original paper detailed all these hypotheses.