We present detection, acquisition, and analysis of eyeblink rates while watching media content.
This article explores a method to detect differences in visual perception in humans. The method used is based on the psychological (or “cognitive”) function of eyeblinks. Participants’ eyeblinks are detected and acquired while watching videos specifically created for the investigation. The detection and acquisition of eyeblinks are carried out with the help of a 20-channel electroencephalographic (EEG) wireless device. The international 10-20 system for electrode placement is followed. A high-definition (HD) video camera is used to record participants’ facial expressions, for contrast purposes. Instead of using pre-existing media content, purpose-made video content has been created following specific criteria of interest for this investigation, with stimuli enabling researchers to manage the precise parameters of interest. Otherwise, results could be contaminated with uncontrolled variables. The synchronization of the presentation of video stimuli with EEG recordings needs to be done in milliseconds. Analysis of collected data is performed with robust software for working with big matrices. Statistically significant differences in eyeblink rate related to media professionalization and editing style are found with the reported experimental procedures.
The Purpose of This Method
This method proposes a dual protocol for detecting eyeblinks. The aim is to analyze viewers' visual perception of media content, specifically created for this investigation, by using EEG recordings and HD video recording systems.
The Rationale Behind the Development and/or Use of This Method
Each eyeblink hides visual flow for 150-400 ms1,2. Blinking has physiological3,4,5 and psychological6,7 functions. The connection between attention and eyeblink rate has been studied and proven in different studies8. A higher level of attention decreases eyeblink rate and according to previous studies, humans share a mechanism for controlling the timing of blinks that searches for the best moment to avoid loss of important visual information9. Thus, analyzing eyeblink behavior of viewers when watching screens could provide information regarding the level of attention given to media contents.
One method for detecting spontaneous eyeblink rate is by using EEG electrodes to record the electrical activity. Eyeblinks can easily be detected by prefrontal and electrooculogram electrodes connected to an EEG recording system. In most EEG analyses, eyeblinks are considered artifacts. For this reason, many software packages designed for analyzing EEG data have eyeblink detectors10. The advantage of using EEG for detecting eyeblinks is the high temporal resolution (in the order of milliseconds) and the possibility of registering brain effects of different narratives and cuts in movies synchronized with those eyeblinks – a matter open to further study. Recording participants' faces with an HD camera can also be useful for matching/contrast purposes9.
The Advantages over Alternative Methods with References to Relevant Studies
There are multiple methods for counting eye blinks. Some dedicated instruments for detecting blinks are magnetic coils, infrared (IR) light beams, optoelectronic motion detectors with eye movement analysis such as eye-tracking techniques, and several techniques based on bioelectrical signals, e.g., electrooculography (EOG), electromyography (EMG), and EEG. Another more accurate, but time-consuming option is manually counting blinks from a frame-by-frame video recording11. The technologies today can be classified broadly into two groups: a) contact-free recording which includes two modalities, the direct blink detection using computer vision and offline blink detection using eye-tracking, and b) contact-based recording using biological signals through EOG and EEG devices12,13.
The eye-tracking system is a widely used technology, ranging from traditional image-based passive designs to the active near-infrared-based approaches mainly used today with a high-resolution camera. The latter exploits the reflective properties of the pupil under IR illumination14. The concept underlying modern eye-tracking methods is Pupil Center Corneal Reflection (PCCR), which involves a camera tracking the center of the pupil, where light reflects from the cornea. However, there is a lack of blink detection algorithms published for eye-tracking protocols. Moreover, although the different models of eye-tracking on the market provide integrated software with blink detection, the source code is not always provided by manufacturers, making it difficult to modify or know how the algorithms work12. Also, during experiments with eye-tracking there are events that cause data loss, such as tracking delays and significant head or gaze movements. The eye area is very small in video captures, which is a problem for calculating the duration of the blink, and which sometimes introduces various types of artifacts15.
In this experiment, EEG and EOG methods are used. EEG is not usually used alone to detect eyeblinks. However, analyzing eyeblinks recorded with EEG electrodes is a standard procedure for the study of eyelid displacements. This procedure enables researchers to have information of exactly when eyeblinks take place. The most common signal pattern for detecting blinks is that of peak points, representing vertical movement responses. There are several peak detection algorithms applicable to raw EEG, time-domain, or frequency-domain signals. Processes involved in peak identification are peak detection, feature extraction, and classification. Eyeblinks have a considerable effect on frontal channels of the EEG signal. Typically, eyeblinks are detected in EEG by using a pre-determined amplitude threshold16. The algorithms in the analysis software used in this experiment are based on the signals' standard deviation and the root mean square (RMS) of the pre-filtered EEG signal; they are open source and available to the scientific community17. However, some eye movements not involving eyeblinks can provoke electrical activity that may be confusing. For that reason, a second method – recording viewers' faces with an HD video camera – allows researchers to match eyeblinks by manually counting them. With such a double method, the investigator achieves a matrix of eyeblinks that can be easily analyzed with statistical tools.
Therefore, the proposed method performs a data triangulation with two different sources to validate the detected eyeblinks. This method is based on Nakano et al. indications9 for confirmation. At the same time, it also enables researchers to collect brain-activity and frequency-band information for further analysis. The experiment described here is part of a wider future investigation into the effects of editing-style cuts on occipital and prefrontal brain areas.
Determine Whether the Method is Appropriate for an Investigation
This experimental protocol enables viewers' eyeblinks while watching video content to be studied under three experimental conditions. First, eyeblink rate is detected by using two complementary techniques: EEG and recorded HD videos. Here, we use a wireless EEG with 20 channels. Second, specific stimuli adapted to the experiment are created, so that the researcher can manage all the variables of the visual content. Here, three videos with the same narrative but different video-editing style were created. The narrative consisted of a man who entered a room, sat at a desk, juggled with three balls, opened a laptop, looked up information in some books, typed something on the laptop, closed it, ate an apple, looked directly into camera, and left the room. The three video stimuli last 198 s each. The first was a one-shot movie; the second was edited according to classical Hollywood-style rules with 33 different shots; and the third was edited following MTV-style rules with 79 shots. A fourth stimulus was also presented in which the narrative was identical, but the format was a real representation with an actor instead of a video. This fourth non-video stimulus was not used in an initial study of editing-style differences but was used in a different investigation to compare eyeblink-rate differences between real representation and screened media8. Third, different groups of participants are selected depending on their previous expertise in the visual analysis of videos. The purpose is to determine differences in eyeblink rates of subject groups watching the same visual stimuli. In this case, 40 subjects took part in the investigation. Half of them were media professionals (16 males and 4 females; age 30-56 years, with an average age of 44.15 ±7.15 years) and the rest were non-media professionals (15 males, and 5 females; age 28-56 years, with an average age of 43.25 ± 8.59 years). Media professionals were chosen with the criterion of more than 6 years of experience in making decisions related to media editing in their everyday work.
All methods described here have been approved by the Ethics Commission for Research with Animals and Humans (CEEAH) of the Universitat Autònoma de Barcelona.
1. Creation and Presentation of the Visual Stimuli
2. Selection of Participants
3. Set-up Preparation
4. Data Acquisition
5. Post-experimental Session
6. Data Analysis
Using the procedure presented here, the eyeblink rate of 40 participants while watching three different video movies was analyzed. For a comparative analysis on media professionalization, half of the participants were media professionals. They were selected based on the criterion of a minimum 6 years of job experience in making decisions related to media editing and audio-visual cuts. For a comparative analysis on editing styles, the three video stimuli were created with exactly the same narrative but a different editing style. One was a one-shot movie with one open shot and no camera movements. Another was a Hollywood-style movie with 33 shots and an average shot length of 5.9 s, as well as continuity throughout the shots, with relationships of content, rhythm, space, and time, in accordance with classical Hollywood editing-style rules. Finally, the third video was an MTV-style movie with 79 shots and an average shot length of 2.4 s. In the latter case, there was discontinuity of content, rhythm, space, and time between the shots.
According to the collected results (Figure 1), the editing style affected viewers' eyeblink rate (X2(2) = 7.2, p= 0.027, Friedman Test). With shorter lengths of the shots and more unexpected cuts, there was a lower spontaneous eyeblink rate in viewers.
The investigation also showed that media professionalization affects eyeblink rate. We found statistically significant differences between the two groups (Mann-Whitney U = 86, n = 20, p= 0.002, Mann-Whitney Rank Sum Test) when comparing all stimuli together. Non-media professionals have a higher eyeblink rate than media professionals. This difference is seen for every editing style analyzed: in the one-shot movie (Mann-Whitney U = 86.5, n = 20,p= 0.002, Mann-Whitney Rank Sum Test); in the Hollywood-style movie (Mann-Whitney U = 90, n = 20,p= 0.003, Mann-Whitney Rank Sum Test); and in the MTV-editing-style movie (Mann-Whitney U = 94.5, n = 20,p= 0.004, Mann-Whitney Rank Sum Test). Media professionals show greater homogeneity in eyeblink rate than non-media professionals, as the standard deviation of their eyeblink rate is much lower, regardless of the editing style analyzed.
Figure 1. Eyeblink rate in different editing styles and in different groups. Spontaneous blink rate (SBR) per min (min-1) in viewers while watching different editing styles. The ends of the boxes define the 25th and 75th percentiles, with a line at the median and error bars defining the 10th and 90th percentiles; unfilled circles represent data points. Viewers' eyeblink rate decreases as the editing of the media content becomes more chaotic. The MTV-editing style provokes a lower eyeblink rate than the Hollywood-editing style or the one-shot movie. Media professionals have a lower eyeblink rate than non-media professionals. This happens for every style of editing. These data were taken and modified from reference 3. Please click here to view a larger version of this figure.
A method for analyzing visual perception of media content with purpose-made video creation is described here. Many other studies attempt to analyze the perception of media content in narrative contexts with pre-existing films. The present method proposes to create visual content with a narrative construction following the criteria of interest, and is based on the suggestion that eyeblink rate is connected to the viewer’s attention9. For that reason, the study detects participants’ eyeblinks while watching media content and compares their rate under different conditions (the editing style and the media professionalization).
The idea that eyeblink rate is connected to cognitive tasks has been developed over several studies during recent decades18,19,20. Eyeblink is also useful as an attentional marker21. A decrease of eyeblink rate has been linked with an increase of attention9,22,23. In this investigation, by using EEG measurements and high-quality video recording for analyzing visual perception of media content, we found that the style of the editing can be used by creators to handle viewers’ attention3. This had already been proposed by the cinema editor Walter Murch, who, while editing films, suspected that eyeblinks may have a comprehension and punctuation marker function in films, related to audio-visual cuts24. This should be of interest in audio-visual production contexts. The fact that the editing style is an element that can affect viewers’ eyeblink rate complements a previous work, which detected that the narrative of a media content affected viewers’ eyeblink rate9. A higher need of attention is created by the MTV-editing style, where cuts occur more often than in the other two styles of editing, and provokes a decrease of eyeblink rate. This is in accord with previous works studying eyeblink inhibition and operational memory25 and in which blink inhibition has been proposed as a mechanism to avoid the loss of information by a task with an increased level of demand7. It is worthwhile to state that in other non-audio-visual platforms, such as paper, the presentation format does not affect the frequency and placement of blinks26.
Previous investigations had found that professionalization is a variable that gives rise to perceptive differences, even cerebral changes, in, e.g., athletes27, architects28, musicians29, or ballet dancers30. Here, we found that media professionalization results in decreasing eyeblink rate while video content is watched, regardless of the editing style8. Since eyeblink also has the physiological function of wetting the eye, the decrease of eyeblink rate in media professionals may be of interest for hazard control and prevention in this professional group.
Critical Steps Within the Protocol
The proposed methodology includes the following critical steps: (i) The selection of the participants since an insufficient description of the criteria used can lead to misunderstandings in possible replications of the experiment. To avoid such problems, it is desirable to explicitly write down in detail which criteria are used in the experiment. (ii) The design and creation of the video stimuli. Traditionally, the analysis of media content has been done with pre-existing narratives (films, advertisements, or TV shows). However, the proposed methodology is based on the preparation of the media content to be analyzed. Since creating professional videos is not easy, researchers may need the help of a professional audiovisual producer. Therefore, this is a critical step within the protocol, where the researcher may need to work with professionals outside the laboratory. It is important to include the variables of interest in the design of the media content. (iii) The synchronization between stimulus presentation and data acquisition. Depending on the hardware and software used by the researcher (such as the software for presenting the stimuli and that used for recording the EEG), the synchronization between them may be confusing. Each manufacturer has their own protocols to manage inputs and outputs for connectivity. In case of doubt, the researcher may need to speak with them.
Significance of the Technique with Respect to Existing/Alternative Methods
Use of the proposed double approach for analyzing eyeblinks helps researchers to contrast the results. While the EEG signal can classify eye movements such as eyeblink, by contrasting results with a recorded HD video of the participants’ faces it is possible to avoid errors in the results. This approach also collects electrical brain activity data for further analysis.
Limitations of the Technique
Different groups of subjects may present significant differences in their eyeblink rates. A limitation of this technique is that, despite obtaining detailed questionnaires from the participants, important information can be missed. For that reason, it is necessary to carefully establish the criteria for selecting subjects with regard to how their professional or social situations can affect their eyeblink rates. This limitation could cause erroneous conclusions, such as misinterpretation of eyeblink patterns linked to subject groups.
The authors have nothing to disclose.
The present study has been supported by a Spanish Ministry of Economy and Competitiveness (BFU2014-56692-R and BFU2017-82375-R) grants.
EEG Device | Neurolectrics | Enobio 20 EEG/EMG system | |
Ag/AgCl Electrodes | Neuroelectrics | [NE022b] GelTrode | |
Recording EEG software | Neuroelectrics | NicOffline software | |
HD-video camera | Sony Corporation | Sony HDR-GW55VE | |
Syringe | Monoject | Monoject 412, curved tip syringe, 50/box | |
Saline electrode EMG gel | Signa-Gel | X32-204: Signa Gel | |
Visual Stimuli Presentation Software | Paradigm Stimulus Presentation | Perception Research System Incorporated | |
EEG software analysis | Centre National de la Recherche Scientifique and Montreal Neurological Institute | Brainstorm3 | |
EEG software analysis | The MathWorks Inc. | MATLAB 2013b | |
TV for video presentation | Panasonic Corporation | PanasonicTH- 42PZ70EA – 50" | |
PC for presenting stimuli | MacBook Air | Year 2013, running Mac OSX Mountain Lion | |
PC for recording stimuli | MacBook | Year 2009 running Windows 7 | With a bootcamp partition of the disk for providing Windows OS |
Statistical Analysis | Systat Software Inc. | Sigmaplot 11.0 |