Object Substitution Masking
6,324 Views
•
•
Overview
Source: Laboratory of Jonathan Flombaum—Johns Hopkins University
Visual masking is a term used by perceptual scientists to refer to a wide range of phenomena in which in an image is presented but not perceived by an observer because of the presentation of a second image. There are several different kinds of masking, many of them relatively intuitive and unsurprising. But one surprising and important type of masking is called Object Substitution Masking. It has been a focus of research in vision science since it was discovered, relatively recently, around 1997 by Enns and Di Lollo.1
This video will demonstrate standard procedures for how to conduct an object substitution experiment, how to analyze the results, and it will also explain the hypothesized causes for this unusual form of masking.
Procedure
1. Stimuli and design
- To run this experiment, you will need a programming environment such as MATLAB or experimental sequencing software such as E-Prime.
- Each trial will consist of four basic components: A fixation cross, a shape stimulus display (called the target display), a mask (four dots), and a response display. Figure 1 depicts the four primary elements in each trial.
- The display background will always be white.
- The fixation cross is a small cross in the center of the display, with each arm of the cross measuring 0.25 cm. The fixation cross will be present at all times, except for when the response display appears.
- The response display will consist of a black outline of a square, a circle, a diamond, and a triangle arranged horizontally and in the center of the screen. The shapes should each be inscribed within an invisible circle with a radius of 0.75 cm.
- The target display will include eight shapes, randomly selected on each trial from the set of four: a black outline of a circle, square, diamond, and triangle. Not all shapes need to appear in each trial, and of course, shapes can (and will be repeated) in a given trial. Again, the shapes will each be 0.75 cm in radius. They will appear in random positions on an invisible circle with a radius of 1.5 cm from the fixation cross in the center of the screen.
- The mask will be four black discs (dots) with a radius of 0.25 cm. The dots should be arranged to form the four corners of a square just large enough to include all of the four target shapes within.
Figure 1: Primary elements of an object substation display. Every trial will begin with a fixation display, and participants will be instructed to fixate the cross before initiating a trial. Every trial will end with a response display, in which the participant will select the shape she remembers seeing between the four dots. Between the fixation and response display, a target display will show a ring of eight shapes and a mask of four dots will also appear, in a position surrounding one of the shapes. As described in more detail in the procedure, the mask and target display can appear in different orders, but each will remain present for 30 ms.
- The sequence of events in each trial will begin with a fixation screen that will remain present until the participant presses the spacebar, and for 200 ms after.
- A trial will always end with a response display that will remain present until the participant enters a response (by clicking on the shape she believes she saw between the four dots on that trial.)
- Between the fixation and response displays a target display and a mask will each be shown for 30 ms. The mask will surround the position of a randomly selected target shape in each trial.
- The critical independent variable in this experiment is the stimulus onset asynchrony (SOA). This is defined as the time difference between the onset of two stimuli, in the case of this experiment, the mask and the target display. Thus, the SOA is the time of onset of the target display minus the time of onset of the mask display. Figures 2-4 schematize several trials of the experiment with different SOAs.
- If the SOA is 0, it means that the stimuli appear together (and since each last for 30 ms, they disappear together as well).
- If the SOA is negative, it means that the mask appeared before the target display.
- If the SOA is positive, it means that the mask appeared after the target display.
- The experiment will include 15 different SOAs, 0 ms as well as positive and negative: 10, 30, 50, 70, 90, 150, and 300 ms.
Figure 2: Stimulus onset asynchrony of 0 ms. In a trial with an SOA of 0 ms, the mask and the targets appear simultaneously. Since each is programmed to remain present for 30 ms, they are present and expire together as well.
Figure 3: Stimulus onset asynchrony of 50 ms. An SOA of 50 ms, whether negative or positive, means that one stimulus will onset 50 ms after the other. But these stimuli are programmed to last only 30 ms, meaning that an SOA of 50 will leave 20 ms with an empty display (except for the fixation cross). For this experiment, we've defined SOA as target onset-mask onset, meaning that negative SOA values are associated with the mask appearing first, and positive values are when the mask appears second.
Figure 4: Stimulus onset asynchrony of 10 ms. With masks and targets programmed to remain present for 30 ms, SOAs of 10 ms leave 20 ms during which the mask and the target overlap.
- Sequence the experiment to include 20 trials with each of the 15 SOAs, randomly intermixing all the different kinds of trials.
- A critical factor is that the participant does not know where the dots will appear in each trial, so be sure that the dots are appearing unpredictably from trial to trial.
- Be sure that the experimental program outputs sufficient information about each trial to allow you to analyze the results later. The relevant information for each trial is: the trial number, the SOA on that trial, the shape shown between the mask, and the shape response given by the participant.
2. Running the experiment
- To run this experiment, you'll want to recruit 10 participants, testing each individually in a quiet room.
- When a participant arrives for the experiment, seat them 60 cm away from the monitor of the testing computer.
- The image on the screen should be of a target display with a mask. Use this image to explain the instructions as follows:
- In each trial of this experiment you will see a ring of shapes, like the one shown here [pointing]. Additionally, you will see four dots, like the ones here. Sometimes, they will overlap, as in this image. But sometimes, one might precede the other. Regardless of their order of appearance, your task is simply to try to see and remember the shape that appears in the space between those four dots. Does this make sense?"
- Great. Sometimes, it might feel hard-you might feel like you really don't know the answer. Just guess in those cases. A few other things. Each trial will start with a display of just a cross, like the one in the middle, here. Before you start a trial, make sure you are fixating the cross, and try your best not to move your eyes. When you are ready, you'll press the spacebar to start a trial."
- After explaining the instructions to the participant, start the experiment, and observe the participant for four or five trials to make sure they have understood the instructions. Then leave them to complete the experiment.
3. Analyzing the results
- To analyze the results, you'll first want to compute response accuracy individually for each subject and SOA.
- Next, you can average together response accuracy by SOA, across subjects.
- To determine whether there is a significant effect of SOA on performance, perform a repeated measures ANOVA on accuracy as a function of SOA using MATLAB or a statistical package such as SPSS.
The visual world is full of objects that interact in space and time, and overlap within these dimensions can influence the conscious perception of them—a concept referred to as visual masking.
Similar to someone wearing a costume disguise, the phenomenon occurs when a target item—such as a face—cannot be perceived due to the presence of a second object—a mask.
When a target is substituted with a stimulus that overlaps in part of the same spatial location, this is a form of visual masking called object substitution.
Based on the methods pioneered by Enns and Di Lollo in 1997, this video demonstrates how to design and implement an object substitution masking experiment, as well as how to analyze the data and interpret the results dealing with the conscious perceptions of shapes.
In this experiment, object substitution masking is induced in participants as they observe the presentation of four elements on a computer screen: a fixation cross, target display, mask, and response choices.
At the start of each trial, a fixation cross is shown, which consists of 50-mm lines in the center of the screen, and ensures that participants are paying attention.
This is followed by the second element, the target display: eight shapes that are randomly selected from a set of four images—a circle, square, diamond, and triangle. They are displayed around an invisible circle with a radius of 150 mm for 30 ms.
Immediately after is the third element, the mask, which consists of four black dots, each with a radius of 25 mm, arranged to form the four corners of a square just large enough to enclose one shape. The mask surrounds the location of the randomly selected target shape and remains visible for 30 ms during a trial.
A critical independent variable here is the stimulus onset asynchrony—SOA for short—defined as the time difference between the appearance of the target display and mask.
A positive stimulus onset asynchrony means that the mask will appear after the target display. With, for instance, an SOA of 50 ms, the target display is shown for 30 ms, followed by a period of 20 ms where only the fixation cross is present before the mask comes on for 30 ms.
For SOAs less than 30 ms, say 10, the target display is shown for 10 ms before the mask becomes visible. After another 20 ms, the target display goes away and the mask remains for 10 more ms.
The time that the target display and mask are both onscreen is referred to as stimulus overlap. This is maximal when the stimulus onset asynchrony is zero.
When the stimulus onset asynchronies are negative, the order of the elements is reversed: the mask appears before the target display. With an SOA of -10 ms, the mask is presented for 10 ms before the target overlaps for 20 ms. It then disappears, leaving the target display visible for an additional 10 ms.
With larger negative values, like an SOA of -50 ms, the mask is shown for 30 ms. There is then a period of 20 ms where only the fixation cross appears before the target display comes on for 30 ms.
Regardless of the SOA, the fourth and final element is the response display: four shapes arranged horizontally in the center of the screen. They are shown until the participant presses the key corresponding to the shape of their choice.
The dependent variable is the percentage of correct responses recorded across the number of SOAs. Object substitution masking is expected to be induced in a discrete range of time, resulting in performance with reduced accuracy during positive SOAs, when the mask occurs shortly after and overlaps with the target display.
To begin the experiment, greet one of the recruited participants in the lab and guide them through the consent forms. Then, have them sit comfortably, 60 cm away from the monitor of the test computer.
Explain the task instructions: they’ll see a ring of eight shapes, along with four dots that will appear at a random location. Indicate that this mask may sometimes overlap a shape in time, but could also either precede or occur after it.
Instruct the participant to remember the shape that appears in the space between, and when in doubt, to simply guess.
Once the main rules are understood, describe a few more points: They should press the spacebar to start each trial; and when the fixation cross appears on the display, they should not move their eyes during the trials.
Now, have the participant start the program and watch them as they complete a couple of trials. At this point, leave the room.
Without supervision, allow the participant to complete all 300 trials—20 for each stimulus onset asynchrony value. Note that there are 15 values—ranging from negative, with the mask preceding the target—to positive, with the mask following, including an overlap zone.
When the participant has completed the task, return to the room and thank them for taking part in the experiment.
To analyze the data, compute the response accuracy—as percent correct—across all of the stimulus onset asynchronies, and graph the averages for the 15 time points.
As predicted, with very large SOAs of 150 or 300 ms, positive or negative, performance was highly accurate because the mask and the target display were perceived as separate events.
Similarly, for the negative SOAs between -90 and -10 ms, performance was fairly accurate since the participant’s attention was directed to the correct location by the appearance of the mask before the target display.
However, accuracy dropped to 50% when the SOAs were near zero, as the stimuli overlapped and appeared too briefly to be perceived.
The critical range of SOAs consisted of values between 10 and 90, where the target display was shown before the mask. Here, performance was poor, dropping near chance level. This suggests that object substitution masking took place and that the four-dot mask was enough to confuse the brain before a conscious perception of the shape formed.
Now that you are familiar with Object Substitution Masking, let’s look at how it’s used in studies of conscious awareness, as well as those investigating the neural circuitry involved in visual perception.
This masking paradigm can be combined with repeated Transcranial Magnetic Stimulation, rTMS, to isolate brain circuits involved in conscious perception. A magnetic coil can be used to repeatedly induce small electrical potentials in the brain, causing a small portion of cortex to briefly deactivate.
In one study, Hirose and colleagues found that if the V5/MT+ regions of visual cortex, known to play a role in the perception of motion, were deactivated during the task, they were able to negate the effects of masking. This suggests that the rTMS disruption caused the mask and target to no longer be perceived as part of the same event, allowing the subject to see both.
Other researchers are investigating whether a stimulus needs to be perceived to influence behavior, like verbal priming. For more details on this effect, check out our video in the Cognitive Psychology collection, Verbal Priming.
In a study conducted by Goodhew and colleagues, they used a variation of the masking task—the four dots in the mask were either pink or blue—and asked participants to remember the color. The masks were presented with target stimuli consisting of the words PINK, BLUE, MAIL, HOUR, JQCG, and AWHF.
With an SOA of 200 ms, participants named the color of the mask faster when the target word was the name of the color than when the target was not. This was true whether or not the participant could correctly identify the target as being either a word or non-word, suggesting stimuli do not need to be perceived or enter consciousness to be useful.
You’ve just watched JoVE’s video on Object Substitution Masking. Now you should have a good understanding of how to design the elements and run the experiment, as well as how to analyze and assess the results.
Thanks for watching!
Results
Figure 5 graphs average response accuracy across participants as a function of SOA. As the graph should make clear, an ANOVA analysing these results would probably show a significant effect of SOA. What kind of effect does it show? It appears that with very large SOAs, negative or positive, the mask does nothing-performance in the task is very good. When separated by 150 or 300 ms, the mask and the target stimulus are really just separate events. But these are critical conditions because they show that the target shapes can be perceived, even related to the position of the four dots, in the 30 ms of exposure. In other words, they show that there is nothing inherently too fast about these presentation times. Similarly, with negative SOAs, performance is mostly pretty good. These are trials in which the mask precedes the target stimulus. Even with an SOA of -10 and 0, performance is 50% or better, and the mask and stimulus overlap during these SOAs for 20 to 30 ms.
Figure 5: Results of Object Substitution Masking experiment. The results plotted are of average response accuracy as a function of SOA. Since there were four shapes in each trial, guessing would produce an average accuracy of 25%, labeled by the red line marked chance. When the mask precedes the target stimulus (when the SOA is negative), performance tends to be very good, better than 50% and often better than 80%. This is because this kind of mask needs to come after a target in order to mask it. With SOAs in the range of 10 to 90 ms, however, accuracy is surprisingly low, at times dropping to 25%. This is the range of SOAs during which an Object Substitution Mask works.
The critical SOAs are the ones between 10 and 90 ms. At these SOAs, performance is very bad, dropping as low as chance-what someone would do if they were just guessing. Performance at these SOAs demonstrates that Object Substitution Masking is taking place. Why?
Remember that the four dots do not overlap or cover up any of the masked shape. But the space that they surround entirely includes the shape. The explanation for this phenomenon is that for a stimulus to be perceived consciously it needs to do more than just stimulate the retina; it needs to be processed and reprocessed. Conscious perception is something that takes time for our brains to create. The four dots appearing to surround a position that was just occupied serve to effectively confuse the brain; they halt the reprocessing of the original stimulus that would be necessary for it to make it into conscious awareness.
Applications and Summary
Among the many applications of Object Substitution Masking in recent years are studies that have utilized it in conjunction with neurophysiological techniques in order to isolate brain circuits involved in the production of conscious experience. Hirose and colleagues2 in 2005 conducted an experiment using a technique known as repeated Transcranial Magnetic Stimulation (rTMS): Researchers use a magnetic coil to induce small electrical potentials in the brain of a subject, and repeated induction can cause a small portion of cortex to deactivate for a brief period of time. In the Hirose et al. study, they deactivated a region of visual cortex called V5/MT+. The effect was that this prevented object substitution masking-positive SOA dot presentations did not prevent perception of the stimuli. V5/MT+ is known to play a large role in the perception of motion. This study suggested that its role might be broader, participating in connecting moments together in perceptual experience. When disrupted, the mask and the target stimulus can’t be seen as part of the same event, and as a result, the mask fails to mask.
Another way that Object Substitution Masking has been used is to investigate questions about whether stimuli need to make it into conscious awareness in order to influence behaviour. For example, a word that is masked is not reportable by an observer. Will it have a priming effect however? Some research suggests that it does.3
References
- Enns, J. T., & Di Lollo, V. (1997). Object substitution: A new form of masking in unattended visual locations. Psychological Science, 8(2), 135-139.
- Hirose, N., Kihara, K., Tsubomi, H., Mima, T., Ueki, Y., Fukuyama, H., & Osaka, N. (2005). Involvement of V5/MT+ in object substitution masking: evidence from repetitive transcranial magnetic stimulation. Neuroreport,16(5), 491-494.
- Goodhew, S. C., Visser, T. A., Lipp, O. V., & Dux, P. E. (2011). Implicit semantic perception in object substitution masking. Cognition, 118(1), 130-134.
Transcript
The visual world is full of objects that interact in space and time, and overlap within these dimensions can influence the conscious perception of them—a concept referred to as visual masking.
Similar to someone wearing a costume disguise, the phenomenon occurs when a target item—such as a face—cannot be perceived due to the presence of a second object—a mask.
When a target is substituted with a stimulus that overlaps in part of the same spatial location, this is a form of visual masking called object substitution.
Based on the methods pioneered by Enns and Di Lollo in 1997, this video demonstrates how to design and implement an object substitution masking experiment, as well as how to analyze the data and interpret the results dealing with the conscious perceptions of shapes.
In this experiment, object substitution masking is induced in participants as they observe the presentation of four elements on a computer screen: a fixation cross, target display, mask, and response choices.
At the start of each trial, a fixation cross is shown, which consists of 50-mm lines in the center of the screen, and ensures that participants are paying attention.
This is followed by the second element, the target display: eight shapes that are randomly selected from a set of four images—a circle, square, diamond, and triangle. They are displayed around an invisible circle with a radius of 150 mm for 30 ms.
Immediately after is the third element, the mask, which consists of four black dots, each with a radius of 25 mm, arranged to form the four corners of a square just large enough to enclose one shape. The mask surrounds the location of the randomly selected target shape and remains visible for 30 ms during a trial.
A critical independent variable here is the stimulus onset asynchrony—SOA for short—defined as the time difference between the appearance of the target display and mask.
A positive stimulus onset asynchrony means that the mask will appear after the target display. With, for instance, an SOA of 50 ms, the target display is shown for 30 ms, followed by a period of 20 ms where only the fixation cross is present before the mask comes on for 30 ms.
For SOAs less than 30 ms, say 10, the target display is shown for 10 ms before the mask becomes visible. After another 20 ms, the target display goes away and the mask remains for 10 more ms.
The time that the target display and mask are both onscreen is referred to as stimulus overlap. This is maximal when the stimulus onset asynchrony is zero.
When the stimulus onset asynchronies are negative, the order of the elements is reversed: the mask appears before the target display. With an SOA of -10 ms, the mask is presented for 10 ms before the target overlaps for 20 ms. It then disappears, leaving the target display visible for an additional 10 ms.
With larger negative values, like an SOA of -50 ms, the mask is shown for 30 ms. There is then a period of 20 ms where only the fixation cross appears before the target display comes on for 30 ms.
Regardless of the SOA, the fourth and final element is the response display: four shapes arranged horizontally in the center of the screen. They are shown until the participant presses the key corresponding to the shape of their choice.
The dependent variable is the percentage of correct responses recorded across the number of SOAs. Object substitution masking is expected to be induced in a discrete range of time, resulting in performance with reduced accuracy during positive SOAs, when the mask occurs shortly after and overlaps with the target display.
To begin the experiment, greet one of the recruited participants in the lab and guide them through the consent forms. Then, have them sit comfortably, 60 cm away from the monitor of the test computer.
Explain the task instructions: they’ll see a ring of eight shapes, along with four dots that will appear at a random location. Indicate that this mask may sometimes overlap a shape in time, but could also either precede or occur after it.
Instruct the participant to remember the shape that appears in the space between, and when in doubt, to simply guess.
Once the main rules are understood, describe a few more points: They should press the spacebar to start each trial; and when the fixation cross appears on the display, they should not move their eyes during the trials.
Now, have the participant start the program and watch them as they complete a couple of trials. At this point, leave the room.
Without supervision, allow the participant to complete all 300 trials—20 for each stimulus onset asynchrony value. Note that there are 15 values—ranging from negative, with the mask preceding the target—to positive, with the mask following, including an overlap zone.
When the participant has completed the task, return to the room and thank them for taking part in the experiment.
To analyze the data, compute the response accuracy—as percent correct—across all of the stimulus onset asynchronies, and graph the averages for the 15 time points.
As predicted, with very large SOAs of 150 or 300 ms, positive or negative, performance was highly accurate because the mask and the target display were perceived as separate events.
Similarly, for the negative SOAs between -90 and -10 ms, performance was fairly accurate since the participant’s attention was directed to the correct location by the appearance of the mask before the target display.
However, accuracy dropped to 50% when the SOAs were near zero, as the stimuli overlapped and appeared too briefly to be perceived.
The critical range of SOAs consisted of values between 10 and 90, where the target display was shown before the mask. Here, performance was poor, dropping near chance level. This suggests that object substitution masking took place and that the four-dot mask was enough to confuse the brain before a conscious perception of the shape formed.
Now that you are familiar with Object Substitution Masking, let’s look at how it’s used in studies of conscious awareness, as well as those investigating the neural circuitry involved in visual perception.
This masking paradigm can be combined with repeated Transcranial Magnetic Stimulation, rTMS, to isolate brain circuits involved in conscious perception. A magnetic coil can be used to repeatedly induce small electrical potentials in the brain, causing a small portion of cortex to briefly deactivate.
In one study, Hirose and colleagues found that if the V5/MT+ regions of visual cortex, known to play a role in the perception of motion, were deactivated during the task, they were able to negate the effects of masking. This suggests that the rTMS disruption caused the mask and target to no longer be perceived as part of the same event, allowing the subject to see both.
Other researchers are investigating whether a stimulus needs to be perceived to influence behavior, like verbal priming. For more details on this effect, check out our video in the Cognitive Psychology collection, Verbal Priming.
In a study conducted by Goodhew and colleagues, they used a variation of the masking task—the four dots in the mask were either pink or blue—and asked participants to remember the color. The masks were presented with target stimuli consisting of the words PINK, BLUE, MAIL, HOUR, JQCG, and AWHF.
With an SOA of 200 ms, participants named the color of the mask faster when the target word was the name of the color than when the target was not. This was true whether or not the participant could correctly identify the target as being either a word or non-word, suggesting stimuli do not need to be perceived or enter consciousness to be useful.
You’ve just watched JoVE’s video on Object Substitution Masking. Now you should have a good understanding of how to design the elements and run the experiment, as well as how to analyze and assess the results.
Thanks for watching!