We discuss a novel method forviewpoint-rotation of visual stimuli, and demonstrate using a mirror stereoscopethe three-dimensional percept of rotation-in-depth. The technique can be used to investigate the role of stereoscopic cues in encoding viewpoint-rotated figures.
The speed and accuracy of object recognition is compromised by a change in viewpoint; demonstrating that human observers are sensitive to this transformation. Here we discuss a novel method for simulating the appearance of an object that has undergone a rotation-in-depth, and include an exposition of the differences between perspective and orthographic projections. Next we describe a method by which human sensitivity to rotation-in-depth can be measured. Finally we discuss an apparatus for creating a vivid percept of a 3-dimensional rotation-in-depth; the Wheatstone Eight Mirror Stereoscope. By doing so, we reveal a means by which to evaluate the role of stereoscopic cues in the discrimination of viewpoint rotated shapes and objects.
Many studies have shown that the speed and accuracy of object recognition is compromised by a change in viewpoint1-7. For example, face recognition is best when the face is viewed in the fronto-parallel plane, i.e. "front on", and declines systematically as the face is rotated in depth away from the fronto-parallel plane. Changes in viewpoint can occur either when the observer, or the viewed object, moves. Importantly, the viewpoint cost is often reduced by the introduction of stereoscopic cues to the observer1-4,6,8. It is, however, important to distinguish between the rotation of 3D and planar objects. The former may reveal new textures or structural information under rotation while this does not occur with planar objects. In this communication we discuss a method for simulating the appearance of planar shapes and objects that have been subject to rotation-in-depth, or RID. We describe an apparatus for turning the two-dimensional projection of a RID image into a vivid 3D (three-dimensional) percept of RID that is based on the Wheatstone mirror stereoscope9. Finally, we report a psychophysical method for determining whether stereoscopic (3D) cues influence our sensitivity to viewpoint change. Movie 1 demonstrates how the appearance of a shape is altered by a viewpoint change/RID.
Simulating a Viewpoint Change
When an object is subject to RID it does not of course physically change in identity or shape. However from the observer’s viewpoint, the shape does change. For instance, if you hold a sheet of paper directly in front of you in landscape view, the image of the paper in your eye, termed the retinal image, is a horizontal rectangle. If you then rotate the paper around its central vertical axis, the retinal image becomes foreshortened along the horizontal (see Figure 1 for a demonstration). Ultimately, if you rotate the paper through 90° the retinal image is a thin strip the thickness of the paper. Thus the largest change to the image of the sheet of paper that has been subject to RID is a horizontal compression of its shape. A smaller change gradient of change occurs in the vertical extent with the image being taller for the nearest edge unchanged at the axis of rotation and shorter for the farther edge.
A RID can be simulated by appropriate transformation of images using software packages or by writing one’s own code8,10. Readers wishing to know more about RID can consult relevant texts and websites concerned with viewpoint transformations, or learn about built-in functions in the image processing software of packages such as those listed in our Materials. However, when transforming an image of a stimulus on a screen to represent a particular angle of RID there is an additional consideration: which type of projection to use, orthographic or perspective? When we view a scene, nearby objects produce larger retinal images than farther away objects. For a 2D (two-dimensional) object that is subject to RID this means that the far side is compressed more than the near side, as Figure 1 illustrates (see in particular the bottom right figure which has a RID of 60°). The magnitude of the compression asymmetry is dependent upon the size of the object being subject to RID, and on our distance from it. Perspective projection incorporates this compression asymmetry whereas orthographic projection simply ignores it by assigning the same dimensions to the near and far sides of the RID object. Since an observer viewing a planar object rotate will be presented with this asymmetry we will use perspective projection here. Another advantage of our method is that the projection takes into account viewing distance from observer to screen. We have previously used our RID method to simulate viewpoint rotated shapes in both 2D perspective projection and in 3D8,10.
Simulating a 3D Viewpoint Change
In order to simulate stereoscopic RID one needs to present slightly different images to the left and right eyes. This difference between the two images is known as binocular disparity11, and is a function of both viewing distance and between-eye separation, specifically the inter-pupillary distance. Returning to the example of our sheet of paper in landscape view, if we rotate the piece of paper around the vertical center, clockwise by 45° (so the left edge is receding), each eye’s view of the paper is slightly different. The image projected onto the left eye is slightly less than 45° rotation while the image projected onto the right eye is slightly more than 45° rotation (see Figure 2 for an illustration – observers who can free fuse the top two images will perceive a RID in 3D). If the appropriate RID angles are presented separately to the left and right eyes (the stereo condition) they will produce a disparity gradient and the observer is likely to report a vivid impression of a 3D RID. However if identical images, representing a 45° rotation from the central viewing point, are presented to both eyes (the nonstereo condition) the observer will see a 2D representation of the RID stimuli. An established method for presenting separate images to the left and right eyes using a single monitor is to use a modification of the mirror stereoscope originally developed by Sir Charles Wheatstone in the mid 19th century12 – see Figure 3. When the two stereo-half-images are positioned at corresponding locations in each eye’s view, they can be binocularly fused to produce a 3D percept (see Howard and Rogers12 for an extensive review of stereo methods and limitations).
By combining a simulated viewpoint change with stereoscopic presentation we can determine, for example, the role that stereoscopic cues play in our ability to detect changes in viewpoint. The demonstration results below reveal that when the size of a planar object, in this case a curved contour, is known or familiar to the observer, the change in the 2D width of the curve (sometimes termed its sag) is used to detect the change in RID, i.e. viewpoint (see Bell et al.10 for the full manuscript on this effect). Below we describe in detail a psychophysical method for measuring sensitivity to RID change for 2D- and 3D-defined planar shapes.
1. Prescreening
2. Achieving Binocular Fusion in the Stereoscope
3. Task Instructions for the Observer
4. Measuring Sensitivity to Viewpoint Change/RID in Contours
Figure 4 plots example results obtained from such an experiment. The y-axis in each figure describes the proportion of times that the test pattern was chosen as having a greater RID. Rather than plotting the absolute number of times this occurred, this is given as a proportion. The x-axis describes the physical RID of the test pattern; remembering that here, the reference pattern was held at a constant 45° RID. The data points represent the proportion of times that the observer chose the test curve as having the greater RID for a range of physical RIDs. The computer only scores the number of times that the test pattern is chosen as having a greater RID; this means that data for an "ideal" observer should range between zero: never chose the test as greater in RID when it had a smaller RID than the reference; and one: always chose the test as greater in RID when it had a larger RID than the reference.
From this plotted data we can determine an observer’s sensitivity to changes in RID. To do this we fit the data to a Cumulative Gaussian function using a statistical package (see Materials). The fit enables us to estimate the precision, or sensitivity of an observer’s judgment of RID. In the figure, it is clear from the steepness of the solid lines in each plot that observers are highly sensitive to a change in RID angle. This means that observers are highly sensitive to a RID manipulation of a planar object; here a curved contour. However, in our paradigm we have also chosen to compare performance in two conditions: stereo and nonstereo. Since the data representing these two conditions physically overlaps, it appears that on this task there is no difference in precision between stereo and nonstereo conditions. An additional statistical comparison of precision estimates can be performed in order to determine whether or not the data statistically differ.
Movie 1. Demonstrates the change in the two dimensional (2D) retinal image of a shape that is being subject to a rotation-in-depth (RID) around vertical. Here the viewer can see that the main change to the image of a shape that has RID is a horizontal compression of its shape.
Figure 1. Static illustration of how the retinal image of a sheet of paper is altered as the paper is RID. Here the viewer can also appreciate the perspective projection method that we apply. That is, as the paper is increasingly RID the reader can see that the far (left) edge of the paper is being more compressed in vertical size than the near edge. This asymmetry in the size of each edge occurs because near objects produce larger retinal images while far objects produce smaller retinal images. Perspective projections, which we have used, incorporate this asymmetry while orthographic projections ignore this effect. The title above each image shows its angle of RID. Click here to view larger image.
Figure 2. Using binocular disparity to present in 3D. If two matched images at slightly different RID angles are presented separately to the left and right eye, a stereo (3D) RID can be perceived. For example, to achieve the percept of a sheet of paper that is RID 45° in 3D (left edge receding) the image projected to the left eye should be one at an angle less than 45° while that projected to the right eye should be at an angle greater than 45°. Click here to view larger image.
Figure 3. A top down view of our adapted Wheatstone eight mirror stereoscope, with top cover removed. The red lines and arrows depict the separate paths of light from the monitor to each eye. This mirror stereoscope incorporates 8 first surface mirrors; four along the path to each eye. Using a series of mirrors in this way allows for a viewing distance of approximately 1 m from the screen to the observer: a comfortable viewing distance. All nonmirrored surfaces have been painted in matte black to avoid shadow and reflectance issues. Click here to view larger image.
Figure 4. Example results for three observers showing sensitivity to RID/viewpoint change (vertical axes) as a function of the RID angle of the stimuli. Raw data points plot the proportion of time that the test stimulus, here a curved contour, was chosen as being greater than 45° RID. The horizontal axis plots the RID of the test curve. The reference curve was constant at 45° in all conditions. Solid lines show the fitted Cumulative Gaussian function to each set of obtained data. Black points and lines show stereo (different RID image projected to each eye) condition data while Grey points and lines show nonstereo (same RID image projected to each eye) condition data. Click here to view larger image.
The methods discussed in this communication can be used by any researcher seeking to address questions related to discrimination and/or recognition of viewpoint rotated objects. Importantly, the method of perspective projection is not limited to the stimuli demonstrated here; instead it can be applied to a wide range of experimental stimuli, from lines to pictures of complex objects. Our introduction describes ways to apply this transformation using custom software, or library software such as MATLAB.
There is considerable interest in understanding how we recognize objects across different viewpoints. It should be noted that there are several ways to simulate a change in the 3D structure and orientation of surfaces and objects, including through the use of shading or brightness variations6,14 and occlusion11. In fact, changes in viewpoint are often correlated with variations in these attributes, and so future studies must investigate the role of different types of cues in isolation as well as in combination. Our method provides a means for contributing to that goal.
Our method also offers promise for researchers outside of the fields of visual perception and cognition. For instance, researchers have begun to investigate the effects of viewpoint change on a number of different behavioral dimensions, including on postural sway15, and also, on the reading of material presented on computerized displays16. Our method provides a means by which to control the 2D and 3D information presented to the observer, during such tasks. This could allow for a more in depth understanding of the visual inputs to the vestibular and language systems.
Creating a Mirror Stereoscope
An exciting aspect of the current method is the use of the mirror stereoscope to facilitate the percept of a 3D rotation-in-depth, or RID. The stereoscope apparatus that we have described here (Figure 3) was custom built for our lab. Should researchers wish to build such a device, it is important to note that they must use front surface, or first surface mirrors. These are mirrors with the mirrored surface on the front, as opposed to the back of the glass, as in traditional mirrors. The importance of this point can easily be appreciated if readers put their finger on the glass surface of a traditional mirror; they will invariably observe a faint ‘ghost’ reflection of their finger caused by light reflecting back from the front surface of the glass. This makes traditional mirrors unsuitable for use in visual experiments, especially when the light path travels across multiple mirrored surfaces as in our stereoscope. First-surface mirrors can be purchased through specialist companies. Alternatively, researchers may prefer to buy a ready-made stereoscope. Many companies offer "ready-to-go" stereoscope solutions that can be fitted directly onto existing monitors and laptops alike. See our Materials section for listings of relevant companies.
Limitations and Troubleshooting
Presenting information in 3D represents an opportunity to move beyond the traditional flat panel display and to create a more realistic representation of your experimental stimulus. It is however critical that you establish first that your experimental participants have stereo vision. There are a number of different prescreening devices for this purpose. Many of these can be purchased on line for a small cost (see Materials for an example). The test we employ here determines whether an individual’s stereo acuity falls into the normal range. If these tests reveal that your observers cannot effectively see in stereo then they are not suitable for testing on stereo tasks. Such observers could however still participate in experiment aimed at exploring 2D representations of a RID.
Before testing for the first time, it is important that the experimenter is trained in the use of the prescreening stereo tests, in running experimental control software in MATLAB, and in the statistical analysis of experimental results from this paradigm. Prior to each testing sessions, ensure that all hardware connections are secure and that the mirror stereoscope is positioned correctly against the display monitor: such that no additional light can enter the stereoscope device.
The authors have nothing to disclose.
This research was supported by the Australian Research council. ARC Grant # DP110101511 given to JB; National Sciences and Engineering Research Council, NSERC of Canada Grant # RGPIN121713-11 given to FK ; ARC grant # DP130102580 and DP110104553 to DRB.
Stereo testing | http://www.stereooptical.com/products/stereotests | Titmus stereofly test | Handheld testing devices |
Prebuilt stereo devices | http://www.stereoaids.com.au | prebuilt stereoscopes | Suitable for desktop and laptop computers |
Mirror stereoscope | Custom made; or, Edmundoptics | 8 mirror stereoscope | www.edmundoptics.com |
Graphics card | NVidia | NVidia or ATI recommended | |
Experimental control software | MATLAB | MATLAB, Python, C++, Visual Basic recommended | |
Experimental control software | Psychtoolbox.org | Psychtoolbox | Free to download but requires MATLAB |
Statistical software | Graphpad | Prism V5 | http://www.graphpad.com/ |
Statistical software | Palamedes | Psychophysics guide | http://www.palamedestoolbox.org |
3D Projection documentation | Wikipedia | http://en.wikipedia.org/wiki/3D_projection |