Measuring Sensitivity to Viewpoint Change with and without Stereoscopic Cues

Many studies have shown that the speed and accuracy of object recognition is compromised by a change in viewpoint^1-7. For example, face recognition is best when the face is viewed in the fronto-parallel plane, i.e. "front on", and declines systematically as the face is rotated in depth away from the fronto-parallel plane. Changes in viewpoint can occur either when the observer, or the viewed object, moves. Importantly, the viewpoint cost is often reduced by the introduction of stereoscopic cues to the observer^1-4,6,8. It is, however, important to distinguish between the rotation of 3D and planar objects. The former may reveal new textures or structural information under rotation while this does not occur with planar objects. In this communication we discuss a method for simulating the appearance of planar shapes and objects that have been subject to rotation-in-depth, or RID. We describe an apparatus for turning the two-dimensional projection of a RID image into a vivid 3D (three-dimensional) percept of RID that is based on the Wheatstone mirror stereoscope⁹. Finally, we report a psychophysical method for determining whether stereoscopic (3D) cues influence our sensitivity to viewpoint change. Movie 1 demonstrates how the appearance of a shape is altered by a viewpoint change/RID.

Simulating a Viewpoint Change

When an object is subject to RID it does not of course physically change in identity or shape. However from the observer’s viewpoint, the shape does change. For instance, if you hold a sheet of paper directly in front of you in landscape view, the image of the paper in your eye, termed the retinal image, is a horizontal rectangle. If you then rotate the paper around its central vertical axis, the retinal image becomes foreshortened along the horizontal (see Figure 1 for a demonstration). Ultimately, if you rotate the paper through 90° the retinal image is a thin strip the thickness of the paper. Thus the largest change to the image of the sheet of paper that has been subject to RID is a horizontal compression of its shape. A smaller change gradient of change occurs in the vertical extent with the image being taller for the nearest edge unchanged at the axis of rotation and shorter for the farther edge.

A RID can be simulated by appropriate transformation of images using software packages or by writing one’s own code^8,10. Readers wishing to know more about RID can consult relevant texts and websites concerned with viewpoint transformations, or learn about built-in functions in the image processing software of packages such as those listed in our Materials. However, when transforming an image of a stimulus on a screen to represent a particular angle of RID there is an additional consideration: which type of projection to use, orthographic or perspective? When we view a scene, nearby objects produce larger retinal images than farther away objects. For a 2D (two-dimensional) object that is subject to RID this means that the far side is compressed more than the near side, as Figure 1 illustrates (see in particular the bottom right figure which has a RID of 60°). The magnitude of the compression asymmetry is dependent upon the size of the object being subject to RID, and on our distance from it. Perspective projection incorporates this compression asymmetry whereas orthographic projection simply ignores it by assigning the same dimensions to the near and far sides of the RID object. Since an observer viewing a planar object rotate will be presented with this asymmetry we will use perspective projection here. Another advantage of our method is that the projection takes into account viewing distance from observer to screen. We have previously used our RID method to simulate viewpoint rotated shapes in both 2D perspective projection and in 3D^8,10.

Simulating a 3D Viewpoint Change

In order to simulate stereoscopic RID one needs to present slightly different images to the left and right eyes. This difference between the two images is known as binocular disparity¹¹, and is a function of both viewing distance and between-eye separation, specifically the inter-pupillary distance. Returning to the example of our sheet of paper in landscape view, if we rotate the piece of paper around the vertical center, clockwise by 45° (so the left edge is receding), each eye’s view of the paper is slightly different. The image projected onto the left eye is slightly less than 45° rotation while the image projected onto the right eye is slightly more than 45° rotation (see Figure 2 for an illustration – observers who can free fuse the top two images will perceive a RID in 3D). If the appropriate RID angles are presented separately to the left and right eyes (the stereo condition) they will produce a disparity gradient and the observer is likely to report a vivid impression of a 3D RID. However if identical images, representing a 45° rotation from the central viewing point, are presented to both eyes (the nonstereo condition) the observer will see a 2D representation of the RID stimuli. An established method for presenting separate images to the left and right eyes using a single monitor is to use a modification of the mirror stereoscope originally developed by Sir Charles Wheatstone in the mid 19^th century¹² – see Figure 3. When the two stereo-half-images are positioned at corresponding locations in each eye’s view, they can be binocularly fused to produce a 3D percept (see Howard and Rogers¹² for an extensive review of stereo methods and limitations).

By combining a simulated viewpoint change with stereoscopic presentation we can determine, for example, the role that stereoscopic cues play in our ability to detect changes in viewpoint. The demonstration results below reveal that when the size of a planar object, in this case a curved contour, is known or familiar to the observer, the change in the 2D width of the curve (sometimes termed its sag) is used to detect the change in RID, i.e. viewpoint (see Bell et al.¹⁰ for the full manuscript on this effect). Below we describe in detail a psychophysical method for measuring sensitivity to RID change for 2D- and 3D-defined planar shapes.