Visualizing Visual Adaptation

The illustrated protocol demonstrates how the effects of adaptation to a change in the environment or the observer can be portrayed in images. The form this portrayal takes will depend on the assumptions made for the model — for example, how color is encoded, and how the encoding mechanisms respond and adapt. Thus the most important step is deciding on the model for color vision — for example what the properties of the hypothesized channels are, and how they are assumed to adapt. The other important steps are to set appropriate parameters for the properties of the two environments, or two observer sensitivities, that you are adapting between.

The model illustrated is very simple, and there are many ways in which it is incomplete and could be expanded depending on the application. For example, color information is not encoded independently of form, and the illustrated simulations take no account of the spatial structure of the images or of neural receptive fields, or of known interactions across mechanisms such as contrast normalization³⁸. Similarly, all pixels in the images are given equal weight, and thus the simulations do not incorporate spatial factors such as how scenes are sampled with eye movements. Adaptation in the model is also assumed to represent simple multiplicative scaling. This is appropriate for some forms of chromatic adaptation but may not correctly describe the response changes at post-receptoral levels. Similarly, the contrast response functions in the model are linear and thus do not simulate the actual response functions of neurons. A further important limitation is that the illustrated simulations do not incorporate noise. If this noise occurs at or prior to the sites of the adaptation, then adaptation may adjust both signal and noise and consequently may have very different effects on appearance and visual performance³⁹. One way to simulate the effects of noise is to introduce random perturbations in the stimulus²⁸. However, this will not mimic what this noise "looks like" to an observer.

As suggested by the illustrated examples, the simulations can capture many properties of color experience that are not evident when considering only the spectral and contrast sensitivity of the observer, and in particular function to highlight the importance of adaptation in normalizing color perception and compensating for the sensitivity limits of the observer. In this regard, the technique provides a number of advantages and applications for visualizing or predicting visual percepts. These include the following:

Better Simulations of Variant Vision

As noted, filtering an image for a different sensitivity reveals what one experiences when information in the image is altered, but does less well at predicting what an observer with that sensitivity would experience. As an example, a gray patch filtered to simulate the yellowing lens of an older observer's eye looks yellower⁹. But older observers who are accustomed to their aged lenses instead describe and probably literally see the stimulus as gray⁴⁰. As shown here, this is a natural consequence of adaptation in the visual system²⁸, and thus incorporating this adaptation is important for better visualizing an individual's percepts.

A Common Mechanism Predicting Differences Between Observers and Between Environments

Most simulation techniques are focused on predicting changes in the observer. Yet adaptation is also routinely driven by changes in the world^18,19. Individuals immersed in different visual environments (e.g., urban Vs. rural, or arid Vs. lush) are exposed to very different patterns of stimulation which may lead to very different states of adaptation^41,42. Moreover, these differences are accentuated among individuals occupying different niches in an increasingly specialized and technical society (e.g., an artist, radiologist, video game player, or scuba diver). Perceptual learning and expertise have been widely studied and depend on many factors^43,44,45. But one of these may be simple exposure^46,47. For example, one account of the "other race" effect, in which observers are better at distinguishing faces with our own ethnicity, is because they are adapted to the faces they commonly encounter^48,49. Adaptation provides a common metric for evaluating the impact of a sensitivity change Vs. stimulus change on perception, and thus for predicting how two different observers might experience the same world Vs. placing the same observer in two different worlds.

Evaluating the Long-term Consequences of Adaptation

Actually adapting observers and then measuring how their sensitivity and perception change is a well-established and extensively investigated psychophysical technique. However, these measurements are typically restricted to short term exposures lasting minutes or hours. Increasing evidence suggests that adaptation also operates over much longer timescales that are much more difficult to test empirically^{50,51,52,53,54}. Simulating adaptation has the advantage of pushing adaptation states to their theoretical long-term limits and thus exploring timescales that are not practical experimentally. It also allows for testing the perceptual consequences of gradual changes such as aging or a progressive disease.

Evaluating the Potential Benefits of Adaptation

A related problem is that while many functions have been proposed for adaptation, performance improvements are often not evident in studies of short-term adaptation, and this may in part be because these improvements arise only over longer timescales. Testing how well observers can perform different visual tasks with images adapted to simulate these timescales provides a novel method for exploring the perceptual benefits and costs of adaptation²⁹.

Testing Mechanisms of Visual Coding and Adaptation

The simulations can help to visualize and compare both different models of visual mechanisms and different models of how these mechanisms adjust their sensitivity. Such comparisons can help reveal the relative importance of different aspects of visual coding for visual performance and perception.

Adapting Images to Observers

To the extent that adaptation helps one to see better, such simulations provide a potentially powerful tool for developing models of image processing that can better highlight information for observers. Such image enhancement techniques are widespread, but the present approach is designed to adjust an image in ways in which the actual brain adjusts, and thus to simulate the actual coding strategies that the visual system evolved to exploit. Preprocessing images in this way could in principle remove the need for observers to visually acclimate to a novel environment, by instead adjusting images to match the adaptation states that observers are currently in^26,29.

It may seem unrealistic to suggest that adaptation could in practice discount nearly fully a sensitivity change from our percepts, yet there are many examples where percepts do appear unaffected by dramatic sensitivity differences⁵⁵, and it is an empirical question how complete the adaptation is for any given case — one that adapted images could also be used to address. In any case, if the goal is to visualize the perceptual experience of an observer, then these simulations arguably come much closer to characterizing that experience than traditional simulations based only on filtering the image. Moreover, they provide a novel tool for predicting and testing the consequences and functions of sensory adaptation²⁹. Again this adaptation is ubiquitous in sensory processing, and similar models could be exploited to explore the impact of adaptation on other visual attributes and other senses.