Summary
In this protocol we describe a method of operant learning using sensory stimuli as a reinforcer in the mouse. It requires no prior training or food restriction, and it allows the study of motivated behavior without the use of a pharmacological or natural reinforcer such as food.
Abstract
Operant methods are powerful behavioral tools for the study of motivated behavior. These ‘self-administration’ methods have been used extensively in drug addiction research due to their high construct validity. Operant studies provide researchers a tool for preclinical investigation of several aspects of the addiction process. For example, mechanisms of acute reinforcement (both drug and non-drug) can be tested using pharmacological or genetic tools to determine the ability of a molecular target to influence self-administration behavior1-6. Additionally, drug or food seeking behaviors can be studied in the absence of the primary reinforcer, and the ability of pharmacological compounds to disrupt this process is a preclinical model for discovery of molecular targets and compounds that may be useful for the treatment of addiction3,7-9. One problem with performing intravenous drug self-administration studies in the mouse is the technical difficulty of maintaining catheter patency. Attrition rates in these experiments are high and can reach 40% or higher10-15. Another general problem with drug self-administration is discerning which pharmacologically-induced effects of the reinforcer produce specific behaviors. For example, measurement of the reinforcing and neurological effects of psychostimulants can be confounded by their psychomotor effects. Operant methods using food reinforcement can avoid these pitfalls, although their utility in studying drug addiction is limited by the fact that some manipulations that alter drug self-administration have a minimal impact on food self-administration. For example, mesolimbic dopamine lesion or knockout of the D1 dopamine receptor reduce cocaine self-administration without having a significant impact on food self-administration 12,16.
Sensory stimuli have been described for their ability to support operant responding as primary reinforcers (i.e. not conditioned reinforcers)17-22. Auditory and visual stimuli are self-administered by several species18,21,23, although surprisingly little is known about the neural mechanisms underlying this reinforcement. The operant sensation seeking (OSS) model is a robust model for obtaining sensory self-administration in the mouse, allowing the study of neural mechanisms important in sensory reinforcement24. An additional advantage of OSS is the ability to screen mutant mice for differences in operant behavior that may be relevant to addiction. We have reported that dopamine D1 receptor knockout mice, previously shown to be deficient in psychostimulant self-administration, also fail to acquire OSS24. This is a unique finding in that these mice are capable of learning an operant task when food is used as a reinforcer. While operant studies using food reinforcement can be useful in the study of general motivated behavior and the mechanisms underlying food reinforcement, as mentioned above, these studies are limited in their application to studying molecular mechanisms of drug addiction. Thus, there may be similar neural substrates mediating sensory and psychostimulant reinforcement that are distinct from food reinforcement, which would make OSS a particularly attractive model for the study of drug addiction processes. The degree of overlap between other molecular targets of OSS and drug reinforcers is unclear, but is a topic that we are currently pursuing. While some aspects of addiction such as resistance to extinction may be observed with OSS, we have found that escalation 25 is not observed in this model24. Interestingly, escalation of intake and some other aspects of addiction are observed with self-administration of sucrose26. Thus, when non-drug operant procedures are desired to study addiction-related processes, food or sensory reinforcers can be chosen to best fit the particular question being asked.
In conclusion, both food self-administration and OSS in the mouse have the advantage of not requiring an intravenous catheter, which allows a higher throughput means to study the effects of pharmacological or genetic manipulation of neural targets involved in motivation. While operant testing using food as a reinforcer is particularly useful in the study of the regulation of food intake, OSS is particularly apt for studying reinforcement mechanisms of sensory stimuli and may have broad applicability to novelty seeking and addiction.
Protocol
1. Write program to run operant test sessions using varied visual and auditory stimuli as a reinforcer
- For fixed ratio (FR) sessions: make sessions one hour in length with house light and fan turned on during session. For progressive ratio sessions, make sessions two hours in length. Have both levers extended for the duration of the session and counterbalance which lever is designated “active” vs. “inactive” across animals (lever assignment for each animal never changes).
- Code the program in such a way that each reinforcer is varied according to the following parameters:
- each reinforcer has one of the following durations chosen randomly: 2, 4, 6, or 8 sec.
- each reinforcer has one of the following stimulus lamp flash rates chosen randomly: 0.625, 1.25, 2.5, or 5 Hz, each with a 50% duty cycle.
- each stimulus lamp flash is randomly on either the left or the right side of the chamber.
- provide an auditory stimulus for the duration of the reinforcer; in our laboratory, we trigger an infusion pump which provides sound approximately 3 dB above the background noise in the chamber.
- Code the program to display the following values in real time: 1. Number of active lever presses, 2. Number of inactive lever presses, 3. Number of reinforcers, 4. Time (in 0.1 second increments).
2. Handle animals (3 day procedure)
- After acclimation to the animal facility, begin handling the animals. This will habituate the animals to being picked up and transported.
- Start by placing gloved hands into the cage and letting them sit for 90 sec. If every mouse has not investigated your hands in that time, gently move hands toward mice and wait for each one to sniff and/or contact your hands before proceeding.
- Carefully pick up each mouse one by one by the base of the tail and place it on your hand, quickly lifting it up and bringing it back down to allow the mouse to walk off of your hand back into the cage.
- Repeat for each mouse 5-10 times, depending on the demeanor of the mouse. The final time this is done, hold the mouse high (with it standing on your hand) for a period of time, depending on the day. Day 1: 5 sec, Day 2: 10 sec, Day 3: 15 sec. Ensure that each mouse meets this criterion for the day. On Day 1, change gloves between cages.
- Beginning on Day 2, stroke the animal’s back while the mouse is on your hand. Also on Day 2, begin weighing the animals daily. Mark each animal’s tail with a Sharpie to denote subject number.
- If injections will be given during the experiment, begin habituating mice to injections on Day 2. This should be done after all mice in the cage have been handled and met the criterion for the day.
3. Clean and test equipment
- Wash bottom pan with hot water.
- Clean operant chamber walls and floors with 30% ethanol.
- Run a test program that turns on house light, fan, and stimulus lights, extends levers, and records lever presses.
- Ensure that all lights and fans are operating correctly and test the program to ensure that all lever presses are recorded.
- Clean levers with 30% ethanol.
4. Perform operant session (sessions should be 5-6 days/week at the same time of day)
- Weigh each mouse.
- Load the program into each chamber and annotate the experiment appropriately. The active lever should be counterbalanced between animals (i.e., the active lever is assigned to the left lever for half of the mice; the right lever is active for the other half), but the active lever side never changes once mouse a mouse has been assigned.
- Transport each mouse to its designated chamber, close the chamber, and start the session.
- After the session ends, promptly remove the mouse and re-mark the tail.
- Clean chambers as described in Section 3.
- Analyze data for number of active and inactive lever presses. Number of reinforcers and/or lever accuracy (% active lever presses) can also be reported. If mice are going to be tested for the effect of a treatment on OSS, ensure that all mice to be tested have met acquisition criteria (for example, >20 active lever presses and >65% active lever presses for final three FR-1 sessions) prior to beginning treatment.
- After acquisition of FR-1 responding, the schedule of reinforcement can be changed (for example, a higher fixed ratio, progressive ratio, random ratio, etc.).
5. Representative Results
An example of OSS acquisition by male C57Bl/6J mice is shown in Figure 1 (reproduced from 23). Control mice underwent identical conditions, except that lever presses on either lever had no consequence. Another cohort of mice is shown in Figure 2. In this experiment, one group of mice received OSS reinforcement, while another group received food reinforcement. We have found that under conditions of ad libitum access to food, both fixed and progressive ratio responding are similar between OSS and responding for 10% Ensure (Figure 2, A and B). This allows for effective comparisons of a manipulation on two different reinforcement types (sensory and food) that avoids potential confounds resulting from hunger state or differences in response rate.
Figure 1: Lever presses by OSS mice and controls. OSS mice received varied visual and auditory stimuli following each active lever press (FR-1 schedule of reinforcement), while inactive lever presses had no consequence. Control mice underwent the same conditions but there was no consequence of pressing either lever (levers are denoted 1 and 2 and are counterbalanced across animals in the same manner that active and inactive levers are counterbalanced for OSS mice). Active lever pressing by OSS mice was increased relative to inactive lever pressing (*p<0.05, **p<0.01) and to non-reinforced lever pressing by controls (n=7 8, $p<0.05, $$p<0.01). Figure reproduced from 23.
Figure 2: Lever presses by OSS mice and mice responding for food. A.) Mice responded on an FR-1 schedule of reinforcement for OSS stimuli or food reinforcer (10% Ensure). All mice had ad libidum access to food for the duration of the experiments. B.) After FR-1 sessions, mice were advanced to a progressive ratio (PR) schedule of reinforcement for five days. Data represent the means of values from days four and five for each animal. The final ratio completed is reported on the right Y-axis and refers to the number of responses required to obtain the corresponding reinforcer (i.e. 30 responses are required to obtain the 10th reinforcer after previously earning nine reinforcers).
Discussion
Operant sensation seeking is a useful alternative to intravenous drug self-administration when the mouse is the animal of choice. The fact that neither surgery nor catheter maintenance is required is advantageous, as these are significant technical hurdles in the mouse. OSS is also attractive because it may be measuring aspects of reinforcement distinct from other natural reinforcers such as food.
It is useful to note that behavioral measures in the mouse can be highly variable across different environmental conditions 24. This issue has arisen in our own laboratory with OSS. When the procedure was first characterized, animals were housed in a facility that had high traffic and a large number of people working in it throughout the day. During this time, mice were housed on a “normal” light cycle (lights on: 0600-1800 h; experiments run ~0800-1400). After moving to a dedicated housing facility within the Vanderbilt Neurobehavioral Laboratory, we found that progressive ratio performance of OSS was much lower than we had previously observed; responding declined over five days instead of remaining stable. This facility has much less traffic and personnel are trained to work quietly and be aware of the sensitive nature of experiments performed in the facility. We have since adjusted the mice to a light cycle that promotes wakefulness during the time of experimentation (lights on: 1500-0300 h; experiments run ~0800-1400 h) and OSS performance has returned to what we have previously noted.
The following is a description of our standard conditions for OSS experiments. Male C57Bl/6J mice are ordered at 3 weeks of age from Jackson Laboratories (Bar Harbor, ME) and housed in the modified light cycle for at least one week prior to experiments. Animals are housed in groups of 2-5 in corn cobb bedding supplemented with a small amount of cellulose (Care Fresh). Experiments are performed 5-6 days per week and cage changes are only performed prior to a day without experiments. While these are our standard conditions, we have found that female and older mice (up to 20 weeks) are also capable of acquiring OSS. We are currently examining other variables that may affect OSS performance. While it is known that static visual stimuli are capable of serving as reinforcers in mice 17, it is unknown whether the approach of enhancing the dynamics of the stimuli that we and others have employed 20,22,23 lead to increased reinforcement in the current conditions. Another variable that may affect OSS is mouse strain. Strain differences have been described in mice for a variety of behavioral and neurological measures 25-29, and differential performance on OSS and operant responding for food may provide insight into genetic underpinnings of this behavior.
Divulgaciones
The authors have nothing to disclose.
Acknowledgements
This project was supported by NIH grants DA19112 (DGW) and DA026994 (CMO). Illustration was provided by Katherine Louderback. Experiments were performed in the Vanderbilt Murine Neurobehavioral Laboratory.
Materials
| Material Name | Tipo | Company | Catalogue Number | Comment |
|---|---|---|---|---|
| Drug self-administration test package for mouse: extra-wide chamber and retractable levers | Med Associates, Inc | MED-307W-CT-D1 | Levers are ultra-sensitive (require ~2 grams force) and are mounted 2.2 cm above the floor. Yellow stimulus lamps are mounted 2 cm above each lever. | |
| Interface and software package | Med Associates, Inc | MED-SYST-16 | This is the package for up to 16 chambers. |
Referencias
- Thomsen, M., Caine, S. B. Intravenous drug self-administration in mice: practical considerations. Behav Genet. 37, 101-118 (2006).
- Koob, G. F. Animal models of motivation for drinking in rodents with a focus on opioid receptor neuropharmacology. Recent Dev Alcohol. 16, 263-281 (2003).
- Koob, G. F., Kenneth Lloyd, G., Mason, B. J. Development of pharmacotherapies for drug addiction: a Rosetta stone approach. Nat Rev Drug Discov. 8, 500-515 (2009).
- Arnold, J. M., Roberts, D. C. A critique of fixed and progressive ratio schedules used to examine the neural substrates of drug reinforcement. Pharmacol Biochem Behav. 57, 441-447 (1997).
- O’Brien, C. P., Gardner, E. L. Critical assessment of how to study addiction and its treatment: human and non-human animal models. Pharmacology & Therapeutics. 108, 18-58 (2005).
- Olsen, C. M., Duvauchelle, C. L. Prefrontal cortex D1 modulation of the reinforcing properties of cocaine. Brain Research. 1075, 229-235 (2006).
- Epstein, D. H., Preston, K. L., Stewart, J., Shaham, Y. Toward a model of drug relapse: an assessment of the validity of the reinstatement procedure. Psychopharmacology (Berl). 189, 1-16 (2006).
- Kalivas, P. W., McFarland, K. Brain circuitry and the reinstatement of cocaine-seeking behavior. Psychopharmacology (Berl). 168, 44-56 (2003).
- Stewart, J. Pathways to relapse: the neurobiology of drug- and stress-induced relapse to drug-taking. J Psychiatry Neurosci. 25, 125-136 (2000).
- Olsen, C. M., Winder, D. G. A method for single-session cocaine self-administration in the mouse. Psychopharmacology (Berl). 187, 13-21 (2006).
- Rocha, B. A. Differential responsiveness to cocaine in C57BL/6J and DBA/2J mice. Psychopharmacology (Berl). 138, 82-88 (1998).
- Caine, S. B., Negus, S. S., Mello, N. K. Method for training operant responding and evaluating cocaine self-administration behavior in mutant mice. Psychopharmacology (Berl). 147, 22-24 (1999).
- Colby, C. R., Whisler, K., Steffen, C., Nestler, E. J., Self, D. W. Striatal cell type-specific overexpression of DeltaFosB enhances incentive for cocaine. J Neurosci. 23, 2488-2493 (2003).
- Schramm-Sapyta, N. L., Olsen, C. M., Winder, D. G. Cocaine self-administration reduces excitatory responses in the mouse nucleus accumbens shell. Neuropsychopharmacology. 31, 1444-1451 (2006).
- Steiner, R. C., Hsiung, H. M., Picciotto, M. R. Cocaine self-administration and locomotor sensitization are not altered in CART knockout mice. Behav Brain Res. 171, 56-62 (2006).
- Marx, M. H., Henderson, R. L., Roberts, C. L. Positive reinforcement of the bar-pressing response by a light stimulus following dark operant pretests with no after effect. J Comp Physiol Psychol. 48, 73-76 (1955).
- Baron, A., Kish, G. B. Low-intensity auditory and visual stimuli as reinforcers for the mouse. J Comp Physiol Psychol. 55, 1011-1013 (1962).
- Stewart, J. Reinforcing effects of light as a function of intensity and reinforcement schedule. Journal of comparative and physiological psychology. 53, 187-193 (1960).
- Caggiula, A. R. Cue dependency of nicotine self-administration and smoking. Pharmacol Biochem Behav. 70, 515-530 (2001).
- Cain, M. E., Green, T. A., Bardo, M. T. Environmental enrichment decreases responding for visual novelty. Behavioural Processes. 73, 360-366 (2006).
- Thompson, T. I. Visual Reinforcement in Siamese Fighting Fish. Science. 141, 55-57 (1963).
- Blatter, K., Schultz, W. Rewarding properties of visual stimuli. Exp Brain Res. 168, 541-546 (2006).
- Olsen, C. M., Winder, D. G. Operant sensation seeking engages similar neural substrates to operant drug seeking in C57 mice. Neuropsychopharmacology. 34, 1685-1694 (2009).
- Crabbe, J. C., Wahlsten, D., Dudek, B. C. Genetics of mouse behavior: interactions with laboratory environment. Science. 284, 1670-1672 (1999).
- Crawley, J. N. Behavioral phenotypes of inbred mouse strains: implications and recommendations for molecular studies. Psychopharmacology (Berl). 132, 107-124 (1997).
- Belknap, J. K., Metten, P., Beckley, E. H., Crabbe, J. C. Multivariate analyses reveal common and drug-specific genetic influences on responses to four drugs of abuse. Trends Pharmacol Sci. 29, 537-543 (2008).
- Mozhui, K. Strain differences in stress responsivity are associated with divergent amygdala gene expression and glutamate-mediated neuronal excitability. J Neurosci. 30, 5357-5367 (2010).
- Hefner, K. Impaired fear extinction learning and cortico-amygdala circuit abnormalities in a common genetic mouse strain. J Neurosci. 28, 8074-8085 (2008).
- Elmer, G. I., Pieper, J. O., Hamilton, L. R., Wise, R. A. Qualitative differences between C57BL/6J and DBA/2J mice in morphine potentiation of brain stimulation reward and intravenous self-administration. Psychopharmacology (Berl). 208, 309-321 (2010).
