Source: Peter Mende-Siedlecki & Jay Van Bavel—New York University
When we are considering a tough choice between two or more attractive options, we often end up actively weighing the pros and cons of each alternative. By reflecting on their advantages and disadvantages, we attempt to fit a complex, subjective decision into an orderly set of criteria. However, research in psychology suggests that this sort of introspective approach might not always yield the most optimal outcomes.1
In other words, sometimes thinking hard about a problem or a choice may not produce desired results. Similar results have been demonstrated in the domains of emotion (participants who ruminated about a bad mood showed less mood improvement than participants who were merely distracted from their mood;2 and memory (verbalizing the details of a criminal’s face led to poorer recognition in a photo array of possible suspects.3 Furthermore, Wilson and colleagues observed that reflecting on the reasons behind one’s attitudes (i.e., considering “why” one feels a certain way) can disrupt the consistency between attitudes and behavior, and can even change attitudes.4
Why might this be the case? Wilson and colleagues speculate that often we don’t typically have a very good understanding of why we actually feel the way we do.5 Upon introspection of our feelings, we may hone in on irrelevant but salient details that may offer plausible explanations, but may also have little direct influence on our actual attitudes. Wilson and Schooler devised an experiment designed to test this possibility in the domain of subjective preferences. Specifically, they compared participants’ evaluations of a series of jams with experts’ evaluations, and tested whether asking participants to analyze the reasons for their choices would have a negative impact on their evaluations.
This investigation capitalizes on what might be termed the unreliability of introspection, or the introspection illusion. Evidence in social psychology suggests that we have very little direct access into the mental processes giving rise to our perceptions and behavior.5 When we attempt to introspect on these processes, we often miss the mark—providing plausible but inaccurate post-hoc explanations based on implicit causal theories of what is most likely to have affected our mental states.
1. Participant Recruitment
2. Data Collection
3. Data Analysis
Decision-making occurs on a continuum of thoughts with varying levels of consciousness. In most cases, people have no idea why they feel the way they do.
At one extreme, individuals may think very little—making decisions quickly and habitually—like pulling out already prepared foods for dinner.
In contrast, choosing where to dine out results in thinking too much—reading through reviews to weigh all of the pros and cons because their friend also asks for an explanation regarding the final selection.
Upon introspection—without knowing why—the individual honed in on irrelevant yet salient details that made sense, but in fact, did not lead to the best decision, at least by experts’ standards.
This example capitalizes on the unreliability of thinking consciously—also called the introspection illusion—where an attempt to understand one’s own perceptions and behaviors results in erroneous post-hoc explanations.
Based on the classic experiment by Wilson and Schooler, this video demonstrates whether asking participants to think about reasons for rating different jams interfered with their preferences compared to those who were simply asked to provide rankings.
In this experiment, two groups of participants—reasons-analysis and controls—are asked to taste test different jams that have been previously analyzed by experts, and then evaluate each one based on their preference.
During the tasting phase, those in the reasons-analysis group are asked to analyze the way they feel about each one by listing five reasons for liking or disliking it after tasting. This step allows participants to organize their thoughts and encourage better preparation for providing final ratings.
Each participant in the control group, on the other hand, is asked to complete a filler questionnaire and list reasons why they chose their college major. In either case, these first forms are discarded.
Subsequently, participants are given a second preference questionnaire by a different experimenter blinded to their condition. They are asked to anonymously evaluate each jam on a 9-point scale ranging from 1-disliked to 9-liked.
In this experiment, the dependent variable is the average preference rating of each jam as a function of group, reasons-analysis versus controls.
Participants who spend time deliberating about the reasons supporting their preferences are expected to show much less correspondence with an objective criterion of quality—the expert ratings—compared to controls. Such findings would support the illusion of introspection, where individuals change decisions because they aren’t truly aware of their attitudes.
Prior to the experiment, conduct a power analysis to recruit a sufficient number of individuals for a study entitled “Jam Taste Test”. Call those enlisted and instruct them not to eat anything for 3 hrs prior to the study.
To continue preparation, acquire five brands of jam that vary in overall quality based on expert opinions, and randomly label them A through E. Now, set up the testing room.
Arrange the five brands on the table, and for each participant, record the order and what label each specific jam received.
Then, place a spoonful of each jar’s content directly in front.
To begin, escort a participant into the testing room, and have them sign the consent forms.
Explain that the purpose of the study is to evaluate different kinds of jams under different conditions and they have been randomly chosen to taste on spoons. Remind them to rate their liking preference for each one.
For participants randomly assigned to the reasons-analysis group, provide additional written instructions stating that they need to list their reasons behind liking or disliking each one to organize their thoughts prior to the final evaluations. Note, for control participants, do not provide any further instructions.
Have each participant sit at the table where the five jams were arranged, ask them to start their evaluation, and leave the room.
Following the tasting period, administer a questionnaire and remind them that the purpose of it is to simply organize their thoughts and that they do not need to hand it in. For the control participants, administer a filler questionnaire. Exit the room while the forms are being filled out.
Upon return, explain that this information will no longer be needed and throw it in the trash can. Now, introduce a second experimenter who is unaware of the participants’ assigned conditions.
To capture participants’ ratings, have this new researcher administer preference forms, in which they should place into a covered box on the table when they finish. Leave the room while they record their evaluations.
To end the experiment, fully debrief all participants and discuss the true purpose and procedures of the study.
To visualize the data, first plot the mean ratings across the five jams, in order of the experts’ ratings, and split by condition—controls versus reasons. Notice that control participants were more accurate compared to those in the reasons group, as their scores patterned the professional rankings.
To quantify the amount of agreement, compute correlation values between each participant’s and the expert’s ratings and plot the averages by condition. The average correlation in the control group was near 60%, whereas the mean value in the reasons condition was less than 20%, not significantly different from zero.
Thus, participants who spent time deliberating about reasons supporting their own evaluations showed less correspondence to an objective criterion, which supports the introspection illusion: Thinking too much does impair decision-making!
Now that you are familiar with how introspective thoughts affect decision making in an experimental setting, let’s look at other cases where deliberating between choices leads to suboptimal results.
When it comes to large purchases, like buying a car, many people take their time to weigh the options and consider the pros and cons at every angle, before committing to such an investment.
Although there may be economic advantages, psychologists have found that consumers who exhaustively deliberate these tough choices end up feeling less satisfied with their purchase—essentially showing buyer’s remorse.
Interestingly, older consumers tend to vacillate much less in the car buying process by sticking to brands that they have remained loyal to over the years.
In addition, relationships call for moments of intensive introspection, but all of this deep thought can cause people to make decisions that they also later regret. Researchers postulate that this happens because the introspective process causes them to lose touch with their feelings, leading them to act based on a false impression of their true emotions.
You’ve just watched JoVE’s video on how thinking too much impairs decision-making. Now you should have a good understanding of how to design, conduct, and analyze an experiment to study how our inner thoughts influence behavior, and how introspecting on one’s preferences can be applied to consumer behavior and interpersonal relationships.
Thanks for watching!
In the original Wilson and Schooler investigation, the authors observed that asking participants to think about their reasons for their evaluations did indeed change their ratings of the set of jams, as compared to the ratings given by control participants.
Critically, when comparing against the objective criterion (e.g., experts ratings), the average correlation between “reasons-analysis” participants’ ratings and experts’ ratings was significantly lower than the average correlation between control participants’ ratings and experts’ ratings. Moreover, while this average correlation between participants’ and experts’ ratings was significantly greater than zero in the control condition, in the “reasons-analysis condition,” it was not.
Figure 1: Mean liking ratings for the five jams as a function condition. Average liking ratings provided by participants in the control (left) and reasons conditions (right) are displayed for all five jams. Jam 1 was the top ranked jam, based on experts’ ratings. Jams 2, 3, 4, and 5 were the 11th-, 24th-, 32nd-, and 45th-ranked jams, respectively. Participants were more accurate in rating the jam in the control condition (left).
Figure 2: Average correlation between participants’ ratings and experts’ ratings as a function of condition. Bars represent the average correlation between participants’ ratings of jam liking and experts’ rankings, as a function of whether they were in the control (left) or reasons condition (right). Within-participant correlations were Fisher-transformed and then compared against zero and against each other. The average correlation in the control condition was different from zero, and significantly stronger than the average correlation in the reasons condition, which was not different from zero.
Based on these results, the authors concluded that while the control participants formed jam preferences that were very similar to an objective criterion of quality (e.g., experts’ ratings), participants who spent time deliberating about the reasons supporting their evaluations showed much less correspondence with this criterion. The authors suggest that these participants’ preferences were influenced by the process of introspection, which likely caused them to focus on salient, but ultimately irrelevant attributes of the jams.
The results of this study have clear implications in the marketing domain, and for consumers in general. Introspecting on one’s preferences—specifically on the reasons supporting those preferences—may lead to less-than-optimal decision-making. In other words, a consumer who exhaustively deliberates on the pros and cons of a choice may end up feeling less satisfied with the ultimate results of that choice.7
These principles may also be extrapolated to other complex decision domains (e.g., legal, economic, and even interpersonal decision-making), though testing the boundary conditions of these results is critical. One could examine how manipulating various aspects of the choice architecture (e.g., the number of alternatives the participant is considering, the objective range of alternatives, the average quality of alternatives, etc.) or the instructions (e.g., asking participants to focus on abstract versus concrete reasons, pitting a “reasons analysis” against a “feelings analysis”, etc.) might exaggerate or attenuate this pattern of results. One could also test the degree to which these results generalize to other decision domains or attitude objects, or consider individual differences in these effects. For example, one might ask for whom will introspection not have a deleterious effect? One possibility is that people who enjoy thinking will prefer to engage in introspective and it might even lead to better outcomes. Of course, this is a question for future research.
Decision-making occurs on a continuum of thoughts with varying levels of consciousness. In most cases, people have no idea why they feel the way they do.
At one extreme, individuals may think very little—making decisions quickly and habitually—like pulling out already prepared foods for dinner.
In contrast, choosing where to dine out results in thinking too much—reading through reviews to weigh all of the pros and cons because their friend also asks for an explanation regarding the final selection.
Upon introspection—without knowing why—the individual honed in on irrelevant yet salient details that made sense, but in fact, did not lead to the best decision, at least by experts’ standards.
This example capitalizes on the unreliability of thinking consciously—also called the introspection illusion—where an attempt to understand one’s own perceptions and behaviors results in erroneous post-hoc explanations.
Based on the classic experiment by Wilson and Schooler, this video demonstrates whether asking participants to think about reasons for rating different jams interfered with their preferences compared to those who were simply asked to provide rankings.
In this experiment, two groups of participants—reasons-analysis and controls—are asked to taste test different jams that have been previously analyzed by experts, and then evaluate each one based on their preference.
During the tasting phase, those in the reasons-analysis group are asked to analyze the way they feel about each one by listing five reasons for liking or disliking it after tasting. This step allows participants to organize their thoughts and encourage better preparation for providing final ratings.
Each participant in the control group, on the other hand, is asked to complete a filler questionnaire and list reasons why they chose their college major. In either case, these first forms are discarded.
Subsequently, participants are given a second preference questionnaire by a different experimenter blinded to their condition. They are asked to anonymously evaluate each jam on a 9-point scale ranging from 1-disliked to 9-liked.
In this experiment, the dependent variable is the average preference rating of each jam as a function of group, reasons-analysis versus controls.
Participants who spend time deliberating about the reasons supporting their preferences are expected to show much less correspondence with an objective criterion of quality—the expert ratings—compared to controls. Such findings would support the illusion of introspection, where individuals change decisions because they aren’t truly aware of their attitudes.
Prior to the experiment, conduct a power analysis to recruit a sufficient number of individuals for a study entitled “Jam Taste Test”. Call those enlisted and instruct them not to eat anything for 3 hrs prior to the study.
To continue preparation, acquire five brands of jam that vary in overall quality based on expert opinions, and randomly label them A through E. Now, set up the testing room.
Arrange the five brands on the table, and for each participant, record the order and what label each specific jam received.
Then, place a spoonful of each jar’s content directly in front.
To begin, escort a participant into the testing room, and have them sign the consent forms.
Explain that the purpose of the study is to evaluate different kinds of jams under different conditions and they have been randomly chosen to taste on spoons. Remind them to rate their liking preference for each one.
For participants randomly assigned to the reasons-analysis group, provide additional written instructions stating that they need to list their reasons behind liking or disliking each one to organize their thoughts prior to the final evaluations. Note, for control participants, do not provide any further instructions.
Have each participant sit at the table where the five jams were arranged, ask them to start their evaluation, and leave the room.
Following the tasting period, administer a questionnaire and remind them that the purpose of it is to simply organize their thoughts and that they do not need to hand it in. For the control participants, administer a filler questionnaire. Exit the room while the forms are being filled out.
Upon return, explain that this information will no longer be needed and throw it in the trash can. Now, introduce a second experimenter who is unaware of the participants’ assigned conditions.
To capture participants’ ratings, have this new researcher administer preference forms, in which they should place into a covered box on the table when they finish. Leave the room while they record their evaluations.
To end the experiment, fully debrief all participants and discuss the true purpose and procedures of the study.
To visualize the data, first plot the mean ratings across the five jams, in order of the experts’ ratings, and split by condition—controls versus reasons. Notice that control participants were more accurate compared to those in the reasons group, as their scores patterned the professional rankings.
To quantify the amount of agreement, compute correlation values between each participant’s and the expert’s ratings and plot the averages by condition. The average correlation in the control group was near 60%, whereas the mean value in the reasons condition was less than 20%, not significantly different from zero.
Thus, participants who spent time deliberating about reasons supporting their own evaluations showed less correspondence to an objective criterion, which supports the introspection illusion: Thinking too much does impair decision-making!
Now that you are familiar with how introspective thoughts affect decision making in an experimental setting, let’s look at other cases where deliberating between choices leads to suboptimal results.
When it comes to large purchases, like buying a car, many people take their time to weigh the options and consider the pros and cons at every angle, before committing to such an investment.
Although there may be economic advantages, psychologists have found that consumers who exhaustively deliberate these tough choices end up feeling less satisfied with their purchase—essentially showing buyer’s remorse.
Interestingly, older consumers tend to vacillate much less in the car buying process by sticking to brands that they have remained loyal to over the years.
In addition, relationships call for moments of intensive introspection, but all of this deep thought can cause people to make decisions that they also later regret. Researchers postulate that this happens because the introspective process causes them to lose touch with their feelings, leading them to act based on a false impression of their true emotions.
You’ve just watched JoVE’s video on how thinking too much impairs decision-making. Now you should have a good understanding of how to design, conduct, and analyze an experiment to study how our inner thoughts influence behavior, and how introspecting on one’s preferences can be applied to consumer behavior and interpersonal relationships.
Thanks for watching!