Conservatism Bias: Why Strong Evidence Moves Beliefs Too Little

The bag task has an exact answer, which is what makes the gap measurable. Because the two bags are equally likely before any chip is seen, every red chip multiplies the odds in favor of the red-heavy bag by 7 to 3, and every blue chip multiplies them by 3 to 7. Six red chips and two blue leave odds of about 30 to 1, which is a probability of 97 percent. Most people who meet the task for the first time answer somewhere between 70 and 80 percent, close to the proportion printed on the bag, as though the strength of the evidence were the same as the strength of a single chip.

The name describes the size of a revision and nothing else. It has no connection to political conservatism, and it is a different idea from the conservatism principle in accounting, which is a rule about how to report uncertain values rather than a finding about how people think.

Two explanations were on the table from the start. People may misperceive how much a single observation is worth, or they may perceive each observation correctly and then fail to combine a run of them, which is the harder arithmetic. Phillips and Edwards found that conservatism grew as each individual chip became more diagnostic, while it stayed roughly flat as the sample got longer, a pattern that points more at the combining step than at the reading of one chip. Either way the error is one of degree: the direction is usually right and the size falls short.

The finding does not generalize to every judgment, and the page would oversell it if it claimed otherwise. In the tasks Kahneman and Tversky studied a few years later, people erred in the opposite direction, treating a small and unrepresentative sample as though it settled the question and ignoring the base rate they had been given. Conservatism appears when the evidence arrives piece by piece and the prior is explicit; base-rate neglect appears when the evidence is a vivid description and the prior is a statistic in the background. Part of the laboratory effect may also be an artifact of the setting. Corner, Harris and Hahn showed that a participant who treats the experimenter as merely reliable rather than perfectly reliable should give exactly the conservative answers the classic studies recorded, which makes a cautious reading look Bayesian rather than biased.

Lawrence Phillips and Ward Edwards published "Conservatism in a simple probability inference task" in the Journal of Experimental Psychology in 1966. Participants were shown two bags of poker chips, told the exact proportion of red and blue in each, and told that one bag had been selected at random, so that their opinion before any evidence had to be 50-50. Chips were then drawn one at a time and participants estimated, after each draw, the probability that the selected bag was the predominantly red one.

The experiment varied how informative a single chip was, with bags that were 60 or 70 percent one color, and how long the run of chips grew. Results were scored with an accuracy ratio: the participant's implied weight of the evidence divided by the weight Bayes' rule gives it, where a value below 1 means the evidence was underused. Those ratios sat below 1 throughout. The paper reported two regularities that later work kept citing: conservatism increased as the diagnostic value of a single chip increased, and it remained approximately constant as the diagnostic value of the whole sample increased.

Edwards gathered this work into a 1968 chapter called "Conservatism in human information processing," which is where most readers meet the standard example: two bags of 1,000 chips, one 700 red and 300 blue, the other the reverse, with 12 draws producing 8 red chips and 4 blue. Bayes' rule puts the red-heavy bag at 97 percent. Edwards gives the typical participant's estimate as between 70 and 80 percent. The chapter is also where the two candidate explanations, misperception of a single datum and failure to aggregate several, were set out.

The background was a decade of work asking whether people are good intuitive statisticians. Peterson and Beach's 1967 review concluded that intuitive judgments of means and proportions track the statistical ones fairly closely, and that the clearest departure was in revising opinion on new evidence.

The rows below are illustrations of where opinion meets new evidence, not findings from the studies cited on this page.

Conservatism is a problem of arithmetic rather than of attitude, so the counters are procedures.

• Write the prior down before the evidence arrives, as a number. A belief that was never stated cannot be revised by a stated amount.
• Do the multiplication. Each piece of evidence has an odds ratio: how much more likely it is if the claim is true than if it is false. Multiply the ratios rather than adding impressions.
• Decide in advance what result would change your mind and by how much. A test with no pre-agreed consequence tends to leave the first opinion in place.
• Rebuild the estimate from scratch rather than editing the old one. Starting from the previous number makes the previous number an anchor.
• Check which error you are at risk of. If the evidence is a single vivid case and the base rate is a statistic you were told, the usual error runs the other way, and the correction is to weight the base rate more, not less.

Conservatism bias is a measured shortfall in the size of a belief revision. Several neighbors describe different failures, and the counters are not the same.

• It is not the backfire effect, where a correction is said to strengthen the original belief. Conservatism moves the belief in the right direction, only not far enough, and the backfire effect has itself proved hard to reproduce outside a few studies.
• It is not the base-rate fallacy, which is the opposite error: a vivid piece of case evidence swamps a prior that should have held much of the weight.
• It is not status-quo bias, a preference for leaving a situation as it is. That is about choosing, this is about believing; a person can revise a belief fully and still prefer to do nothing about it.
• It is not the Semmelweis reflex, the rejection of evidence that contradicts an established paradigm. That names a social and professional response to an unwelcome finding rather than a quantified shortfall in updating.