Overconfidence Effect: When Certainty Runs Ahead of Accuracy

Overconfidence is not one finding but three, and keeping them apart is the only way to read the literature. Don Moore and Paul Healy set out the distinction in 2008. Overestimation is thinking you performed better than you did. Overplacement is thinking you performed better than other people, the better-than-average pattern. Overprecision is holding beliefs that are too tight: a range that should contain the answer nine times in ten contains it half the time. The three can point in opposite directions in the same task, which is why a single number for "how overconfident people are" does not exist.

The mechanism behind overprecision, the most stubborn of the three, is a search problem. When you look for a value, your mind produces a best guess and then reasons that support it. Reasons that would stretch the range in either direction are not generated unless someone asks for them, so the interval is built around the anchor rather than around the uncertainty. Asher Koriat, Sarah Lichtenstein and Baruch Fischhoff tested this directly in 1980: people who were asked to write down one reason contradicting their chosen answer became better calibrated, while writing a supporting reason, or both at once, did not help.

Overestimation depends on the task, and in a way that surprises people. Moore and Healy found that on hard tasks participants overestimated their own scores while placing themselves below others, and on easy tasks they underestimated their scores while placing themselves above others. This is the hard-easy effect, and it is why a demo built from difficult questions shows a larger gap than one built from easy ones. The ten questions in the demo above are deliberately unfamiliar, so a prediction above the score is the expected result and not a verdict on the person who made it.

Feedback is the other half of the story. Weather forecasters and experienced bridge players, who get scored on thousands of probabilistic judgments, are close to calibrated. Most judgments in work and life are never scored, so there is nothing to correct the gap, and confidence is rewarded socially whether or not it is accurate.

The task everything else is measured against comes from Marc Alpert and Howard Raiffa, whose "A progress report on the training of probability assessors" circulated at Harvard from 1969 and was published in Kahneman, Slovic and Tversky's Judgment under Uncertainty in 1982. They asked students for quantities they could not look up, such as the number of eggs produced in the United States in 1965, and did not ask for a single answer. Instead they asked for fractiles of each student's own belief: a low value such that there was only a 1 percent chance the truth fell below it, a high value with only a 1 percent chance above it, and the 25th, 50th and 75th in between.

The ranges came out far too narrow. The interval between the 1st and 99th fractiles, which should have contained the true value 98 times in 100, contained it about 60 percent of the time, and the interquartile range, which should have caught half of the answers, caught about a third. Alpert and Raiffa's response was an instruction printed in the paper, that respondents should spread their extreme fractiles and admit what they did not know. Repeating the task after that instruction improved the hit rate only modestly.

J. Edward Russo and Paul Schoemaker carried the test into management in 1992, asking roughly 2,000 managers for ranges wide enough to be 90 percent sure on ten questions in or near their own industries. Instead of missing one answer in ten, most groups missed between four and six, and the pattern held among advertising, oil and investment professionals answering questions about their own fields.

Moore and Healy's 2008 paper in Psychological Review is the reconciliation rather than a new demonstration. Across a series of quizzes they showed that overestimation and overplacement move in opposite directions as the task gets harder, that overprecision persists in both, and that much of the apparent conflict in 40 years of overconfidence research came from treating the three as one effect.

Overconfidence is visible wherever an estimate gets written down and later compared with an outcome. The rows below are illustrations, not findings from the studies cited on this page.

Calibration responds to procedure rather than to resolve, and the procedures are specific.

• Write down one reason your answer could be wrong. In Koriat, Lichtenstein and Fischhoff's 1980 experiment, listing a contradicting reason improved calibration; listing a supporting reason did not.
• Widen the range further than feels right. If a 90 percent range feels comfortable, the published hit rates say it is still too narrow, so stretch both ends before you commit.
• Use outside numbers. For anything that has happened before, the distribution of past cases beats a judgment about this case: what did the last ten projects, renovations or hires actually cost.
• Keep a score. Record predictions with dates and ranges, then check them. Calibration improves where feedback exists, which is why forecasters who are scored daily are close to accurate.
• Ask what the estimate is for. An interval that will be used to plan needs a hit rate attached: a 50 percent range and a 90 percent range are different promises.
• Separate the three effects before arguing about them. "I am better than most drivers" is overplacement; "the job takes six weeks" is overprecision. The evidence for one is not evidence for the other.

Overconfidence is the family name for several effects that are often used interchangeably.

• It is not the Dunning-Kruger effect, which is a claim about how the confidence gap changes with skill, largest among the least skilled. The overconfidence effect is the gap itself, measured across the whole range of ability.
• It is not illusory superiority, the finding that most people rate themselves above average on desirable traits. That is overplacement, a comparison with other people; overprecision and overestimation need no comparison at all.
• It is not the planning fallacy, which is the specific tendency to underestimate how long a task will take and what it will cost. Underestimating a duration is one case of miscalibration, with its own literature and its own repair in reference-class forecasting.
• It is not the hard-easy effect, which describes how the direction of the error flips with task difficulty: overestimation on hard tasks, underestimation on easy ones.