Anecdotal Fallacy: Why One Good Story Beats Ten Thousand Data Points

A survey of 10,000 owners and a friend's story are both evidence, and the survey is about ten thousand times more of it. The friend's Avera is one more data point: the failure rate becomes 601 in 10,001 rather than 600 in 10,000, still half the Corwin's. What the story has that the table lacks is not information but impact. It has a face, a highway, rain, a four-hour wait and a voice saying never again, and each detail is processed, remembered and felt in a way that a figure in a column is not.

Richard Nisbett and Eugene Borgida showed in 1975 that people neglect base rates even when they have them. Participants were told the results of two classic experiments, one in which most people accepted a great deal of electric shock before stopping and one in which most failed to help a person apparently having a seizure, then predicted how particular individuals, and they themselves, would behave. Knowing what the majority had done made almost no difference; most predicted that they and others would do what the minority had done. The authors also observed the reverse: a single individual's behavior was readily taken as a guide to what most people would do. The particular moved people; the general did not.

Nisbett, Borgida, Crandall and Reed drew the practical lesson in 1976 with an example that has been retold ever since: a person about to buy a car on the strength of consumer survey data is told at a party that a brother-in-law's model of the same car broke down repeatedly, and the survey loses. Ruth Hamill, Timothy Wilson and Nisbett then showed in 1980 that a single vivid case shifts beliefs about a whole group even when the reader is told the case is atypical: a magazine-style profile of one welfare recipient moved attitudes toward welfare recipients in general, whether participants were told she was typical, told she was unusual, or told nothing.

The mechanism is availability and weight. Concrete, emotional, personally delivered information is easier to picture and recall, so it comes to mind first when the decision is made, and it is weighted as though it were representative. Statistical summaries are abstract and pallid, in Borgida and Nisbett's phrase, and they are used in proportion to their vividness rather than their evidential value. The same asymmetry explains why a friend's opinion of a restaurant beats the ratings.

Eugene Borgida and Richard Nisbett published "The differential impact of abstract vs. concrete information on decisions" in the Journal of Applied Social Psychology in 1977. The question was practical: when students choose courses, do they use the systematic evidence available to them? Undergraduates considering psychology courses received information about a set of courses in one of two forms. Some received mean course evaluations, the averaged ratings of the students who had previously taken each course, the summary a department publishes. Others heard brief face-to-face comments about the same courses from a small number of students who had taken them.

The statistical summaries had little impact on which courses participants said they would take. The face-to-face comments, from a handful of people, had a substantial one. The authors concluded that information is used in proportion to its vividness, and that base-rate data are treated as uninformative because of their abstract, pallid nature rather than because they are doubted.

The 1980 study by Hamill, Wilson and Nisbett extended the point to social attitudes. Participants read a vivid case description of a welfare recipient modeled on a magazine profile, with one of three framings: that the case was typical of welfare recipients, that it was highly atypical, or no information about typicality at all. Attitudes toward welfare recipients as a group moved in response to the case in all three conditions, and the typicality information made little difference. The same pattern appeared with a profile of a prison guard. A single case, the authors concluded, is generalized to a population with little regard to whether it is a fair sample.

The demo on this page is a reconstruction of the car example in the design of the 1977 study: statistical information in one version, the same information plus one vivid case in the other. The cars and the survey are invented; the figures were chosen so that the story adds one case to ten thousand.

Anecdotes are how people talk, so the fallacy is built into ordinary advice. The rows below are illustrations, not findings from the studies cited on this page.

Stories cannot be made less vivid, so the counters make the statistics more concrete and the story smaller.

• Ask for the denominator. A story is one case; the question is how many cases there are and how many came out the other way. If no one can say, the story is all there is.
• Add the story to the data rather than replacing it. The friend's Avera makes the rate 601 in 10,001. Doing that sum out loud shows what the story is worth.
• Make the numbers vivid. Picture 100 owners in a room with 6 of them stranded, and 100 in another room with 12. Frequencies in small concrete groups are easier to feel than percentages.
• Notice delivery. Face-to-face, emotional and personal accounts are weighted most, as Borgida and Nisbett found; the weight comes from the delivery, not from the evidence.
• Use stories for questions, not answers. An anecdote is a good source of things to check and a poor source of conclusions.

The anecdotal fallacy is one of a cluster of errors about evidence and sample size, and its neighbors are easy to confuse with it.

• It is not the availability heuristic, which judges how common something is by how easily examples come to mind. Availability is one reason anecdotes carry weight, but the anecdotal fallacy is specifically the preference for one concrete case over a statistical summary that is already in hand.
• It is not the base rate fallacy, which ignores the prior probability of a category when judging an individual case. The two overlap in the Nisbett and Borgida work; the base rate fallacy concerns reasoning about a single case, while the anecdotal fallacy concerns reasoning from one.
• It is not insensitivity to sample size, the failure to see that small samples vary more than large ones. A person who is insensitive to sample size treats ten cases like a thousand; a person committing the anecdotal fallacy treats one vivid case like a thousand because it is vivid.