Social Desirability Bias: Why Surveys Get Flattering Answers

Answering a question about yourself involves two jobs: working out the answer, and deciding what to say. The second job takes account of who is listening. If the honest answer would make you look careless, selfish, unhealthy or unkind, the reported answer drifts toward the version that would not, and the drift is largest where the subject is sensitive and the audience is closest.

Delroy Paulhus distinguished two parts of this in 1984. Impression management is the deliberate part: editing an answer for an audience. Self-deceptive enhancement is the part the person is not aware of, an honestly held but inflated view of themselves. The practical consequence is that the two respond to different things. Anonymity and self-administration can reduce impression management, because they remove the audience; they do nothing about a belief the person actually holds.

Roger Tourangeau and Ting Yan reviewed the survey evidence in 2007 and reached a conclusion that matters for anyone reading self-reported numbers: misreporting on sensitive topics is common and largely situational. It depends on whether the respondent has anything embarrassing to report, on whether an interviewer is present, on how the question is worded, and on who else might see the answer. It is better described as editing than as a personality trait.

The size of the editing can be measured against records. Hadaway, Marler and Chaves compared what Americans told pollsters about church attendance in 1993 with attendance counted in sampled congregations and dioceses, and estimated real attendance at about half the self-reported level. The same gap appears wherever a validated record exists: reported voting exceeds verified turnout, reported exercise exceeds measured activity, reported drinking falls short of alcohol sales.

A longstanding instrument in this area, the Marlowe-Crowne scale, is also a caution. Crowne and Marlowe built it in 1960 from 33 true-false statements, 18 of them unlikely virtues to claim and 15 of them common faults to deny, intending to measure the need to answer in culturally sanctioned ways. A high score was long treated as a warning that the rest of a questionnaire was inflated. The problem is that the need to be approved of is itself a stable trait, so a high score can be an accurate self-description rather than evidence of distortion, which is one reason survey methodologists have moved toward changing the procedure rather than scoring the respondent.

Douglas Crowne and David Marlowe published "A new scale of social desirability independent of psychopathology" in the Journal of Consulting Psychology in 1960. Earlier measures of faking good had been drawn from clinical inventories, so a high score could mean distortion or could mean genuine distress. Crowne and Marlowe assembled statements with no clinical content, some describing approved behaviors that are unlikely to be true of anyone and others describing disapproved behaviors that are true of almost everyone, and had judges rate their desirability. The 33 items that survived became the scale that is still in use: 18 keyed true, so that agreeing claims an improbable virtue, and 15 keyed false, so that disagreeing denies an ordinary fault.

The experimental work that the demo on this page follows came later and changed the method rather than scoring the person. Roger Tourangeau and Tom Smith reported it in Public Opinion Quarterly in 1996. Working with an area probability sample of adults in Cook County, Illinois, they assigned respondents at random to one of three modes: an interviewer asking the questions and entering the answers, the respondent reading the questions on a laptop and entering their own answers, or the respondent hearing the questions through headphones and answering on the laptop. The questions covered sexual behavior and illicit drug use. The procedures were otherwise identical, and about 365 interviews were completed in the first wave.

Self-administration raised the reports. Compared with the interviewer condition, respondents who answered through headphones on the laptop were 1.48 times as likely to report ever using marijuana and 1.81 times as likely to report ever using cocaine, with similar ratios across shorter reporting periods. The difference between the two self-administered modes was not significant for any of the drug variables, which points at the interviewer rather than at the computer.

The same study showed that wording does its own work: when the answer options for a question about numbers of sexual partners emphasized the low end of the range, respondents reported fewer partners than when the options emphasized the high end. Nothing about the respondents differed between conditions. What differed was who would hear the answer, and what the form made easy to say.

The bias runs wherever a number comes from someone describing themselves. The rows below are illustrations, not findings from the studies cited here.

Because the bias is produced by the situation, the counters are changes to the situation rather than appeals to honesty.

• Remove the audience. Answers entered by the respondent rather than given to an interviewer raised reports of sensitive behavior by about half again in the 1996 experiment's audio condition, and the two self-administered modes did not differ significantly, which locates the effect in the presence of an interviewer.
• Say what happens to the data, in specifics. Who sees it, at what level of aggregation, for how long. A general promise of confidentiality is weaker than a described procedure.
• Word the question so the awkward answer is easy. A longer reference period, a forgiving preamble and response options that do not make the honest answer the extreme one all help.
• Ask about behavior, not about virtue. What someone did last Tuesday is harder to dress up than what they usually do.
• Validate against something. Records, device data, sales figures or a known base rate turn a self-report into a measurement with an error bar.
• When you read a survey, ask who collected it and how. Mode, sponsor and wording change the number, sometimes by more than the effect being reported.

Social desirability bias is mostly about the answer that gets reported. Its neighbors are about the belief behind the answer and about behavior under observation.

• It is not the same as illusory superiority, the sincere belief that you are above average on desirable traits, though the two meet. Paulhus's model puts a sincerely held flattering self-view under self-deceptive enhancement, one of the two components of socially desirable responding. What this page is mostly about is the other component, impression management: the editing done for an audience, which anonymity can reduce while a genuinely held belief stays where it is.
• It is not the self-serving bias, which credits yourself for successes and blames circumstances for failures. That shapes the explanation; social desirability shapes the disclosure.
• It is not the observer effect, where being watched changes the behavior itself. Here the behavior has already happened and what changes is the account of it.
• It is not lying, though it includes some. Most of the effect is editing at the margin: a rounded number, a vaguer answer, the omission of a detail nobody asked twice about.