WATCH OUT: SOCIAL DESIRABILITY BIAS AND ‘’OFF’’ PREDICTIONS
A common concern for all pollsters is whether their interviewees are telling the truth. By giving wrong answers, participants may undermine the accuracy and reliability of poll results. But why would anyone do that, other than for being pissed off by intruding interviewers and online surveys? Well, it turns out that even when individuals feel comfortable with taking the time to fill out surveys, they may not feel comfortable expressing their true opinion, especially if this opinion isn’t socially acceptable.
What is Social Desirability Bias?
Social desirability bias occurs when survey respondents inaccurately report a socially acceptable or favorable answer to present themselves in a better light. It has been argued that the motivation behind social desirable responding may be either self-deception or impression.1 Therefore, social desirability bias shouldn’t be confused with intentional ‘’lying’’.
Social desirability bias is one of the limitations of surveys and is different than sampling bias. That is, even when your sample is representative of the population, social desirability bias may skew your findings. Particularly, social desirable responding results in the over reported desirable behaviors and underreported undesirable behaviors.
How Does Social Desirable Responding Influence Political Polls?
When there are social norms around which political opinion is the “desirable” one, individuals may feel prompted to hide their deviating opinions. For example, the results of a study conducted during the 1989 Virginia gubernatorial election showed that when asked by a Black interviewer, respondents were more likely to report supporting a Black candidate.2 Likewise, interviewees have been found to over-report their support for female candidates and for same-sex marriage when they are asked openly, compared to when they are asked unobtrusively.3,4
Similarly, it has been suggested that the results of 2016 Presidential Elections of United States were unpredicted by polls due to social desirable responding. Though both candidates were presented negatively in the media, the media coverage of Trump was much more negative than that of Clinton. Trump has been repeatedly associated with White Supremacy, Ku Klux Klan and misogyny in the media, which led to criticisms toward both Trump and his supporters. Studies with different methodologies have corroborated that social desirability bias accounts for less reported support for Trump.5,6
How to Overcome Social Desirability Bias?
1) Providing Anonymity
Though anonymous responding has been found to decrease socially desirable responding7, its implications on the online data has been less conclusive. Meta-analyses are used to take an aggregate of research that has been conducted to test whether online surveys yield different results than surveys conducted on pen and pencil, on telephone or conducted face-to-face. Though earlier meta-analyses8,9 suggested that online surveys result in less distorted results, a recent meta-analysis10 failed to find a difference between the modes. The researchers suggest that the “increasing awareness that online data are monitored, controlled for and leaked” may be responsible for the results10. Thus, even with online surveys pollsters may benefit from emphasizing the anonymity of their survey.
2) Assessing Social Desirability Bias
A large number of scales have been developed to measure whether respondents are telling the truth or distorting their answers to present themselves more favorably. Respondents are asked to report the frequency with which they display socially desirable or undesirable behavior, with improbable occurrence. A sample question would be “I never feel angry at anyone” or “I feel the urge to swear sometimes”. The most commonly used and validated Social Desirability scales are listed below:
- Edwards Social Desirability Scale
- Marlowe-Crowne Social Desirability Scale (MCSD),
- Validity Scales of Minnesota Multiphasic Personality Inventory
- Self-Deception Questionnaire and Other-Deception Questionnaire
Once the social desirability scores of participants are obtained, the question is what to do with the data of high scorers. A common method is to eliminate the data, while a less common and more questionable method is to correct for data. Using the data as it is also provides another choice, if the proportion of high scorers is negligible. All methods have their statistical limitations, and the decision must be made carefully.
So far we have been talking more about the limitations and less about the solutions. Unfortunately, the methods to prevent or capture and correct for social desirability bias are more complicated than it is presented here. However, now that you know of this bias, you may look further to how to cope with it. The bottomline suggestion is to question, question and better yet, to question again!
- 7. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of personality and social psychology, 46(3), 598.
- Finkel, Steven E., Thomas M. Guterbock, and Marian J. Borg. 1991. “Race-of-Interviewer Effects in a Preelection Poll: Virgina 1989.” The Public Opinion Quarterly 55 (3): 313–330.
- Streb, M. J., Burrell, B., Frederick, B., & Genovese, M. A. (2007). Social desirability effects and support for a female American president. Public Opinion Quarterly, 72(1), 76-89.
- Powell, R. J. (2013). Social desirability bias in polling on same-sex marriage ballot measures. American Politics Research, 41(6), 1052-1070.
- Brownback, A., & Novotny, A. M. (2017). Social Desirability Bias and Polling Errors in the 2016 Presidential Election.
- Klar, S., Weber, C. R., & Krupnikov, Y. (2016, December). Social Desirability Bias in the 2016 Presidential Election. In The Forum(Vol. 14, No. 4, pp. 433-443). De Gruyter.
- Richman, W. L., Kiesler, S., Weisband, S., & Drasgow, F. (1999). A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84(5), 754.
- Dwight, S. A., & Feigelson, M. E. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and psychological measurement, 60(3), 340-360.
- Dodou, D., & de Winter, J. C. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487-495.