Every election season, polling data fills headlines, drives campaign strategies, and sways public policy decisions. But what if the numbers aren’t as straightforward as they seem? And if subtle biases in how we ask questions or who asks them distorts our understanding of public sentiment?
Our recent research sheds light on this issue by exploring a specific, yet often overlooked, factor in survey responses—the influence of the interviewer. In our study of survey data from 26 Latin American and Caribbean countries, we found that the gender of the interviewer significantly affected responses on sensitive topics like gender roles and political attitudes. For instance, respondents interviewed by men were more likely to support traditional gender roles and accept domestic violence in certain contexts than those interviewed by women. Such biases can meaningfully misshape the findings, especially on topics where social norms are in play.
The implications, of course, extend far beyond elections. Polls are a vital tool for understanding the pulse of a nation — essential for governments, businesses, and researchers in shaping policies, influencing public perceptions, and allocating resources.
Inaccuracies That Could Reinforce Bias
If public opinion polls are affected by biases, public policies crafted in response could be based on a misrepresentation of the public’s true attitudes. Consider how policy might shift if surveys overestimate support for traditional gender roles or underestimate opposition to domestic violence. Such distortions could lead to policies that don’t accurately reflect the needs or values of a population, potentially reinforcing biases rather than addressing them.
In our analysis of surveys throughout the Latin American and Caribbean region, we found that respondents, when interviewed by men, were 4.5 percentage points more likely to perceive men as superior political leaders than when interviewed by women. Moreover, acceptance of domestic violence in polls was more than 5 percentage points higher when the interviewer was male. And those were just averages. In Guyana, for example, the acceptance of domestic violence could climb to as high as 59% if all interviews were conducted by men and drop to as low as 24% if they were all conducted by women.
The Social and Economic Realms
Surveys that underestimate or overestimate support for certain behaviors can have an enormous impact in the policy realm. And they may also affect attitudes in the more strictly social and economic ones. For example, in one study, when young married men underestimated the support among other men for women working outside the home, the likelihood of women working outside the home dropped compared to when those men were correctly informed, illustrating how inaccurate information about social norms can have highly significant and negative welfare effects.
Designing Better Polls and Surveys
To address these biases, our study recommends a few practical steps. First, survey organizations should disclose interviewer characteristics, such as gender, in their datasets, allowing researchers to adjust for them. Alternatively, anonymous methods like online surveys could reduce interviewer effects altogether. As polling organizations in the U.S. and around the world refine their methods in light of recent election cycles, tackling interviewer effects could improve the reliability of survey data and lead to better-informed, more representative policy decisions.
Understanding and minimizing biases in survey responses is essential for capturing a true reflection of public opinion. Polling, when designed and interpreted with care, can offer valuable insights into society’s beliefs, preferences, and needs. When it is not, it can similarly skew public behavior and policymaking, with far reaching negative implications for society at large. By recognizing the nuances in survey design and response patterns, we can work toward polls that better serve the public, ensuring that policies align with the realities and aspirations of those they affect.
Leave a Reply