Alex Coppock and Matthew Graham have published a new paper, “Asking About Attitude Change” in Public Opinion Quarterly, that examines how changes in attitude can be better measured with less bias depending on how these types of questions are posed to survey respondents.
Paper Abstract: Surveys often ask respondents how information or events changed their attitudes. Does [information X] make you more or less supportive of [policy Y]? Does [scandal X] make you more or less likely to vote for [politician Y]? We show that this type of question (the change format) exhibits poor measurement properties, in large part because subjects engage in response substitution. When asked how their attitudes changed, people often report the level of their attitudes rather than the change in them. As an alternative, we propose the counterfactual format, which asks subjects what their attitude would have been in the counterfactual world in which they did not know the treatment information. Using a series of experiments embedded in four studies, we show that the counterfactual format greatly reduces bias relative to the change format.
Image credit: Shutterstock, Andrii Yalanskyi