Questions - Heart of All Polls - continued
In other instances, non-attitudes are more difficult to weed out. Questions that catch respondents unaware will produce doorstep opinions. The very fact that a question is included in an opinion poll implies that someone, somewhere, believes the issue is interesting or significant, and that it is worth going to some effort to learn what many people think about it. That is all it means. Including one or a series of questions about X in a survey instrument does not make X an issue that is objectively or subjectively significant to most Americans. If large numbers of respondents do not know or care about an issue they are likely to respond with doorstep opinions, or confess that they do not know. Either way, the results of the poll tell us little that is useful about the views of the bulk of the public.
Pretesting a survey instrument can help identify questions that create problems. Respondents may find the wording of one question to be unclear. A large number of "don't know" responses may indicate a need for a second look at the topic being examined.
On occasion, instead of leaving out the topic entirely, or relying on the respondents to be knowledgeable about a given issue, the interviewer summarizes salient points or describes opposing viewpoints about the issue, and then asks for the respondent's opinion. Here, as with the question itself, the words used and the points emphasized can influence the responses. Respondents are most likely to get some information if they are being asked about an issue that is relatively complex, or new on the public agenda. A major piece of new legislation is one example. Since most people could not be expected to be familiar with the details of the bill, a screening question would eliminate an unacceptably large proportion of respondents. Therefore, the pertinent provisions might be explained to respondents in broad outline before they are asked for their opinions.
If the presumption is correct and most people know little about the bill, the explanation they receive from the interviewer can have a large impact on the response. Views based on such limited information may be little better than doorstep opinions. Certainly they are not matters of deep conviction. A slightly different explanation of the issue or some new information could cause people to adopt altogether different positions. Consequently, results from different polls, or different points in time, could show widely varying percentages for and against. In this highly unstable situation, opinions are said to be volatile.
The more complex the issues and choices involved, the harder it is to determine public sentiment. The less the public knows and cares about an issue, the more difficult it is to gather accurate and reliable data.