In the book Public Opinion, the authors discuss how the way in which a survey question is worded can affect the response of the survey participants. On page 84, the authors state, "Survey designers need to explain topics and options clearly, in everyday, user-friendly terms." The authors go on to say that putting a question regarding a complex issue into everyday language that respondents will readily understand "blunts the subtleties of the issue. This is a common trade-off for survey researchers: precision of meaning versus getting the general message across to a broad audience" (p. 84).
Do you feel that the "trade-off" made by many researchers is necessary and justifiable? Also, do you believe that the validity of the survey results are skewed by this "trade-off"?
Subscribe to:
Post Comments (Atom)
I think the trade-off described in the book is impossible to avoid. There are always going to be differences in the public’s knowledge on any given issue. I think that away to minimize this trade-off effect is to understand the demographics of the people you are surveying. Things to keep in mind are their knowledge of the subject, education level, and general views of the world. This way you can construct questions that will accommodate to their demographics. If researchers fail to conduct their own research on survey participants, then it is very easy for the results to be skewed.
ReplyDeleteWhen in doubt, I feel that it is in the best interest of the researcher to make the topic and questions as easy to understand as possible. A sure thing to avoid is using technical jargon or sang terms that some might not know the meaning of. If the topic is generally understood by all participants then the data will have more validity.
I believe that this question is relative to the population receiving the survey. If a survey is being sent out to a group of college graduate students, then the “trade-off” should not be an issue. On the other hand, if a survey is being sent to a group of high school students, then the questions should be worded in more blatant terms. I agree with Lauren’s statement about how it is the researcher’s responsibility to construct a survey that the population will understand.
ReplyDeleteIf the survey was being sent to Americans as a whole, I believe the trade-off is necessary and justifiable. According to Informatics Review, “Nearly 50% of the Americans surveyed cannot read well enough to find a single piece of information in a short publication, nor can they make low level inferences based on what they read.” Sadly, I do not think this statistic is very startling. In order for all survey participants to have an equal chance of understanding and answering the questions, it is important to put the questions in blunt terms. If I were given a choice of whether to “dumb down” a survey, or keep the survey in a more meaningful way, I would have to choose to make the survey more understandable and easy to comprehend.
I do not believe that the results of a survey would be skewed due to the trade-off. On the contrary, I believe that a survey would be skewed if the questions were not blunt. Some individuals could have a very hard time what it is they are being asked and therefore, not answer the way they might have wanted to.
I think that the "trade off" mentioned in this chapter is necessary and unavoidable. I believe the trade-off is necessary because in most cases the researchers are always going to be more knowledgeable than the participants. The researchers have been studying the topic for a long time and are wishing to extend on the current research.
ReplyDeleteI also think that putting a survey into simple terms makes the participants more likely to complete the survey honestly and as accurately as possible. I think that if the survey questions are put into everyday language and the participants understand what they are being asked, the results will not be skewed because all of the participants have the same understanding of the survey.
I think the “trade-off’ discussed in the book is unavoidable but should be considered. It is nearly impossible to make an in-depth survey that society as a whole is able to answer accurately without the survey being skewed. Society’s knowledge is very broad and impossible to measure. Individuals’ educational level, common sense levels, experience and personal views vary incredibly from one individual to another.
ReplyDeleteResearchers conducting surveys need to consider the “trade-off” in order to ensure the survey will be answered with as much accuracy as possible. Researchers need to keep in mind the group they are focusing on, especially the demographics and psychographics. You want to make the survey as simple, yet thorough as necessary. Researchers also need to keep in mind the size of the survey.
Our society is a fast-paced society and many don’t have time to answer complex and lengthy questions. I believe the best advice for a researcher is to use everyday language of their target audience and keep surveys short. Individuals are more likely to participate in more surveys if they are quick and easy.
While there are certain limitations to the "trade-off" that is mentioned in the book, I think that many surveyors "dumb down" the surveys to much. If you are surveying about a top that you have to change you may not be getting the right cross sample of respondents and should consider revising the distribution process rather than the survey.
ReplyDeleteHowever, there are certain instances that the "trade-off" is necessary. Take for instance a survey about the public's knowledge of a new virus sweeping the country. If you want to know the general public opinion the wording will need to be different than if you wanted a cross sample of doctors.
I think that the trade-off made by researchers is necessary because education levels tend to vary from person to person. Take the example given in the book. “Do you agree or disagree that U.S. foreign policy should give more sympathetic consideration than it has in the past to Bosnian Serb appeals for greater autonomy?” versus “Do you agree or disagree that the United States should side more with the Bosnian Serbs?” Not everyone asked a question like the first example will know the involvement of the U.S. with Bosnian Serbs, but they may know more about a present situation from the news. If a researcher is going to survey the general population then I think that it is necessary to accommodate those whose education levels are not as high as others in order for them to better understand the questions raised.
ReplyDeleteI definitely think that survey results are skewed by the "trade-off" researchers use when creating surveys but I think that it works in two ways. First, if the researcher asks questions that are too complex, people are not going to understand and therefore, may not answer the way they really want to. On the other hand, no matter how simple the question is, if you ask it to someone who has no knowledge on the topic, they will be unable to answer it no matter how the question is worded. Researchers need to consider the population they are surveying when wording their questions.
ReplyDeleteI think that it makes sense to simplify a survey that is going out to a large population, but I don't think that it is necessarily wrong to ask more complex questions if the situation permits. For example, if asking upper level college students or graduate students, the survey might call for the participants to put more thought into their answers and therefore be more complex.
I believe that the trade-off depends on the research being conducted, the topic being discussed, and the population that is being surveyed.
ReplyDeleteIf the topic at hand is a very well-known topic and if the questions are being asked of the general population (or at least a sampling of the general population), the questions can be worded simply. It might even be alright to use slang as a way to make sure that people understand the questions being asked. On the other hand, if questions on Communication theory were being asked of Comm. department faculty from all over the country, the questions can be worded in a technical manner. The faculty would have much more knowledge about the topic than a less specific population would.
I believe that the "trade-off" the book discusses is often unavoidable. It is impossible to know 100% if everyone receiving the survey will understand what exactly each question is trying to ask. The best thing a researcher can do is consider the population being surveyed--such as demographics and the size of the population.
ReplyDeleteThe larger the population, the more a researcher would want to simplify the questions because that means that the educational backgrounds will be broadened.Whereas if the population is smaller and more concentrated within one group, the questions may be worded a bit more complex.
I think the main concern of the researcher needs to be exactly who their population is and how large becuase how the survey questions are worded and how long/short they are is going to affect the number of responses and the number of responses that are actually truthful.
Upon reading the chapter in the book it mentions that “user friendly terms” should be used which I think is essential because different language has different meaning behind it changing their opinion in the process (Public Opinion, 84). The skewing of the results can be drastically reduced by the vocabulary being as neutral as possible. I think the trade of is justifiable because if the majority of the population understands what you mean than it should be all right, not everyone can interpret what is desired always. The validity of the surveys done with using of trade off concept is just more valid to me than anyone who doesn’t understand just isn’t the right candidate for taking it in the first place. Whenever you survey a population there is going to be some groups of people that know absolutely nothing on a topic and answer anyway to be polite, that’s the nature of people which and data collection.
ReplyDeleteUpon reading the chapter in the book it mentions that “user friendly terms” should be used which I think is essential because different language has different meaning behind it changing their opinion in the process (Public Opinion, 84). The skewing of the results can be drastically reduced by the vocabulary being as neutral as possible. I think the trade of is justifiable because if the majority of the population understands what you mean than it should be all right, not everyone can interpret what is desired always. The validity of the surveys done with using of trade off concept is just more valid to me than anyone who doesn’t understand just isn’t the right candidate for taking it in the first place. Whenever you survey a population there is going to be some groups of people that know absolutely nothing on a topic and answer anyway to be polite, that’s the nature of people which and data collection.
ReplyDelete