10 Tips for Writing Well-Worded Survey Questions
When you need data for planning and tracking, doing a survey might seem like an easy solution. Unfortunately, many surveys end up not being very useful. They get too few responses, biased responses, or irrelevant responses. You can increase the chances that your survey will get you useful results by making sure your survey questions are well-constructed.
Writing Survey Questions for Impact
Incorporating these tips as you develop your survey questions will increase the chances that people will answer the questions in a way that gets you the information you need for effective decision-making, communication, and action.
Tip 1: Make Response Options Exhaustive
In the 1950s, the Washington, DC, police department asked people this question.
Q1: What is your occupation?
Many survey questions have the same problem: the response options are not exhaustive. If you have an occupation other than clerk or mechanic, the response options to that question leave out the option that applies to you.
You can fix this problem by making sure that your responses include all possible options. To be on the safe side, add an “other (please specify)” option to capture responses that you had not anticipated.
Tip 2: Make Response Options Mutually Exclusive
Another problem is questions that say to select one response option when more than one option might apply. That is, the responses are not mutually exclusive. For example, I’ve sometimes seen survey questions like this:
Q2a: How long have you been in business?
☐ Less than a year
☐ 1 to 3 years
☐ 3 to 5 years
☐ 5 years or more
This question may be hard for some people to answer because the response options overlap. If someone’s been in business exactly three years, they may be unsure whether to check “one to three years” or “three to five years.” You can avoid this problem by making sure that the choices are mutually exclusive. In other words, check that each option excludes the other options:
Q2b: How long have you been in business?
☐ Less than a year
☐ 1 to 2 years
☐ 3 to 4 years
☐ 5 years or more
Tip 3: Split up Overly Complicated Questions
When survey questions take too much time to answer, people may skip the question or answer in different way than expected. Many people also find it hard to answer questions that require rank ordering a list of things, like the question below. Ranking items takes a lot of thought. What if you like or dislike all options equally? I could tell you how much I like each flavor, but not rank them.
Q3: Rank the following jelly bean flavors in order from 1 to 8, 1 being the flavor you like the most and 8 being the flavor you like the least.
You can simplify such questions by splitting them into separate questions. For example, we could split the question above into 8 separate short and simple questions, one question asking how much they liked each flavor.
Tip 4: Split up “Double-pronged” Questions
“Double-pronged” questions are hard for people to answer because they are asking for one answer to more than one question. An example is:
Q4: “Was the training informative, fun, and useful?”
Actually, that question was “triple-pronged.” You’d find it hard to answer this question if you thought the training was informative and fun, but not useful, or fun and useful but not informative.
You can solve this problem by, again, splitting those into separate questions. For example, we could split question four above into three clearer questions, asking about 1) whether the training was informative, 2) whether it was fun, and 3) whether it was useful. Or, we could ask 1) how informative it was, 2) how fun it was, and 3) how useful it was.
Tip 5: Rephrase “Agree” Questions
Many surveys include questions asking people to check how much they agree, such as:
Q5a. The training was useful.
☐ Strongly agree
☐ Neither agree nor disagree
☐ Strongly disagree
Studies have revealed a problem with “agree” questions. Asking people how much they agree with something biases the results, because people like to be agreeable. That can mean your survey ends up with misleading data, such as not showing that something made a difference when it did.
You can solve the problem of “agree” questions by re-wording them to ask about the thing that you want to measure. For example, in Q5a, we can reword the question to ask how useful they thought the training was:
Q5b. How useful was the training for you?
☐ Extremely useful
☐ Very useful
☐ Somewhat useful
☐ Not so useful
☐ Not at all useful
Tip 6: Make Response Options Clear
We could have worded the question above as:
Q6. Rate how useful the training was for you on a scale from 1 to 5, where 1 equals “Extremely useful” and 1 equals “Not at all useful.”
People tend to find these questions unclear. If they’re in a hurry, they might not notice that 1 equals “Extremely useful” and check “5” thinking that a higher number meant more useful. Make the question clearer by putting the response options next to the checkboxes, as in question 4. If you need the responses to be in the form of a number for data analysis, you can convert the responses to numbers when you analyze the data.
Tip 7: Promise (and Keep) Privacy Protections
Asking for personally identifying information can make people uncomfortable about taking your survey. People may get suspicious of why you want that information.
Don’t ask for any information unless it’s something you need to know. In some situations, you may need to collect personally identifying information about survey respondents for an important reason:
- To track survey response and send reminders to people who haven’t responded
- To survey the same group more than once and match responses by person
- To send a summary of survey results to participants who request one
- To send incentive gifts to survey participants
You can get that information and still protect survey participants’ privacy by creating a non-identifying survey participant identification number for each survey participant. You can then use those non-identifying numbers to track and match surveys by person while keeping identifying information separate from people’s responses. Before asking any questions asking for personal information, such as a person’s name or email address, explain what you will do with that information, as in the example below.
Q7: Enter your email below if you’d like to be entered into a raffle for a $500 gift card. Your email will be kept separate from your survey results and will be used only for the raffle.
Your email: ______________________
Tip 8: Use Their Language
I was once involved in creating a survey about a particular health care service. When we developed we used the same term for the service that government programs used. That was the term that we knew as policymakers and researchers. When we got feedback on our draft survey from people in the field, many people thought that the survey didn’t apply to them because the term that the government used was not the term they used for that service.
Getting feedback on a draft of your survey from a small group is a great way to make sure that your survey uses wording that is familiar and to catch and fix any other problems before surveying everyone in your study.
If you must use ask about terms that may be unfamiliar, you can include definitions, to ensure that survey respondents know what you mean.
Tip 9: Be Specific
Say you’re conducting a “Gardening Workshop Participants Survey” to get feedback about your gardening workshop. Your survey might include the question:
Q9a: “Please provide any suggestions for improving the workshop.”
Besides taking your gardening workshops, people receiving the survey may have also participated in other workshops that your organization provides or other workshops held at the same building. They might be thinking of those other workshops as they’re answering your survey. You can avoid confusion by specifying which workshop you’re asking about:
Q9b: “Please provide any suggestions for improving the ‘Home Gardening 101’ Workshop”
Making your questions more specific helps ensure that you’ll get relevant results.
Tip 10: Don’t Ask Too Many Questions
If you’re survey is long, people may get “survey fatigue.” They might start skipping questions, or they start answering questions quickly in a way that does not reflect their true opinion.
I could say a lot more, but I don’t want you to get “article fatigue” and stop reading!
I hope these tips are helpful for your next survey. Let me know if you have any questions.
We'll inform you about just-published articles, our upcoming books, professional Summits, live interviews, webinars, and more
If you are an experienced nonprofit-sector practitioner who wants to share your expertise by contributing down-to-earth articles written in a conversational style, we'd love to hear from you!
Request Permission to Reprint This Article
Copyright © 1992-2019 CharityChannel LLC.