What are some of the best practices around asking good survey questions?

There is an art and a science to survey design. You need your questions to be valid and provide high-quality data. But, methodology aside, survey questions need to be written for real people — diverse, busy people whose voices aren’t often heard. If you’re wondering or worried about how to do this well, we have some guidelines to help.

Focus on the purpose of each question

What do you most want to learn? What concept are you really trying to get at? What questions align with your organization’s/program’s key purposes, intentions, and data needs? What would you do differently depending on the answers you receive?

• Determine the question’s purpose and content before structure and wordsmithing. Once you’ve nailed down the key concept you want to ask about, then you can think about details (e.g., open-ended, scale, or check all; number and wording of response options) and look to examples, such as the L4G Custom Question Bank.
• Only ask questions that will yield useful, actionable information. If you’re just asking out of general curiosity, or if the question isn’t truly a priority, cut it. Make sure that the benefit of knowing justifies/outweighs the cost of asking.  
• Focus on clients’ perceptions, experiences, and feedback. Minimize outcome questions, and only include essential demographic questions to understand the representativeness of the survey sample and to segment the data.
• Don’t ask what already is or could be asked elsewhere, such as in an intake form, exit survey, focus group, or other non-anonymous data collection point.
• Make sure each question asks about a different/unique concept from all other questions in your survey and adds unique value to your dataset. If a question can be blended with another or cut, do that.

Design with the respondent in mind

How can you design each question, and the entire survey, to be as enticing, relevant, and easy as possible for respondents? How can you ensure that respondents have a good survey-taking experience?

• Use simple, easy-to-understand wording and avoid jargon, and be consistent in wording and phrasing throughout the survey. For example, always refer to the organization/program in the same way (acronym or abbreviated name), capitalize (or lowercase) program/service names consistently, and refer to things in a way that makes sense to respondents (e.g., do they call the staff their case manager, advocate, or navigator — use only one).
• Prioritize ease and clarity. Ensure the respondent has to do little work to read and understand different response scales, question types, phrasing, etc. Group questions asking about similar topics together.
• Don’t ask double-barreled questions (i.e., two or more concepts in the same question). Instead of, “How effective are the organization’s job search and resume writing services?” you should ask about multiple concepts one by one, or unite them into one broader theme: “Overall, how effective are the employment services (e.g., job search, resume writing, interview techniques)?”
• Minimize complexity and the number of answer options. For scaled questions, we recommend 5 response options. Check all that apply questions should have no more than 5-8 answer options.
• Use open-ended questions sparingly (i.e., 2-3 total). Open-ended questions can take longer to respond to and can be particularly difficult for low-literacy populations – and they require more staff capacity for coding and analysis. If you can ask in a multiple choice or checkbox format instead, it would be a better approach. We recommend no more than 2-4 open-ended questions in your entire L4G survey.
• Use consistent types of answer scales, and order options from lowest to highest value. To help respondents quickly understand and respond, try to use the 5-point scales already in the L4G survey to measure extent (not at all, a little bit, fairly, very, extremely) or frequency (never, rarely, sometimes, usually, always). You can adapt the extent scale with other descriptive words, such as safe, helpful, satisfied, challenging, etc.
• Make sure every respondent can find an answer that suits them. Determine if a “none of the above,” “NA,” and/or “don’t know” option is needed. Consider offering an “other, please specify” option to solicit additional answers. And, make sure that numeric response options include room for all possible low and high values (e.g., less than one month; 65 years or older).