Good Practice When Designing Evaluation Questionnaires: General Advice
Before you even start:
- Make sure you have a clear idea of what exactly you want to get out of the questionnaire, why and how the data will be used.
- If you are undertaking an evaluation of a service or activity, make sure you are clear on what outcomes you are measuring and that the measures you are using are appropriate and effective.
- Make sure you are clear how the questionnaire fits into your overall research aims or evaluation plan and how it will relate to data you may be collecting via other methods (for example, through focus groups or interviews).
Designing your questionnaire:
- Keep the questionnaire as short as possible to maximise your chances of getting a good response rate. Only ask for information that you will definitely use!
- Express your questions as simply and clearly as possible.
- Think about whether a question actually has to be mandatory. Too many questions forcing a response may put respondents off part-way through.
- Think through what sort of demographic information you need to ask for if you want to break down your analysis by different groups of respondents (for example to see how responses differ by gender or age). Make sure you are collecting this data in an effective way to enable this kind of analysis.
Structuring your questionnaire:
- Think about layout. If your questionnaire is well laid out, uncluttered, easy to read and possibly even attractive, you will probably get a better response rate.
- Make sure your questionnaire has a logical flow and keeps respondents in the same ‘headspace’ for each block of questions. If your questions appear to jump about, some respondents could give up part-way though. (e.g. Don’t jump from questions about venue, to the quality of the presentations, back to location etc.)
- Also be alert to the fact that the order of the questions can introduce bias. Early questions, for example, can ‘prime’ respondents (Darren Brown style) to answer later questions in a particular way. When you have your questionnaire laid out, read it through with this in mind.
- Use fonts or bold and italic text and headings to help clarify the structure of the questions.
- Consider the structure and order of your questionnaire. Some experts advise starting with easy and / or interesting / enjoyable questions, or starting with general questions and working towards more specific questions. Also think about leaving difficult or personal questions (such as demographic data) to the end.
Testing your draft questionnaire:
- Use the questionnaire checklist (available from the download box on the top right) to make sure you’ve not fallen into any of the common questionnaire pitfalls!
- TEST IT with other people. They will pick up on ambiguous or confusing questions or jargon.
- TEST SAMPLE DATA! Ask colleagues to fill in a batch of questionnaires. Then analyse the data as if it was real. Does it tell you what you were hoping it will tell you? Does it work and give you ‘clean’, ‘analysable’ and sensible data? This bit is hard to get right first time, and you will often have to go back and rethink your question(s) to make sure you get the data you need. Think about the kinds of arguments and outcomes the data could give you. Will you have the appropriate data you need?