Survey Goals and Question Design: Expert Tips

Set survey goals and make your questions specific

However different survey projects may be, they always aim to

• achieve broad acceptance of the survey within the target group and a high response rate,
• obtain meaningful and reliable results, and
• complete the project on time and within budget.

The quality of the results depends above all on four prerequisites:

1. questions that are on target and complete,
2. a well-designed questionnaire,
3. good communication with the survey's stakeholders, and
4. systematic execution.

Precision and rigour

The key question(s)

To get clear about the goal of your survey from the start, it helps to answer three questions: What exactly do I need to know, from whom, and why?

Example: A provider of online survey software wants to ask its customers about the product and the support. The key questions could look like this:

1. In what context is the software used (length of experience with the product, areas of use in the company)?
2. How good is the support?
3. How satisfied are users?

Tip: Because the research question and goal can shift over the course of a project, it makes sense to keep a running record of your initial question(s), assumptions and hypotheses. That way you can trace any change of focus later.

Making questions specific

With factual questions (here, for example, “the context in which the software is used”), it is usually clear straight away what is meant. Things get more complicated when a concept (e.g. “user satisfaction”) has several layers or cannot be measured directly.

Example: You ask: “How satisfied are you with the XYZ software?” You would get a general picture of sentiment, but you would not know which aspects of the software respondents are basing their judgement on (features? user interface? error tolerance? stability? customisation options? all of them? something else?). Nor would you know how the individual ratings shape the overall verdict. In short: at first glance, it is not clear what “satisfaction” actually covers.

Tip: A proven method for measuring complex issues is to make them concrete step by step, known as “operationalisation”. List the relevant topics and break each one down into observable facts. The aim is a complete and consistent set of questions in which no two questions overlap.

Tip: Check your set of questions for completeness and test each question for its explanatory value: is this particular piece of information essential to your research question, or merely interesting at first glance? If in doubt, picture in advance what the results are likely to look like and what conclusions, if any, you could draw from them.

Tip: LamaPoll offers a wide range of questionnaire templates with standard questions for many occasions. If you use one of these templates, treat it as a checklist and keep only the questions that serve what you actually want to find out.

Tip: Think about the results while you are still planning. A rough analysis plan, which you can refine once the results are in, gives you direction. Decide on the report format early on, too.

Tip: Outline your scope for action while you are still planning. You don't need a finished list of measures in the drawer (that would make the survey pointless). But asking about topics where no changes are up for discussion can backfire. If, for example, you ask in detail about service quality and it turns out that many customers take a critical view of your service, you will have to respond …

Author of the guest article “Expert Tips”:
Elisabeth Kerkhoff, graduate social scientist

kerkhoff@extern.lamapoll.de