Randomisation serves several purposes in online surveys. In A/B tests run as a survey, for example, participants are usually split evenly between two groups. Randomisation makes sure they are assigned to the test groups by chance. In a classic A/B test, every participant should ideally have the same probability of ending up in group A or group B. In online surveys, this equal allocation can be achieved with a random number or with random candidates. The sections below explain what randomisation and randomised equal allocation actually mean, how to make sure participants are distributed evenly in an online survey, and where this approach is used.
Table of Contents
Randomisation: definition and meaning
In general terms, randomisation means deliberately introducing chance into a process to remove any systematic pattern or predictability. It is a methodological tool: controlled chance is used to obtain results that are as objective and unbiased as possible. Chance is applied deliberately and methodically, not haphazardly. Randomised procedures are used in A/B tests (assigning users), scientific experiments (assigning participants), statistical sampling, clinical trials (assigning treatments) and cryptography (generating keys).
Examples of randomisation in online surveys
Online surveys use randomisation for different purposes. One is to show questions, answer options or entire survey pages in random order. Survey creators who rotate questions, pages or items usually hope to improve response quality, for instance by reducing order effects. However, randomising questions and items can also have drawbacks.
Another example is assigning participants to groups at random. This is common in A/B test studies. The aim is to split participants evenly between two (or more) groups so that the data is robust and the groups can be compared meaningfully.
Random group allocation is used in two ways. In the first, each group is shown a different version of something, such as a new app, and participants rate it. In the second, participant randomisation is used to study the effect of different variants, such as two communication strategies, by comparing how the groups respond. In this case, participants do not rate the versions themselves. Instead, the survey creator analyses the results afterwards for differences in how each group answered.
Randomisation is used in online surveys to...
- ... show questions, pages or answer options in random order, i.e. rotate questionnaire elements
(to avoid order effects). - ... display images, videos or other media in random order
(to avoid position effects in ratings). - ... assign participants to groups at random so they rate different versions of an object or stimulus
(A/B testing, experimental designs). - ... assign participants to groups at random to compare how they respond
(split tests, control group designs).
Randomisation, equal allocation, user assignment: what the terms mean
Several terms are used for assigning participants to groups at random: even distribution, equal allocation, unconditional equal allocation, participant randomisation, even participant or user assignment, simple randomisation and so on. So which one is correct? In A/B testing practice, people usually just say “randomisation”, because the simple form is the standard and other variants (such as stratified or block randomisation) are named explicitly when they are used.
In the context of A/B tests, participant randomisation or equal allocation therefore usually means that each participant has the same probability of ending up in group A or group B. In scientific publications and experimental research, “simple randomisation” is the established term for exactly what is commonly called equal allocation.
Randomised equal allocation of survey participants means:
- No stratification: participants are not assigned on the basis of demographics, prior experience or any other characteristics.
- Purely random allocation: every participant has exactly the same probability of being placed in group A or B (typically 50:50).
- No pre-filtering: no particular group of people is favoured or excluded when groups are assigned.
- Even distribution: across a sufficiently large sample, both groups should be about the same size and have similar characteristics.
Equal allocation of all participants (including dropouts)
In LamaPoll, the split into group A and group B that is typical of an A/B test is done with the “random number” function. The system draws a random number for each participant. You then use it, for example with the visibility settings, to show each participant only one of two questionnaire versions (question variants, pages, etc.). In a standard A/B test, the random number is usually drawn right at the start: as soon as a participant begins the survey, the system draws a number from a defined value range at random.
The more people take part, the closer the random values come to an even distribution. Think of rolling a die: you might roll a 1 four times in a row, but after 1,000 rolls every side will have come up roughly equally often.
With this approach, every participant gets a random number at the start, including those who may drop out before finishing the questionnaire. Statistically, this usually evens out. If you want to be sure of equal allocation even with a smaller sample, you can use the “least filled quota” feature.
If dropouts must definitely be excluded, so that equal allocation is based only on participants who completed the survey, you need a different approach. We explain it in the next section.
Equal allocation of completed responses only
If only completed questionnaires should count towards equal allocation, dropouts need to be excluded. In this kind of “A/B test without dropouts”, only participants who have worked through the entire questionnaire are counted. Instead of random values, it uses “random candidates” defined by the survey creator. Relative weighting must also be enabled, and the questionnaire needs to be structured slightly differently. With these settings, equal allocation counts only the responses of participants who reached the end of the questionnaire. Dropouts are effectively filtered out. A typical application is an A/B test (also called a split test). The method is also used for control group designs in market research and medical research.
Our helpdesk has a detailed step-by-step guide on how to ensure equal allocation of completed responses only: A/B test: quotas without dropouts
Create an A/B test questionnaire
Randomisation in practice: an A/B test in health communication
Use case:
A typical use case for randomisation in an online survey is an A/B test study on how effective different approaches to health education are.
Scenario:
A public health authority wants to find out which type of information better motivates people to attend preventive check-ups. The question is which communication strategy works better: an emotional appeal or factual arguments? A/B tests in online surveys make it possible to investigate this systematically.
Method:
In this A/B test, survey participants are randomly split into two groups of equal size. Group A receives an emotionally worded version of the message, while group B is shown the same content in factual form. Randomised allocation creates comparable test groups that differ only in the communication strategy. In this use case, only the responses of participants who completed the entire questionnaire should be counted.
Process and questionnaire structure:
- Shared opening pages: demographics, health status, previous check-ups, attitudes towards preventive check-ups
- Experimental group: receives emotional information with personal stories and images
- Control group: receives factual information with statistics and facts
- Shared closing pages: willingness to attend check-ups (after the intervention), trust in the information, additional demographic questions
Why only completed responses should count:
- Dropouts may differ systematically. For example, anxious people or those with a fear of doctors (iatrophobia) are more likely to drop out when they reach certain content.
- The intervention is only complete if the participant finished the whole survey.
- Equal allocation ensures that both groups end up the same size, so they can be compared meaningfully.
Practical benefit:
This allows the authority to decide on a sound scientific basis which communication strategy will work best for future screening campaigns. The method provides data-driven insights for optimising communication strategies, for example in public awareness campaigns. Instead of relying on assumptions, it produces robust data on how the target group actually responds to different communication approaches.
Uses of survey-based A/B tests
An A/B test (also known as a split test) is a method of comparing two or more versions of a website or app to find out which one performs better. A/B test studies can also be used to assess the impact of different versions of an advertising or marketing campaign, a communication strategy or product messaging in advance. Split tests are also used in change management.
LamaPoll supports different kinds of split test. In the first, the A/B test runs as an online survey: participants are randomly shown, for example, different versions of a website (or an ad banner, product packaging, a slogan, etc.) and asked to rate them. In the second, split tests are used to examine communication strategies or campaigns. Here the focus is on how participants respond, for example to compare the effect, and therefore the likely success, of different interventions. Instead of relying on assumptions or gut feeling, split test results let you make data-driven decisions.
For more on running A/B tests with LamaPoll, see the two step-by-step guides in our helpdesk: Running an A/B test and A/B test: quotas without dropouts, for A/B tests that should include only completed questionnaires.
With LamaPoll, surveys can be created quickly and easily. The tool offers many options for individual adjustments. The help pages are useful for solving many questions independently. For special requests, competent and easily accessible customer support is available.
What is LamaPoll?
LamaPoll is a survey tool for creating online questionnaires. Our survey tool lets you create, run and analyse online questionnaires quickly and easily. Create your questionnaire in just a few steps, customise it for your organisation and conveniently invite customers or employees by email. An online questionnaire enables you to gather insights quickly and easily to improve customer or employee satisfaction. You can register for the LamaPoll survey tool free of charge and immediately create and run online surveys with up to 50 participants. If you need more responses for your online questionnaire, simply choose one of our plans: billed monthly, cancel any month. You can also choose from our online questionnaire templates and examples.
