An A/B test (also called a split test) compares two or more versions of a website or app to find out which one performs better. With LamaPoll, you can run an A/B test as an online survey in which participants are randomly shown questions or pages to rate. This guide shows you how to set up an A/B test survey.
A/B tests rely on the random number feature, which is available from the Platinum plan upwards.
Contents
1. Getting Started with Building Your Questionnaire
Creating a survey with LamaPoll is quick and easy. It works much like PowerPoint: you add slides or pages and place elements such as questions, text blocks and images on each one. Along the way you can use features like mandatory questions (respondents cannot continue without answering), show and hide rules (questions or pages only appear if a condition is met), skips (if X, jump to page Y) and more.
Here are the most important features:
2. Using a Random Number to Split Your Target Group
A/B tests split the target group into group A and group B. In LamaPoll, you do this with the “Random number” feature. The system draws a random number for each participant, which you as the survey creator then use (for example with the visibility feature) to show each participant only one of two possible questions or pages.
You will find the random number in the questionnaire menu under “Control” > “New random draw”.
Using the Random Number for an A/B Test with Two Variants
Setting up the random number takes two steps: defining the value range and specifying when the random number is drawn.
1. Define the value range
The value range is the number of questions or pages the system should show at random.
In a standard A/B test, this is usually “1 to 2”, so the range ends at 2: two questions or pages (variant A and variant B) are shown at random. This range is set by default and does not need to be changed.
So for your A/B test, keep the default value range: “From 1 to 2”.
If you want participants to compare and rate more than two versions, increase the value range accordingly.
A/B tests that compare more than two versions are also possible with LamaPoll. If your project goes beyond the standard case shown here, feel free to contact our support team.
2. When should the random number be drawn?
Next, specify when the system should draw the random number. In a standard A/B test, this usually happens right at the start of the survey, which is also the default setting.
With this setup, the system randomly draws a number from the value range (1 or 2) for every participant as soon as they start the survey.
3. Showing Variants A and B to Participants
Once you have set up the random number as described in step 2, you still need to adjust your questionnaire: you tell LamaPoll which question or page to show for which number. You can do this with either the visibility feature or branching (skips).
Example: setting visibility for an A/B test with two randomly shown pages
You set the visibility directly on each of the two pages. Click the eye icon at the top right of the page.
Use the drop-down fields to set the condition for the first page so that it is only shown when the random number is 1.
Do the same for the second page with the value 2, so that it is only shown when the random number is 2.
4. Running and Analysing the A/B Test
Running the test
In most cases, an A/B test is run as an open survey, where all participants use the general survey link to submit their ratings. You can of course also run it as a closed survey with individual access codes. This might suit an A/B test among employees who are asked to rate different versions of the company logo, for example.
Find out more here: Open versus closed surveys
Analysis
As with any survey, the raw data and the overall results are available by default for analysing your A/B test.
The result filters are also useful for A/B tests, as they let you filter by random number and look only at the matching results. The Compare feature is another key tool: it puts the results of both variants side by side in a table so you can analyse them.
5. Notes and Tips for Your A/B Test
- Make sure your A/B test only ever changes and rates one feature of the original version. Never change two or more features at the same time.
- Match the number of variants you test to the number of participants you expect. If, for example, fewer than 100 people are likely to take part, do not test more than two versions, or your results will not be meaningful.
- If your A/B test is more complex than the example described here and you need help, feel free to contact us.
For our free trial, all you need is an email address and a password.

