Running an A/B Test (Split Test) with LamaPoll

Set up, run and analyse

An A/B test (also called a split test) compares two or more versions of a website or app to find out which one performs better. With LamaPoll, you can run an A/B test as an online survey in which participants are randomly shown questions or pages to rate. This guide shows you how to set up an A/B test survey.

A/B tests rely on the random number feature, which is available from the Platinum plan upwards.

1. Getting Started with Building Your Questionnaire

Creating a survey with LamaPoll is quick and easy. It works much like PowerPoint: you add slides or pages and place elements such as questions, text blocks and images on each one. Along the way you can use features like mandatory questions (respondents cannot continue without answering), show and hide rules (questions or pages only appear if a condition is met), skips (if X, jump to page Y) and more.

Here are the most important features:

A/B Test Template

Sample A/B test: in our template, each participant is randomly shown one of two versions of an info box from a website and asked to rate it. Nobody sees both variants. Each version sits on its own questionnaire page together with its rating question.

View Use

2. Using a Random Number to Split Your Target Group

A/B tests split the target group into group A and group B. In LamaPoll, you do this with the “Random number” feature. The system draws a random number for each participant, which you as the survey creator then use (for example with the visibility feature) to show each participant only one of two possible questions or pages.

You will find the random number in the questionnaire menu under “Control” > “New random draw”.

Using the Random Number for an A/B Test with Two Variants

Setting up the random number takes two steps: defining the value range and specifying when the random number is drawn.

1. Define the value range

The value range is the number of questions or pages the system should show at random.

In a standard A/B test, this is usually “1 to 2”, so the range ends at 2: two questions or pages (variant A and variant B) are shown at random. This range is set by default and does not need to be changed.

So for your A/B test, keep the default value range: “From 1 to 2”.

If you want participants to compare and rate more than two versions, increase the value range accordingly.

A/B tests that compare more than two versions are also possible with LamaPoll. If your project goes beyond the standard case shown here, feel free to contact our support team.

2. When should the random number be drawn?

Next, specify when the system should draw the random number. In a standard A/B test, this usually happens right at the start of the survey, which is also the default setting.

With this setup, the system randomly draws a number from the value range (1 or 2) for every participant as soon as they start the survey.

3. Showing Variants A and B to Participants

Once you have set up the random number as described in step 2, you still need to adjust your questionnaire: you tell LamaPoll which question or page to show for which number. You can do this with either the visibility feature or branching (skips).

Example: setting visibility for an A/B test with two randomly shown pages

You set the visibility directly on each of the two pages. Click the eye icon at the top right of the page.

Use the drop-down fields to set the condition for the first page so that it is only shown when the random number is 1.

Do the same for the second page with the value 2, so that it is only shown when the random number is 2.

4. Running and Analysing the A/B Test

Running the test

In most cases, an A/B test is run as an open survey, where all participants use the general survey link to submit their ratings. You can of course also run it as a closed survey with individual access codes. This might suit an A/B test among employees who are asked to rate different versions of the company logo, for example.

Find out more here: Open versus closed surveys

Analysis

As with any survey, the raw data and the overall results are available by default for analysing your A/B test.

The result filters are also useful for A/B tests, as they let you filter by random number and look only at the matching results. The Compare feature is another key tool: it puts the results of both variants side by side in a table so you can analyse them.

5. Notes and Tips for Your A/B Test

  • Make sure your A/B test only ever changes and rates one feature of the original version. Never change two or more features at the same time.
  • Match the number of variants you test to the number of participants you expect. If, for example, fewer than 100 people are likely to take part, do not test more than two versions, or your results will not be meaningful.
  • If your A/B test is more complex than the example described here and you need help, feel free to contact us.

Start your A/B test now!

For our free trial, all you need is an email address and a password.

Why LamaPoll?

  • GDPR-compliant
  • Certified servers
  • Hosting in Germany

Start for free

Create a free account – no commitment required

We’re happy to help!

Write to us!

support@lamapoll.com

Last updated on November 19, 2026


  • Allianz für Cybersicherheit participant logo
  • TÜV certificate

A clean website: no trackers, no cookies!

We REALLY respect your privacy: we set NO tracking, advertising or third-party cookies on this website.

And of course we do NOT track what you do on our site either!