Synonyms: split testing
Method of testing when two or more options are shown to different groups of users. The aim is to find the best option. For example, сhoose between two buttons, pictures or any other elements of design.
The study helps to optimize conversion rate and understand users’ behaviour.
This is what you want to find out with the test. To set a goal you should understand the project goal. For example, you have a landing page and you need to generate more leads. This is your project goal. To achieve this goal you can improve different elements of the page. To find a better element you can test different options of offer text, placement or design of call to action button.
Tips & Tricks: you can set a goal through a question. For example, ‘’which button gets more clicks’’ or ‘’will changing of offer text generates more buyers’’.
This is the basics of your test. It is a combination of what you want to change and your assumption why the change is possible.
Tips & Tricks: to find a hypothesis you can ask yourself a question. For example, “why changing this element can increase the number of downloads”. If you can find an answer, testing is appropriate.
The secret here is that two versions of one component shouldn’t be too different. Try to find only one variable. For example, you can test button color or button copy, but not both things at once. It is important for analyzing results. If you test both color and copy how can you tell what actually works better.
Although, you can test different elements and test more than two options, but this is more complex multi-variant testing.
Tips & Tricks: to find ideas for testing, check out different case studies. For instance, you can learn what options you can change to increase conversion rate. There might be different elements: from texts and hyperlinks to button placement and shape, form shields and length of forms to images of different styles.
The option that shows better results is usually called the winner. Metrics can be different but the most common ones are conversion or click rate.
Tips & Tricks: make sure that you have enough data to determine a winner. This means that your test is fully completed without errors: there are enough users participating in the test and there is enough time for testing. Both variables are different for different projects. In most cases, it is okay to run the test on 5–10 % of all the project users. As for time, it is more complicated, but 1–2 weeks is usually enough.
A/B Testing is effective when you have enough time and users. There is no sense in testing options when you have 30 users as results won’t be representative. A/B Testing is great for optimizing and updates, but in earlier stages other methods of testing will be better. For instance, 5 Second Test to evaluate the structure of the site or page or Hallway testing to find obvious design errors that are getting in the way of users.