Skip to main content


Why A/B Tests are the Workhorse of your CRO Research Phase (Part 4/6)


This week in our six part series on Conversion Rate Optimisation (CRO) tools, we’ll discuss the ins and outs of A/B testing for research.

As with the previous blog posts in this series (The 5 Types of Tools overview, Surveys and Maps), this instalment is based on our Periodic Table of CRO Success Factors. (This infographic covers the aspects of CRO that are crucial to its success. If you haven’t seen it, we suggest you have a look before reading further.)

What is A/B testing?

A/B testing is essentially a method used to test how visitors respond to your website, and the various elements of it.

It works in sending 50% of your website visitors to see a new version of your website, and 50% to the existing version. This allows you to compare scientifically, whether it was the ‘variation’ (new version) or the ‘control’ (existing version) that generated the most orders and online revenue.

Businessman doing online shopping on couch in living room.jpeg

A/B testing platforms use statistics to validate that one version is outperforming the other, and this form of testing can be useful both in the Research & Analysis phase as well as Execution, when you have developed a strong hypothesis you want to test.

How is A/B testing used for research?

Data gathered from A/B testing is invaluable, as it’s based on real-time visitor actions as opposed to visitor feedback, which is relayed after they’ve visited your site. In other words, this data is unsullied. It’s this kind of objective data that should inform the rest of your CRO process.

In the Research & Analysis phase, A/B testing can be used to test the importance of individual elements of your website, unlike during the Execution phase, when it is used to validate that changes to these website elements result in higher conversion and revenues.

Exclusion A/B testing

While there are several different types of A/B tests, exclusion A/B testing can be most useful in the research phase.

Exclusion tests entail the creation of a new variation of a page, where one aspect is hidden. For example, the words ‘Free delivery on order over £X’. This will enable you to discover how important this ‘offer’ is to your visitors, and crucially, whether it impacts sales. Alternatively, you could test hiding the offer at various aspects of the customer journey and analyse the impact – whether or not it lowers, or perhaps increases, sales.

Delivery guy holding package while woman is signing documents.jpeg

The insights you gain from this enable you to understand the customer’s decision-making process from the time they first land on your homepage to the time they complete a purchase or close the tab.

A/B testing tools

You don’t have to be a coding aficionado to implement (and reap the rewards) of A/B testing.

There are lots of A/B testing tools on the market, many of which feature easy-to-use, ‘drag and drop’ interfaces which simplify and speed up the process.

These are suitable at the research stage but bear in mind, A/B tests done in the Execution phase are likely to be far more complex.  As such, it’s advisable to have a front-end developer (either in-house or outsourced) who’s well-versed in Javascript, jQuery and CSS at your disposal for Execution.

Popular simple A/B testing tools for small to medium-sized businesses:

Platforms favoured by larger businesses include:

The platform you choose will be dictated by your specific CRO requirements, budget and CRO goals. Bear in mind that as Avinash Kaushik (author of Web Analytics 2.0) points out,for every $1 you spend on tools you should be spending $9 on people to make use of these tools, so choose your tools wisely.


Next time, we’ll be discussing another potent CRO tool: heuristics. We’ll explain what it is, how to integrate it into your CRO process, and the considerations you need to be aware of when doing so. Make sure you don’t miss out on this invaluable six-part blog series, by signing up for emails using the form at the bottom of the page.  

We’re a Conversion Rate Optimisation agency with extensive experience in helping businesses just like yours. Find out how we can help you, here.


Keep up-to-date

People from Facebook, FarFetch and RS Components receive our newsletter. You can too. Subscribe now.

Interested in turning experimentation and testing into an advantage for your entire business?