Lynne Wright, Operations Manager at AWA digital, takes a look back at the Conversion Rate Optimisation (CRO) tools and industry news from 2015.
This month I investigate the two big themes to emerge from the CRO world in 2015 - split testing tools changing the statistical methods they use to report results; and tool providers expanding their services to offer more of a one-stop-shop solution.
Statistics for the internet age
Until now, the statistical models used by split testing tools have made it difficult to peek early at results as this can lead to a false positive reading – torture for us naturally curious marketers operating in a fast-paced data-driven world.
2015 saw leading players Optimizely and Visual Website Optimizer (VWO) both switch from the frequentist t-test method to Bayesian-powered stats engines. The former tells you how often you'll see your current results purely by chance by asking "is this variation different from the control" whilst the later answers the real question “what is the probability of your new web page performing better than your existing web page?”
What does that mean for you? As well as being more confident in the results you read, Bayesian-powered statistics use a sequential testing method to evaluate results as data comes in. This means that you can look at the results any time without being misled and identify a best performer faster.
It’s worth mentioning though that even with the new Bayesian method, statistical significance increases over time, so in some cases, the test will still have to run for a period of time to be sufficiently confident to make a business decision.
This development is set to revolutionise the industry as the new stats engines are designed for the way we work in the CRO industry and give us more control over tests. It means we can get results around 40 per cent faster to help us make quick decisions based on the data that is constantly received and harvested from websites. The new method also allows tests to be paused – for example over the Christmas period – without invalidating the results.
We are partners of both Optimizely and VWO, and are delighted that both have embraced a Bayesian-based stats model, striking the right balance of speed versus accuracy.
I’ll be watching with interest to see if other split testing tools follow suit in 2016.
Moving towards one-stop-shop solutions
When we work on website optimisation projects, we typically have between eight and ten different CRO tools on the go. This results in a lot of time spent switching between applications, so I’m delighted to see a trend towards providers developing all-in-one conversion suites.
It started in April with the launch of HotJar - a new CRO service combining both analysis and feedback tools showing how users interact across multiple devices. The analysis tools include heatmaps, scrollmaps, full visitor session playback, funnel analysis, and form abandonment information and the feedback tool recruits usability testers and allows you to hear what they say using pop-up polls and full surveys.
Although HotJar doesn’t manage A/B split tests, it says that it will partner with the biggest testing platforms to make the process of collecting insight from tests as seamless as possible.
Visual Website Optimizer (VWO) is also expanding its conversion optimisation offering with the launch of on-page surveys and visitor behaviour analysis tools to enable users to run split tests based on solid research data.
VWO’s new suite of tools including heatmaps, visitor recordings and form analysis and its parent company, Wingify, has announced its acquisition of US-based Navilytics, a provider of visitor analytics solutions. This is all part of VWO’s ambition to build the most comprehensive conversion optimisation platform on the market.
We’re looking forward to trying out all the new features soon and comparing them with the standalone tools they are aiming to replace, so look out for updates in the course of 2016.
In addition, Optimizely has launched a new personalisation tool run on the same platform as it split testing function and collaboration tool Trello has also picked up on the trend. Its new Power-Ups development means that the shared space now integrates with a number of our favourite tools such as Dropbox, Evernote and Google Drive.
A prediction from Stewart Rogers, director, Marketing Technology, for VB Insight, says that the next big CRO vendor will be one that “offers A/B/(n) split testing, multivariate testing, heatmapping, scrollmapping, web analytics, funnel analysis, mobile analytics, user testing, expert feedback, and concept testing – across all content channels and within a package as easy to use as Crazy Egg, and priced to compete with Optimizely”.
Perhaps that day is not too far away and we’ll only need to apply one tool to carry out an entire website CRO project… now that would be something to celebrate.